Apollo2 7B GGUF

🚀 QuantFactory/Apollo2-7B-GGUF

This is a quantized version of FreedomIntelligence/Apollo2-7B created using llama.cpp. It aims to democratize medical LLMs for a wide range of languages.

📚 Documentation

Original Model Card

Democratizing Medical LLMs For Much More Languages

It covers 12 major languages including English, Chinese, French, Hindi, Spanish, Arabic, Russian, Japanese, Korean, German, Italian, Portuguese and 38 minor languages so far.

Paper • Demo • ApolloMoEDataset • ApolloMoEBench • Models • Apollo • ApolloMoE

⏱️ Update

• [2024.10.15] ApolloMoE repo is published.

🌐 Languages Coverage

It covers 12 major languages and 38 minor languages.

🏗️ Architecture

📊 Results

Dense

• Apollo2-0.5B
• Apollo2-1.5B
• Apollo2-2B
• Apollo2-3.8B
• Apollo2-7B
• Apollo2-9B

Post-MoE

• Apollo-MoE-0.5B
• Apollo-MoE-1.5B
• Apollo-MoE-7B

📝 Usage Format

Apollo2

• 0.5B, 1.5B, 7B: User:{query}\nAssistant:{response}<|endoftext|>
• 2B, 9B: User:{query}\nAssistant:{response}<eos>
• 3.8B: <|user|>\n{query}<|end|><|assisitant|>\n{response}<|end|>

Apollo-MoE

• 0.5B, 1.5B, 7B: User:{query}\nAssistant:{response}<|endoftext|>

📦 Dataset & Evaluation

Dataset

• ApolloMoEDataset

Evaluation

• ApolloMoEBench

📥 Model Download and Inference

We take Apollo-MoE-0.5B as an example:

1. Login Huggingface

huggingface-cli login --token $HUGGINGFACE_TOKEN

2. Download model to local dir

from huggingface_hub import snapshot_download
import os

local_model_dir=os.path.join('/path/to/models/dir','Apollo-MoE-0.5B')
snapshot_download(repo_id="FreedomIntelligence/Apollo-MoE-0.5B", local_dir=local_model_dir)

3. Inference Example

from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
import os

local_model_dir=os.path.join('/path/to/models/dir','Apollo-MoE-0.5B')

model=AutoModelForCausalLM.from_pretrained(local_model_dir,trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(local_model_dir,trust_remote_code=True)
generation_config = GenerationConfig.from_pretrained(local_model_dir, pad_token_id=tokenizer.pad_token_id, num_return_sequences=1, max_new_tokens=7, min_new_tokens=2, do_sample=False, temperature=1.0, top_k=50, top_p=1.0)

inputs = tokenizer('Answer direclty.\nThe capital of Mongolia is Ulaanbaatar.\nThe capital of Iceland is Reykjavik.\nThe capital of Australia is', return_tensors='pt')
inputs = inputs.to(model.device)
pred = model.generate(**inputs,generation_config=generation_config)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))

📈 Results reproduction

📄 License

The model is released under the Apache 2.0 license.

📖 Citation

Please use the following citation if you intend to use our dataset for training or evaluation:

@misc{zheng2024efficientlydemocratizingmedicalllms,
    title={Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts}, 
    author={Guorui Zheng and Xidong Wang and Juhao Liang and Nuo Chen and Yuping Zheng and Benyou Wang},
    year={2024},
    eprint={2410.10626},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2410.10626}, 
}