Cantonesellmchat V1.0 7B

🚀 CantoneseLLMChat-v1.0-7B

The first generation Cantonese LLM from hon9kon9ize, excelling in Hong Kong related specific knowledge and Cantonese conversation.

Cantonese LLM Chat v1.0 is the first generation Cantonese LLM from hon9kon9ize. Building upon the success of v0.5 preview, the model excels in Hong Kong related specific knowledge and Cantonese conversation.

✨ Features

• Specific Knowledge: Specialized in Hong Kong related knowledge.
• Language Proficiency: Skilled in Cantonese conversations.

📚 Documentation

Model description

The base model is obtained via Continuous Pre-Training of Qwen 2.5 7B with 600 million publicly available Hong Kong news articles and Cantonese websites. The instructions fine-tuned model is trained with a dataset consisting of 75,000 instruction pairs. 45,000 pairs are Cantonese instructions generated by other LLMs and reviewed by humans.

The model is trained with 1 Nvidia H100 80GB HBM3 GPU on Genkai Supercomputer.

Model Information

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "hon9kon9ize/CantoneseLLMChat-v1.0-7B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16, 
    device_map="auto", 
)

def chat(messages, temperature=0.9, max_new_tokens=200):
    input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt').to('cuda:0')
    output_ids = model.generate(input_ids, max_new_tokens=max_new_tokens, temperature=temperature)
    response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=False)
    return response

prompt = "邊個係香港特首？"

messages = [
    {"role": "system", "content": "you are a helpful assistant."},
    {"role": "user", "content": prompt}
]

print(chat(messages)) # 香港特別行政區行政長官係李家超。<|im_end|>

📈 Performance

This model is the best in class open source LLM in understanding Cantonese and Hong Kong culture in the HK-Eval Benchmark. However, as one could observe, reasoning models have performed dramatically better than their counterparts. We are currently working on reasoning models for v2.

📄 License

The license for this model is other.