Midm 2.0 Base Instruct Gguf

🚀 Mi:dm 2.0

Mi:dm 2.0 is a "Korea-centric AI" model developed using KT's proprietary technology. It deeply internalizes the unique values, cognitive frameworks, and commonsense reasoning of Korean society, going beyond simple text processing and generation to reflect a profound understanding of Korean socio - cultural norms and values.

✨ Features

• Two Model Versions:
  • Mi:dm 2.0 Base: An 11.5B parameter dense model that balances size and performance. It extends an 8B - scale model using the Depth - up Scaling (DuS) method, suitable for real - world applications requiring both performance and versatility.
  • Mi:dm 2.0 Mini: A lightweight 2.3B parameter dense model optimized for on - device environments and systems with limited GPU resources. It is derived from the Base model through pruning and distillation for compact deployment.
• No KT User Data: Neither the pre - training nor the post - training data includes KT users' data.

🚀 Quick Start

Here is the code snippet to run conversational inference with the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

model_name = "K - intelligence/Midm - 2.0 - Base - Instruct"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
generation_config = GenerationConfig.from_pretrained(model_name)

prompt = "KT에 대해 소개해줘"

# message for inference
messages = [
    {"role": "system", 
     "content": "Mi:dm(믿:음)은 KT에서 개발한 AI 기반 어시스턴트이다."},
    {"role": "user", "content": prompt}
]

input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
)

output = model.generate(
    input_ids.to("cuda"),
    generation_config=generation_config,
    eos_token_id=tokenizer.eos_token_id,
    max_new_tokens=128,
    do_sample=False,
)
print(tokenizer.decode(output[0]))

⚠️ Important Note

The transformers library should be version 4.45.0 or higher.

📦 Installation

Not explicitly provided in the original README, so this section is skipped.

💻 Usage Examples

Run on Friendli.AI

You can try our model immediately via Friendli.AI. Simply click Deploy and then Friendli Endpoints.

⚠️ Important Note

Please note that a login to Friendli.AI is required after your fifth chat interaction.

Run on Your Local Machine

We provide a detailed description about running Mi:dm 2.0 on your local machine using llama.cpp, LM Studio, and Ollama. Please check our [github](https://github.com/K - intelligence - Midm/Midm - 2.0) for more information

Deployment

To serve Mi:dm 2.0 using [vLLM](https://github.com/vllm - project/vllm)(>=0.8.0) with an OpenAI - compatible API:

vllm serve K - intelligence/Midm - 2.0 - Base - Instruct

Tutorials

To help our end - users easily use Mi:dm 2.0, we have provided comprehensive tutorials on [github](https://github.com/K - intelligence - Midm/Midm - 2.0).

📚 Documentation

Evaluation

Korean

* indicates KT proprietary evaluation resources.

English

🔧 Technical Details

Not explicitly provided in the original README, so this section is skipped.

📄 License

Mi:dm 2.0 is licensed under the MIT License.

Limitation

• The training data for both Mi:dm 2.0 models consists primarily of English and Korean. Understanding and generation in other languages are not guaranteed.
• The model is not guaranteed to provide reliable advice in fields that require professional expertise, such as law, medicine, or finance.
• Researchers have made efforts to exclude unethical content from the training data — such as profanity, slurs, bias, and discriminatory language. However, despite these efforts, the model may still produce inappropriate expressions or factual inaccuracies.

Contact

Mi:dm 2.0 Technical Inquiries: midm - llm@kt.com