Cotype Nano

🚀 Cotype-Nano

Cotype Nano is a lightweight LLM, designed to perform tasks with minimal resources. It is optimized for fast and efficient interaction with users, providing high performance even under resource-constrained conditions.

🚀 Quick Start

Inference with vLLM

python3 -m vllm.entrypoints.openai.api_server --model MTSAIR/Cotype-Nano --port 8000

Recommended generation parameters and system prompt

import openai
import pandas as pd
from tqdm import tqdm

openai.api_key = 'xxx'

endpoint = 'http://localhost:8000/v1'
model = 'MTSAIR/Cotype-Nano'
openai.api_base = endpoint

# Possible system prompt:
# {"role": "system", "content": "Ты — ИИ-помощник. Тебе дано задание: необходимо сгенерировать подробный и развернутый ответ."},


response = openai.ChatCompletion.create(
    model=model,
    temperature=0.4, # 0.0 is also allowed
    frequency_penalty=0.0,
    max_tokens=2048,
    top_p=0.8, # 0.1 is also allowed
    messages=[
          {"role": "user", "content": "–ö–∞–∫ –º–Ω–µ –æ–±—É—á–∏—Ç—å –º–æ–¥–µ–ª—å meta-llama/Llama-3.2-1B —Å –ø–æ–º–æ—â—å—é –±–∏–±–ª–∏–æ—Ç–µ–∫–∏ transformers?"}
        ]
    )

answer = response["choices"][0]["message"]["content"]
print(answer)

Inference with Huggingface

from transformers import pipeline

pipe = pipeline("text-generation", model="MTSAIR/Cotype-Nano", device="cuda")

messages = [
  {"role": "system", "content": "Ты — ИИ-помощник. Тебе дано задание: необходимо сгенерировать подробный и развернутый ответ."},
  {"role": "user", "content": "–†–∞—Å—Å–∫–∞–∂–∏ –º–Ω–µ –ø—Ä–æ –ò–ò"},
]

res = pipe(messages, max_length=1024)
print(res[0]['generated_text'][-1]['content'])

💻 Usage Examples

Basic Usage

# Inference with vLLM
python3 -m vllm.entrypoints.openai.api_server --model MTSAIR/Cotype-Nano --port 8000

Advanced Usage

# Recommended generation parameters and system prompt
import openai
import pandas as pd
from tqdm import tqdm

openai.api_key = 'xxx'

endpoint = 'http://localhost:8000/v1'
model = 'MTSAIR/Cotype-Nano'
openai.api_base = endpoint

# Possible system prompt:
# {"role": "system", "content": "Ты — ИИ-помощник. Тебе дано задание: необходимо сгенерировать подробный и развернутый ответ."},


response = openai.ChatCompletion.create(
    model=model,
    temperature=0.4, # 0.0 is also allowed
    frequency_penalty=0.0,
    max_tokens=2048,
    top_p=0.8, # 0.1 is also allowed
    messages=[
          {"role": "user", "content": "–ö–∞–∫ –º–Ω–µ –æ–±—É—á–∏—Ç—å –º–æ–¥–µ–ª—å meta-llama/Llama-3.2-1B —Å –ø–æ–º–æ—â—å—é –±–∏–±–ª–∏–æ—Ç–µ–∫–∏ transformers?"}
        ]
    )

answer = response["choices"][0]["message"]["content"]
print(answer)

Example request

Request: –ù–∞–ø–∏—à–∏ –ø—Ä–æ—Å—Ç–æ–π —Å–µ—Ä–≤–µ—Ä –Ω–∞ —Ñ–∞—Å—Ç–∞–ø–∏

Cotype-Nano response:

–§–∞—Å—Ç–∞–ø–∏ (FastAPI) is a framework that simplifies the creation of APIs in Python, allowing you to quickly build high - performance web applications. To use FastAPI, you need to follow these steps:

1. Install FastAPI and Uvicorn.
2. Create an App instance.
3. Define API endpoints.
4. Run the application.

Step 1: Install FastAPI and Uvicorn

To install FastAPI and Uvicorn, you can use the following command:

pip install fastapi uvicorn

Step 2: Create an App instance

Create an App instance using the following code:

from fastapi import FastAPI

app = FastAPI()

Step 3: Define API endpoints

Define an API endpoint, for example, a GET endpoint:

@app.get("/")
async def read_root():
    return {"message": "Hello, FastAPI!"}

Step 4: Run the application

Run the application using Uvicorn:

uvicorn main:app --reload

Here, main is the name of the Python file, and app is the App instance. The --reload option enables auto - reload during development.

After running the application, you can access it at http://127.0.0.1:8000. When you make a GET request to /, you will receive the following response:

{
    "message": "Hello, FastAPI!"
}

If you want to create more complex endpoints, you can define different routes and request methods according to your needs.

🔧 Technical Details

Training process

The model was trained in two stages. In the first stage, MLP layers were trained on mathematics and code. In the second stage, the entire model was trained on internal and open synthetic instructional datasets.

ru-llm-arena: 30.2 (local measurement)

📄 License

The model is under the Apache 2.0 license.