Kodify Nano GGUF

🚀 Kodify-Nano-GGUF 🤖

Kodify-Nano-GGUF is the GGUF version of the MTSAIR/Kodify-Nano model. It is optimized for CPU/GPU inference with Ollama/llama.cpp. This lightweight LLM is designed for code development tasks with minimal resource requirements.

🚀 Quick Start

You can run Kodify Nano on OLLAMA in two ways:

1. Using Docker
2. Locally (provides faster responses than Docker)

✨ Features

• Supports both Docker and local deployment on OLLAMA.
• Offers multiple quantization variants for different quality and performance needs.
• Can be integrated with Python through the Ollama Python library.
• Has plugins available for Visual Studio Code and JetBrains IDEs.

📦 Installation

Method 1: Running Kodify Nano on OLLAMA in Docker

Without NVIDIA GPU:

docker run -e OLLAMA_HOST=0.0.0.0:8985 -p 8985:8985 --name ollama -d ollama/ollama

With NVIDIA GPU:

docker run --runtime nvidia -e OLLAMA_HOST=0.0.0.0:8985 -p 8985:8985 --name ollama -d ollama/ollama

⚠️ Important Note

• Ensure Docker is installed and running.
• If port 8985 is occupied, replace it with any available port and update plugin configuration.

Load the model:

docker exec ollama ollama pull hf.co/MTSAIR/Kodify-Nano-GGUF

Rename the model:

docker exec ollama ollama cp hf.co/MTSAIR/Kodify-Nano-GGUF kodify_nano

Start the model:

docker exec ollama ollama run kodify_nano

Method 2: Local Kodify Nano on OLLAMA

1. Download OLLAMA:
   https://ollama.com/download

2. Set the port:

export OLLAMA_HOST=0.0.0.0:8985

⚠️ Important Note

If port 8985 is occupied, replace it and update plugin configuration.

3. Start OLLAMA server:

ollama serve &

4. Download the model:

ollama pull hf.co/MTSAIR/Kodify-Nano-GGUF

5. Rename the model:

ollama cp hf.co/MTSAIR/Kodify-Nano-GGUF kodify_nano

6. Run the model:

ollama run kodify_nano

Plugin Installation

For Visual Studio Code

1. Download the latest Kodify plugin for VS Code.
2. Open the Extensions panel on the left sidebar.
3. Click Install from VSIX... and select the downloaded plugin file.

For JetBrains IDEs

1. Download the latest Kodify plugin for JetBrains.
2. Open the IDE and go to Settings > Plugins.
3. Click the gear icon (⚙️) and select Install Plugin from Disk....
4. Choose the downloaded plugin file.
5. Restart the IDE when prompted.

Changing the Port in Plugin Settings (for Visual Studio Code and JetBrains)

If you changed the Docker port from 8985, update the plugin's config.json:

1. Open any file in the IDE.
2. Open the Kodify sidebar:
   • VS Code: Ctrl+L (Cmd+L on Mac).
   • JetBrains: Ctrl+J (Cmd+J on Mac).
3. Access the config.json file:
   • Method 1: Click Open Settings (VS Code) or Kodify Config (JetBrains), then navigate to Configuration > Chat Settings > Open Config File.
   • Method 2: Click the gear icon (⚙️) in the Kodify sidebar.
4. Modify the apiBase port under tabAutocompleteModel and models.
5. Save the file (Ctrl+S or File > Save).

💻 Usage Examples

Basic Usage

import ollama

response = ollama.generate(
    model="kodify-nano",
    prompt="Write a Python function to calculate factorial",
    options={
        "temperature": 0.4,
        "top_p": 0.8,
        "num_ctx": 8192
    }
)

print(response['response'])

Advanced Usage

response = ollama.generate(
    model="kodify-nano",
    prompt="""<s>[INST] 
Write a Python function that:
1. Accepts a list of numbers
2. Returns the median value
[/INST]""",
    options={"max_tokens": 512}
)

### Code Refactoring
response = ollama.generate(
    model="kodify-nano",
    prompt="""<s>[INST] 
Refactor this Python code:

def calc(a,b):
    s = a + b
    d = a - b
    p = a * b
    return s, d, p
[/INST]""",
    options={"temperature": 0.3}
)

🔧 Technical Details

Available quantization variants:

• Kodify_Nano_q4_k_s.gguf (balanced)
• Kodify_Nano_q8_0.gguf (high quality)
• Kodify_Nano.gguf (best quality, unquantized)

Download using huggingface_hub:

pip install huggingface-hub
python -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='MTSAIR/Kodify-Nano-GGUF', filename='Kodify_Nano_q4_k_s.gguf', local_dir='./models')"

Python Integration

Install Ollama Python library:

pip install ollama

📄 License

This project is licensed under the Apache-2.0 license.