Mellum 4b Sft Rust GGUF

🚀 Etherll/Mellum-4b-sft-rust

Etherll/Mellum-4b-sft-rust is a large language model (LLM) fine-tuned for Rust code Fill-in-the-Middle (FIM) tasks, built on the JetBrains/Mellum-4b-base model.

🚀 Quick Start

Etherll/Mellum-4b-sft-rust is a specialized large language model (LLM) crafted for Rust code Fill-in-the-Middle (FIM) tasks. It's based on the JetBrains/Mellum-4b-base model.

The model has undergone fine - tuning on the Etherll/CodeFIM-Rust-Mellum dataset, which consists of around 57,000 Rust - specific FIM examples. This fine - tuning enhances its ability to accurately and contextually complete Rust code snippets.

There's also a GGUF version available for CPU inference: Etherll/Mellum-4b-sft-rust-GGUF.

✨ Features

• Specialized for Rust: Optimized for Fill - in - the - Middle tasks in Rust.
• Based on Mellum-4b-base: Benefits from JetBrains' robust base model.
• Efficient: Suitable for both cloud and local deployment.
• IDE Integration Ready: Designed for use in developer tooling, and works great with Continue.dev for an enhanced coding assistant experience.

📚 Documentation

Model Description

This model utilizes the LLaMA - style architecture of Mellum-4b-base (4 billion parameters) and its extensive pre - training on over 4 trillion tokens. The fine - tuning process was centered around adapting the model to the subtleties of Rust syntax and common coding patterns for FIM tasks.

Fine - tuning Data

FIM Format

This model is trained to recognize a specific format for Fill - in - the - Middle tasks. When providing input for FIM, use the following structure:

<filename>{{{filename}}}
<fim_suffix>{{{suffix_code}}}<fim_prefix>{{{prefix_code}}}<fim_middle>

💻 Usage Examples

With Continue.dev

For the best integrated development experience, it's highly recommended to use this model with Continue.dev.

Refer to the Continue.dev documentation for instructions on how to add custom LLMs.

GGUF Version

A GGUF version is available at Etherll/Mellum-4b-sft-rust-GGUF. This format is suitable for local inference on CPU (and GPU with appropriate llama.cpp/Ollama builds) using tools like:

• llama.cpp
• Ollama
• LM Studio

📄 License

This model is released under the apache - 2.0 license.

Support & Community

If you need any help, have questions, or just want to chat, feel free to message me on Discord: etherl