Goppa LogiLlama

🚀 LogiLlama

LogiLlama is a fine - tuned language model developed by Goppa AI. It is built on a 1B - parameter base from LLaMA, enhanced with injected knowledge and logical reasoning abilities, aiming to make smaller models smarter.

🚀 Quick Start

LogiLlama is a fine - tuned language model developed by Goppa AI. Built upon a 1B - parameter base from LLaMA, it has been enhanced with injected knowledge and logical reasoning abilities. The goal is to make smaller models smarter, delivering improved reasoning and problem - solving capabilities while maintaining a low memory footprint and energy efficiency for on - device applications.

✨ Features

While recent trends in language models have leaned towards scaling up parameters, LogiLlama demonstrates that “less can be more.” By fine - tuning a 1B parameter base model with advanced logical reasoning techniques, LogiLlama offers:

• Enhanced Reasoning: Improved logical thinking and knowledge integration for more accurate and context - aware responses.
• Efficiency: Designed for on - device processing with a low memory and energy footprint.
• Transparency: The training process and configuration files are fully open - source, reflecting the commitment to transparent and reproducible research.

LogiLlama is the first step in Goppa AI's journey to develop efficient, intelligent, and resource - friendly models that challenge the notion that bigger is always better.

📚 Documentation

Model Description

Configuration Files

The model repository includes several key configuration files that ensure LogiLlama functions seamlessly within the Hugging Face ecosystem:

• config.json: Contains the model architecture settings, including hidden size, number of layers, attention heads, and other hyperparameters.
• generation_config.json: Defines generation parameters such as temperature, top - p sampling, and end - of - sequence tokens.
• special_tokens_map.json: Maps special tokens (e.g., beginning - of - text, end - of - text, padding) used during tokenization.
• tokenizer_config.json: Provides metadata and settings for the tokenizer, ensuring consistency with the model’s vocabulary and special tokens.

Training Details

LogiLlama was fine - tuned by injecting logical reasoning and domain - specific knowledge into a 1B parameter LLaMA base. By carefully curating training data and employing specialized techniques, the model's capability to handle reasoning tasks was enhanced without significantly increasing its size. This project marks the commitment to advancing small, efficient models that do not compromise on performance.

Inference & Deployment

• Model Size: 1B parameters
• Tensor Type: float32 (F32)
• Deployment: Optimized for on - device inference and resource - constrained environments. Currently available for local deployment; stay tuned for updates on hosted inference solutions.

💻 Usage Examples

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load tokenizer and model from our repository
tokenizer = AutoTokenizer.from_pretrained("goppa-ai/Goppa-LogiLlama", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("goppa-ai/Goppa-LogiLlama", trust_remote_code=True)

model.to('cuda')
text = "When faced with a complex problem, what should one do?"
input_ids = tokenizer(text, return_tensors="pt").to('cuda').input_ids
outputs = model.generate(
    input_ids,
    max_length=2500,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.2,
    pad_token_id=tokenizer.eos_token_id
)
print(tokenizer.batch_decode(outputs[:, input_ids.shape[1]:-1])[0].strip())

📄 License

LogiLlama is released under the MIT License.

📚 Citation

If you use LogiLlama in your research, please cite:

@misc{goppa2025logillama,
      title={LogiLlama: Injecting Logical Reasoning into Small Language Models}, 
      author={Goppa AI},
      year={2025},
      note={https://github.com/GoppaAI/LogiLlama}
}