Meta Llama 3.1 8B Text To SQL

🚀 Meta LLaMA 3.1 8B 4-bit Finetuned Model

This model is a fine - tuned version of Meta - Llama - 3.1 - 8B, designed for text generation tasks. It uses 4 - bit quantization, enhancing inference efficiency while maintaining high - quality natural language generation performance.

🚀 Quick Start

This fine - tuned model of Meta - Llama - 3.1 - 8B is developed by ruslanmv for text generation. It utilizes 4 - bit quantization, which makes it more efficient for inference and still performs well in natural language generation.

✨ Features

• Text Generation: This model is fine - tuned to generate coherent and contextually accurate text based on the provided input.

📦 Installation

To use this model, you will need to install the necessary libraries:

pip install transformers accelerate 

💻 Usage Examples

Basic Usage

Here’s an example of how to load this fine - tuned model using Hugging Face's transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
model_name = "ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL"

# Ensure you have the right device setup
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the model and tokenizer from the Hugging Face Hub
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Initialize the tokenizer (adjust the model name as needed)
# Define EOS token for terminating the sequences
EOS_TOKEN = tokenizer.eos_token

# Define Alpaca-style prompt template
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
"""

# Format the prompt without the response part
prompt = alpaca_prompt.format(
    "Provide the SQL query",
    "Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"
)
# Tokenize the prompt and generate text
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)

# Decode the generated text
generated_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

# Extract the generated response only (remove the prompt part)
response_start = generated_text.find("### Response:") + len("### Response:\n")
response = generated_text[response_start:].strip()

# Print the response (excluding the prompt)
print(response)

and the answer is

SELECT * FROM table1 WHERE anni = 2020

📚 Documentation

Model Details

License

This model is licensed under the Apache 2.0 License. You are free to use, modify, and distribute this model, provided that you comply with the license terms.

Acknowledgments

This model was fine - tuned by ruslanmv based on the original work of unsloth and the meta - llama - 3.1 - 8b - bnb - 4bit model.