Meta Llama 3.1 8B Text To SQL

🚀 Meta LLaMA 3.1 8B 4位微调模型

本模型是 Meta-Llama-3.1-8B 的微调版本，由 ruslanmv 开发，用于文本生成任务。它采用了4位量化技术，在自然语言生成中保持强大性能的同时，让推理过程更加高效。

🚀 快速开始

本模型是为文本生成任务对 Meta-Llama-3.1-8B 进行微调后的版本。它借助4位量化技术，在保持自然语言生成强大性能的同时，提高了推理效率。

✨ 主要特性

• 文本生成：该模型经过微调，能够根据输入生成连贯且上下文准确的文本。

📦 安装指南

要使用此模型，你需要安装必要的库：

pip install transformers accelerate 

💻 使用示例

基础用法

以下是使用Hugging Face的 transformers 库加载此微调模型的示例：

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
model_name = "ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL"

# Ensure you have the right device setup
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the model and tokenizer from the Hugging Face Hub
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Initialize the tokenizer (adjust the model name as needed)
# Define EOS token for terminating the sequences
EOS_TOKEN = tokenizer.eos_token

# Define Alpaca-style prompt template
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
"""

# Format the prompt without the response part
prompt = alpaca_prompt.format(
    "Provide the SQL query",
    "Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"
)
# Tokenize the prompt and generate text
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)

# Decode the generated text
generated_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

# Extract the generated response only (remove the prompt part)
response_start = generated_text.find("### Response:") + len("### Response:\n")
response = generated_text[response_start:].strip()

# Print the response (excluding the prompt)
print(response)

答案是：

SELECT * FROM table1 WHERE anni = 2020

📚 详细文档

模型详情

许可证

此模型遵循 Apache 2.0许可证。只要你遵守许可证条款，就可以自由使用、修改和分发此模型。

致谢

此模型由 ruslanmv 基于 unsloth 的原始工作和 meta-llama-3.1-8b-bnb-4bit 模型进行微调。