Meta Llama 3.1 8B Text To SQL

🚀 Meta LLaMA 3.1 8B 4位微調模型

本模型是 Meta-Llama-3.1-8B 的微調版本，由 ruslanmv 開發，用於文本生成任務。它採用了4位量化技術，在自然語言生成中保持強大性能的同時，讓推理過程更加高效。

🚀 快速開始

本模型是為文本生成任務對 Meta-Llama-3.1-8B 進行微調後的版本。它藉助4位量化技術，在保持自然語言生成強大性能的同時，提高了推理效率。

✨ 主要特性

• 文本生成：該模型經過微調，能夠根據輸入生成連貫且上下文準確的文本。

📦 安裝指南

要使用此模型，你需要安裝必要的庫：

pip install transformers accelerate 

💻 使用示例

基礎用法

以下是使用Hugging Face的 transformers 庫加載此微調模型的示例：

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
model_name = "ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL"

# Ensure you have the right device setup
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the model and tokenizer from the Hugging Face Hub
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Initialize the tokenizer (adjust the model name as needed)
# Define EOS token for terminating the sequences
EOS_TOKEN = tokenizer.eos_token

# Define Alpaca-style prompt template
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
"""

# Format the prompt without the response part
prompt = alpaca_prompt.format(
    "Provide the SQL query",
    "Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"
)
# Tokenize the prompt and generate text
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)

# Decode the generated text
generated_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

# Extract the generated response only (remove the prompt part)
response_start = generated_text.find("### Response:") + len("### Response:\n")
response = generated_text[response_start:].strip()

# Print the response (excluding the prompt)
print(response)

答案是：

SELECT * FROM table1 WHERE anni = 2020

📚 詳細文檔

模型詳情

許可證

此模型遵循 Apache 2.0許可證。只要你遵守許可證條款，就可以自由使用、修改和分發此模型。

致謝

此模型由 ruslanmv 基於 unsloth 的原始工作和 meta-llama-3.1-8b-bnb-4bit 模型進行微調。