Mistral Portuguese Luana 7b Chat

🚀 Mistral-portuguese-luana-7b-chat

This model, fine - tuned from Mistral 7B, is trained on a superset of 250,000 Portuguese chats to address the shortage of Portuguese language models, especially optimized for chat scenarios.

🚀 Quick Start

This model was trained with a superset of 250,000 chat in Portuguese. The model aims to fill the gap in Portuguese language models. Tuned from the Mistral 7B, it is mainly adjusted for chat.

✨ Features

• Trained on a large Portuguese chat dataset.
• Suitable for various chat scenarios.
• Supports different quantization levels for better memory management.

📦 Installation

FULL MODEL : A100

HALF MODEL: L4

8bit or 4bit : T4 or V100

You can use the model in its normal form up to 4 - bit quantization. Below are the installation steps:

!pip install -q -U transformers
!pip install -q -U accelerate
!pip install -q -U bitsandbytes

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model = AutoModelForCausalLM.from_pretrained("rhaymison/Mistral-portuguese-luana-7b-chat", device_map= {"": 0})
tokenizer = AutoTokenizer.from_pretrained("rhaymison/Mistral-portuguese-luana-7b-chat")
model.eval()

💻 Usage Examples

Basic Usage

inputs = tokenizer([f"""<s>[INST] Estou pensando em levar minha esposa e meu gatinha para fazer algo diferente. Você teria alguma coisa em mente ?.
[/INST]"""], return_tensors="pt")

inputs.to(model.device)

streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
_ = model.generate(**inputs, streamer=streamer, max_new_tokens=128)

Advanced Usage

If you are having a memory problem such as "CUDA Out of memory", you should use 4 - bit or 8 - bit quantization.

4bits example

from transformers import BitsAndBytesConfig
import torch
nb_4bit_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True
)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map={"": 0}
)

LangChain Example

📚 Documentation

Open Portuguese LLM Leaderboard Evaluation Results

Detailed results can be found here

📄 License

This model is licensed under the Apache - 2.0 license.

Comments

Any idea, help or report will always be welcome.

Email: rhaymisoncristian@gmail.com