Kraken Multilingual

🚀 Kraken-Multilingual

Kraken-Multilingual is a sophisticated machine learning model jointly developed by Cognitive Computations, VAGO Solutions, and Hyperspace.ai. It supports multiple languages and offers dynamic text generation capabilities.

🚀 Quick Start

The Kraken-Multilingual model and Architecture Kraken is a joint effort between Cognitive Computations, VAGO Solutions and Hyperspace.ai. It's created by Fernando Fernandes Neto, David Golchinfar, Lucas Atkins and Eric Hartford. This model supports German, English, Italian, French, Swedish, Finnish, Danish and Norwegian languages.

The Kraken Architecture is a machine learning framework for dynamic text generation. It uses the Hugging Face transformers library to manage multiple causal language models (CLMs) and routes input through different models based on context. It's powered by a custom configuration class (KrakenConfig) for integrating and managing components like tokenizers, models, and routing mechanisms.

✨ Features

• Dynamic Model Routing: Uses a sequence classification model to route inputs to the most suitable language model according to input characteristics.
• Multiple Language Models: Supports the integration of various pre - trained causal language models, enabling flexible and context - appropriate responses.
• Customizable Templates: Supports input formatting with predefined templates, enhancing the model's adaptability to different conversational contexts.
• Extensible Configuration: Leverages a custom configuration setup that can be easily extended and adapted for various causal language modeling use cases.

📦 Installation

Selected Models as Experts:

      "German/English Expert": "VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct",
      "Function Italian Expert": "mii-community/zefiro-7b-dpo-ITA",
      "French Expert": "paulml/Hermes-2-Pro-French",
      "Scandinavian Expert": "norallm/normistral-7b-warm-instruct",

How to load and call Kraken - Multilingual model:

from transformers import AutoModelForCausalLM
device = "cuda:0" ## Setup "cuda:0" if NVIDIA, "mps" if on Mac

# Load the model and config:
model = AutoModelForCausalLM.from_pretrained("./kraken_model", trust_remote_code=True)

💻 Usage Examples

Basic Usage

Call the German expert:

messages = [
    {'role': 'system', 'content': 'Du bist ein freundlicher und hilfreicher deutscher KI-Assistent'},
    {'role': 'user', 'content': "Erzähle mir eine kurze Gute Nacht Geschichte in 2 Sätzen."}
    ]

tokenizer = model.tokenizer
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda:0")
output_ids = model.generate(input_ids, max_length=150)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Call the English expert:

messages = [
    {'role': 'system', 'content': '"You are a helpful AI Assistant'},
    {'role': 'user', 'content': "Find the mass percentage of Ba in BaO"}
    ]

tokenizer = model.tokenizer
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(device)
output_ids = model.generate(input_ids, max_length=250)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Call the Italian expert:

messages = [
    {'role': 'system', 'content': 'Sei un utile assistente AI.'},
    {'role': 'user', 'content': 'Hai qualche idea su cosa potrei fare a Roma?'}
    ]

tokenizer = model.tokenizer
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(device)
output_ids = model.generate(input_ids ,temperature=0.6, do_sample=True, top_p=0.9,top_k=20, max_length=500)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Call the French expert:

messages = [
    {'role': 'system', 'content': 'Vous êtes un assistant IA allemand sympathique et serviable'},
    {'role': 'user', 'content': 'J\'aimerais faire du shopping à Paris. Que pouvez-vous recommander?'}
    ]

tokenizer = model.tokenizer
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
print(input_text)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(device)
output_ids = model.generate(input_ids ,temperature=0.6, do_sample=True, top_p=0.9,top_k=20, max_length=250)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Call the Scandinavian expert:

messages = [
    {'role': 'system', 'content': 'Du är en hjälpsam AI-assistent'},
    {'role': 'user', 'content': 'Jag kommer från Tyskland och skulle vilja resa till Sverige. Är en färja över Danmark ett bra sätt att resa?'}
    ]

tokenizer = model.tokenizer
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(device)
output_ids = model.generate(input_ids ,temperature=0.1, do_sample=True, top_p=0.9,top_k=20, max_length=250)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Advanced Usage

Switch expert and or quantization:

Go into the config file of the kraken_model folder

    "models": {
      "expert1": "VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct", # Switch to a german/english model of your choice
      "expert2": "mii-community/zefiro-7b-dpo-ITA",                # Switch to a italian model of your choice
      "expert3": "paulml/Hermes-2-Pro-French",                     # Switch to a french model of your choice
      "expert4": "norallm/normistral-7b-warm-instruct"             # Switch to a scandinavian model of your choice
    },
    # Currently supported: "4bit","8bit" and "awq"
    "quantization": {
      "expert1": null,
      "expert2": null,
      "expert3": null,
      "expert4": null
    },
    "router": "kraken_router",
    # Adjust the tokenizer to your selected model
    "tokenizers": {
      "expert1": "VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct",
      "expert2": "mii-community/zefiro-7b-dpo-ITA",
      "expert3": "paulml/Hermes-2-Pro-French",
      "expert4": "norallm/normistral-7b-warm-instruct"
    }
  },
  "model_type": "kraken",
  "torch_dtype": "float32",
  "transformers_version": "4.41.0"
}

📚 Documentation

Disclaimer

⚠️ Important Note

Despite our best efforts in data cleansing, the possibility of uncensored content slipping through cannot be entirely ruled out. We cannot guarantee consistently appropriate behavior. If you encounter any issues or inappropriate content, please inform us through the provided contact information. Also, the licensing of these models does not constitute legal advice, and we are not responsible for the actions of third - party users.

Contact

💡 Usage Tip

If you're interested in customized LLMs for business applications, contact us via our websites. We appreciate your feedback and suggestions.

Collaborations

We're seeking support and investment for our startups, VAGO solutions and Hyperspace. We're continuously advancing the development of robust language models for diverse purposes. If you're excited about collaborative challenges, reach out to us at VAGO solutions, Hyperspace.computer and Cognitive Computations.

Cite As

Fernando Fernandes Neto, David Golchinfar, Lucas Atkins, Eric Hartford - Kraken: An OpenSource Collection of Experts Model, 2024