mistral_tv-neural-marconroni Open-source Multilingual Large Model - Specializing in Text Generation, Free Deployment and Highly Practical

Mistral Tv Neural Marconroni

Developed by aqweteddy

A multilingual large language model based on Mistral 7B, incorporating chat vector technology, focused on text generation tasks

Large Language Model

Transformers

EnglishOpen Source License:MIT #Multilingual Chat Vector #Instruction Following Optimization #Few-shot Efficient Learning

Downloads 94

Release Time : 12/29/2023

Model Overview

This model combines the Mistral 7B base model with chat vector technology to achieve efficient text generation capabilities in multilingual environments, with special emphasis on alignment with human preferences.

Model Features

Chat Vector Technology

Utilizes innovative chat vector methods to effectively combine base model knowledge with conversational behaviors

Multilingual Support

Validated for adaptability in languages including Traditional Chinese, Korean, and Simplified Chinese

Efficient Alignment

Achieves efficient alignment with human preferences through streamlined training processes

Model Capabilities

Text Generation

Multi-turn Dialogue

Instruction Following

Reasoning Ability

Question Answering System

Use Cases

Dialogue Systems

Multilingual Chatbot

Build intelligent dialogue systems supporting multiple languages

Excellent performance in toxicity control, instruction following, and multi-turn dialogue

Educational Assistance

Knowledge Q&A

Knowledge question-answering systems for educational fields

Achieved 69.2% accuracy on the AI2 Reasoning Challenge

🚀 Mistral TV - Neural Marconroni

This project combines Mistral 7B with chat vectors to equip LLMs with new language chat capabilities, achieving high performance in multiple language tasks.

🚀 Quick Start

This README provides an overview of the Mistral TV - Neural Marconroni project, including its core concepts, evaluation results, and related research.

✨ Features

Chat Vector

New Approach for LLMs: CHAT VECTOR offers a simple yet effective way to endow Large Language Models (LLMs) with new language chat capabilities. It focuses on developing LLMs for non - English languages and aligning them with human preferences.
Computationally Efficient: By leveraging the “chat vector,” it restructures the traditional training paradigm from continual pretrain + SFT + RLHF to continual pretrain + chat.
Multilingual Adaptability: Empirical studies are conducted on Traditional Chinese, Korean, and Simplified Chinese, demonstrating the versatility of the approach.

Merged LM

Combination of Models: The project merges Mistral 7B with chat vectors, including neural - chat and marconroni, to enhance the model's performance.

📚 Documentation

Chat Vector Research

The research paper "Chat Vector: A SIMPLE APPROACH TO EQUIP LLMS WITH NEW LANGUAGE CHAT CAPABILITIES" details the methodology and experiments. With the development of conversational AI like ChatGPT, this paper explores developing LLMs for non - English languages. It introduces a method using “chat vector” to combine pre - existing knowledge and behaviors in LLMs. Empirical studies mainly focus on Traditional Chinese, using LLaMA2 as the base model. The chat vector is obtained by subtracting the pre - trained weights of LLaMA2 from the weights of LLaMA2 - chat. Evaluations from toxicity, instruction - following ability, and multi - turn dialogue show the chat vector's superiority in “chatting”. Experiments are also extended to Korean and Simplified Chinese pre - trained models, confirming the adaptability of the approach.

Merged LM Composition

The Merged LM consists of Mistral 7b and chat vectors, specifically neural - chat and marconroni.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Property	Details
Model Type	Mistral TV - Neural Marconroni
Evaluation Metrics
	Avg.
	AI2 Reasoning Challenge (25 - Shot)
	HellaSwag (10 - Shot)
	MMLU (5 - Shot)
	TruthfulQA (0 - shot)
	Winogrande (5 - shot)
	GSM8k (5 - shot)

📄 License

This project is released under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご