Chronos 13b 4bit

🚀 chronos-13b-4bit

A 4-bit (int4) quantized version of the chronos-13b model, optimized for chat, roleplay, and storywriting.

This is a 4-bit (int4) quantized version using true-sequential and groupsize 128 of https://huggingface.co/elinas/chronos-13b. This model is mainly focused on chat, roleplay, and storywriting. Additionally, it can handle other tasks like simple reasoning and coding. Due to the human inputs in its training data, Chronos can generate long and coherent text outputs.

This model follows the Alpaca formatting. For optimal performance, use the following format:

### Instruction:
Your instruction or question here.
### Response:

GGML Version provided by @TheBloke

📚 Documentation

Model details

Intended use

Primary intended uses The main use of LLaMA is for research on large language models, including:

• Exploring potential applications such as question answering, natural language understanding, or reading comprehension.
• Understanding the capabilities and limitations of current language models and developing techniques to improve them.
• Evaluating and mitigating biases, risks, toxic and harmful content generations, and hallucinations.

Primary intended users The primary users of the model are researchers in natural language processing, machine learning, and artificial intelligence.

Out-of-scope use cases LLaMA is a base or foundational model. It should not be used in downstream applications without further risk evaluation and mitigation. In particular, this model has not been trained with human feedback and may generate toxic, offensive content, incorrect information, or generally unhelpful answers.

Factors

Relevant factors One of the most significant factors affecting model performance is the language used. Although the training data includes 20 languages, most of the dataset consists of English text. Therefore, the model is expected to perform better in English than in other languages. Previous studies have also shown that performance may vary across different dialects, and the same is expected for this model.

Evaluation factors Since the model is trained on web data, it is expected to reflect biases from this source. The model was evaluated on RAI datasets to measure biases related to gender, religion, race, sexual orientation, age, nationality, disability, physical appearance, and socio-economic status. The toxicity of model generations was also measured based on the toxicity of the prompting context.

Metrics

Model performance measures The following metrics were used to evaluate the model:

• Accuracy for common sense reasoning, reading comprehension, natural language understanding (MMLU), BIG-bench hard, WinoGender, and CrowS-Pairs.
• Exact match for question answering.
• The toxicity score from Perspective API on RealToxicityPrompts.

Decision thresholds Not applicable.

Approaches to uncertainty and variability Due to the high computational requirements of training large language models, only one model of each size was trained. Therefore, the variability of pre-training could not be evaluated.

Evaluation datasets

The model was evaluated on the following benchmarks: BoolQ, PIQA, SIQA, HellaSwag, WinoGrande, ARC, OpenBookQA, NaturalQuestions, TriviaQA, RACE, MMLU, BIG-bench hard, GSM8k, RealToxicityPrompts, WinoGender, and CrowS-Pairs.

Training dataset

The model was trained using the following data sources: CCNet [67%], C4 [15%], GitHub [4.5%], Wikipedia [4.5%], Books [4.5%], ArXiv [2.5%], and Stack Exchange [2%]. The Wikipedia and Books domains include data in the following languages: bg, ca, cs, da, de, en, es, fr, hr, hu, it, nl, pl, pt, ro, ru, sl, sr, sv, uk. See the paper for more details about the training set and corresponding preprocessing.

Quantitative analysis

Hyperparameters for the model architecture

Table 1 - Summary of LLama Model Hyperparameters

Results on eight standard common sense reasoning benchmarks

Table 2 - Summary of LLama Model Performance on Reasoning tasks

Results on bias

Table 3 - Summary bias of our model output

Ethical considerations

Data The data used to train the model is collected from various sources, mainly the web. It contains offensive, harmful, and biased content. Therefore, the model is expected to exhibit biases from the training data.

Human life The model is not intended to inform decisions about matters central to human life and should not be used in such a way.

Mitigations The web data was filtered based on its similarity to Wikipedia text and references. A Kneser - Ney language model and a fastText linear classifier were used for this purpose.

Risks and harms Risks and harms associated with large language models include the generation of harmful, offensive, or biased content. These models are also prone to generating incorrect information, sometimes referred to as hallucinations. This model is not expected to be an exception.

Use cases LLaMA is a foundational model. It should not be used in downstream applications without further investigation and risk mitigation. These risks and potential problematic use cases include, but are not limited to, the generation of misinformation and harmful, biased, or offensive content.

📄 License

The license for this model is other.