Extended Mind Mpt 7b

🚀 Extended-Mind-Mpt-7b Model Card

This model is part of the Extended Mind Transformers collection. It implements the methods described in our paper. The model retrieves and attends to an external cache of key - value pairs (or memories), and its original model weights have not been edited.

• Github: https://github.com/normal - computing/extended - mind - transformers/
• ArXiv: https://arxiv.org/abs/2406.02332

Original architecture and code: By Mosaic ML.

• Developed by: Normal Computing, Adapted from Mosacic ML
• License: Apache 2.0

🚀 Quick Start

This section provides a quick overview of how to use the model.

✨ Features

• This model is part of the Extended Mind Transformers collection.
• It can retrieve and attend to an external cache of key - value pairs (memories).
• The original model weights remain unedited.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

External Memory

Passing external memories to the model is straightforward. Just pass the token ids to the model during instantiation, as shown in the following examples. Generating and caching the memories are handled internally during the first model.generate() call. You can update the memories using the following commands:

model.clear_memories()
model.memory_ids = list_of_new_token_ids

Set trust_remote_code=True to avoid warnings. Pass the memories to the model as a list of token ids.

from transformers import AutoModelForCausalLM, AutoTokenizer

ag_wiki_entry = """Alexander Grothendieck (/ˈɡroʊtəndiːk/; German pronunciation: [ˌalɛˈksandɐ ˈɡʁoːtn̩ˌdiːk] (listen); French: [ɡʁɔtɛndik]; 28 March 1928 – 13 November 2014) was a stateless (and then, since 1971, French) mathematician who became the leading figure in the creation of modern algebraic geometry.[7][8] His research extended the scope of the field and added elements of commutative algebra, homological algebra, sheaf theory, and category theory to its foundations, while his so - called "relative" perspective led to revolutionary advances in many areas of pure mathematics.[7][9] He is considered by many to be the greatest mathematician of the twentieth century.[10][11]"""

tokenizer_hf = AutoTokenizer.from_pretrained("normalcomputing/extended-mind-mpt-7b")
memories = tokenizer_hf(ag_wiki_entry).input_ids

model_hf = AutoModelForCausalLM.from_pretrained("normalcomputing/extended-mind-mpt-7b", external_memories=memories, trust_remote_code=True)

After this, you can generate text with the model as usual. The model will automatically use the memories during generation. You can update any config parameters (we set topk below) by passing new values to the model.generate() method.

inputs = "When did Alexander Grothendieck become a French citizen?"
inputs = tokenizer(inputs, return_tensors="pt").input_ids

outputs = model.generate(inputs, max_length=40, topk=2)
tokenizer.decode(outputs_hf['sequences'][0], skip_special_tokens=True)

Advanced Usage

Citations

By simply setting output_retrieved_memory_idx=True in the model.generate() method, you can retrieve the memory indices used during generation. We walk through an example in the demo notebook.

Additional configuration

LongLLaMA has several other parameters:

Additionally, the stride used to compute the memory representations can be set within the generate_cache() method. Smaller strides generate higher - quality representations, while larger strides require fewer computations.

📚 Documentation

The model's basic and advanced usage, as well as its additional configuration parameters, are described in the "Usage Examples" section.

🔧 Technical Details

No specific technical details (more than 50 words) are provided in the original document, so this section is skipped.

📄 License

This model is licensed under the Apache 2.0 license.

Limitations

This model is part of ongoing research at Normal Computing.