Gte Large Gguf

🚀 gte-large-gguf

This repository provides GGUF format files for the gte-large embedding model, enabling applications in text embeddings downstream tasks.

🚀 Quick Start

This repo offers GGUF format files for the gte-large embedding model. You can use these files with llama.cpp or LM Studio to compute text embeddings.

✨ Features

• Wide Application: Trained on a large - scale corpus of relevance text pairs, suitable for various text embedding downstream tasks such as information retrieval, semantic textual similarity, and text reranking.
• Multiple Quantization Options: Provides multiple quantization methods, allowing users to choose according to their own needs and hardware conditions.
• High Compatibility: Compatible with llama.cpp and LM Studio.

📦 Installation

Prerequisites

• Install llama.cpp as of commit 4524290e8.
• For LM Studio, download version 0.2.19 from [here](https://releases.lmstudio.ai/windows/0.2.19/beta/LM - Studio-0.2.19-Setup-Preview-1.exe) (Windows), here (MacOS), or here (Linux).

💻 Usage Examples

Basic Usage with llama.cpp

To compute a single embedding, build llama.cpp and run:

./embedding -ngl 99 -m [filepath-to-gguf].gguf -p 'search_query: What is TSNE?'

You can also submit a batch of texts to embed: texts.txt:

search_query: What is TSNE?
search_query: Who is Laurens Van der Maaten?

Compute multiple embeddings:

./embedding -ngl 99 -m [filepath-to-gguf].gguf -f texts.txt

Usage with LM Studio

1. Download the 0.2.19 beta build from the links provided above.
2. Open the app. Search for "ChristianAzinn" in the main search bar or in the "Search" tab on the left menu.
3. Select your model and the desired quantization method. It is recommended to choose Q8_0 for this model.
4. Wait for the model to download.
5. Navigate to the "Local Server" tab on the left menu and open the loader for text embedding models.
6. Select the downloaded model from the dropdown. Adjust configurations if necessary.
7. Click the "Start Server" button.

Example curl request to the API endpoint:

curl http://localhost:1234/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Your text string goes here",
    "model": "model-identifier-here"
  }'

📚 Documentation

Original Model

• Model Creator: thenlper
• Original Model: gte-large
• Model Type: BERT
• License: MIT

The GTE models are trained by Alibaba DAMO Academy on a large - scale corpus of relevance text pairs, covering a wide range of domains and scenarios.

This Repo

This repo contains GGUF format files for the gte-large embedding model. These files were converted and quantized with llama.cpp PR 5500, commit 34aa045de, on a consumer RTX 4090. This model supports up to 512 tokens of context.

🔧 Technical Details

Compatibility

These files are compatible with llama.cpp as of commit 4524290e8, as well as LM Studio as of version 0.2.19.

Explanation of Quantisation Methods

Provided Files

📄 License

This project is licensed under the MIT license.

Acknowledgements

Thanks to the LM Studio team and everyone else working on open - source AI.

This README is inspired by that of nomic-ai-embed-text-v1.5-gguf, another excellent embedding model, and those of the legendary TheBloke.