Bge Base En V1.5 Gguf

🚀 bge-base-en-v1.5-gguf

Quantized and unquantized embedding models in GGUF format for use with llama.cpp, offering speed improvements.

🚀 Quick Start

These models are quantized and unquantized embedding models in GGUF format, designed for use with llama.cpp. They almost certainly offer a large benefit over transformers, and the benefit over ONNX varies based on the application. They provide a significant speedup on CPU and a modest speedup on GPU for larger models. Due to the relatively small size of these models, quantization doesn't bring huge benefits, but it can generate up to a 30% speedup on CPU with minimal loss in accuracy.

📦 Files Available

💻 Usage Examples

Basic Usage

from llama_cpp import Llama
model = Llama(gguf_path, embedding=True)
embed = model.embed(texts)

Here texts can either be a string or a list of strings, and the return value is a list of embedding vectors. The inputs are grouped into batches automatically for efficient execution. There is also LangChain integration through langchain_community.embeddings.LlamaCppEmbeddings.

📄 License

This project is licensed under the MIT license.