Bge Large Zh V1.5 Gguf

🚀 bge-large-zh-v1.5-gguf

This project offers quantized and unquantized embedding models in GGUF format for llama.cpp, providing significant speed - ups on CPU and GPU.

🚀 Quick Start

The source model can be found at: https://huggingface.co/BAAI/bge-large-zh-v1.5. These models in GGUF format are designed for use with llama.cpp. Compared to transformers, they almost always offer benefits, and the advantage over ONNX varies depending on the application. For larger models, it provides a significant speed - up on CPU and a moderate one on GPU. Given the relatively small size of these models, quantization doesn't bring huge benefits, but it can generate up to a 30% speed - up on CPU with minimal accuracy loss.

📦 Files Available

💻 Usage Examples

Basic Usage

from llama_cpp import Llama
model = Llama(gguf_path, embedding=True)
embed = model.embed(texts)

Here texts can either be a string or a list of strings, and the return value is a list of embedding vectors. The inputs are grouped into batches automatically for efficient execution. There is also LangChain integration through langchain_community.embeddings.LlamaCppEmbeddings.

📄 License

This project is licensed under the MIT license.