đ cm4ker/USER-bge-m3-Q4_K_M-GGUF
This model focuses on sentence similarity and offers feature extraction capabilities. It was converted to the GGUF format from the base model, enabling efficient use in various applications.
đĻ Installation
Install llama.cpp
You can install llama.cpp through brew (works on Mac and Linux):
brew install llama.cpp
đģ Usage Examples
Use with llama.cpp
CLI
llama-cli --hf-repo cm4ker/USER-bge-m3-Q4_K_M-GGUF --hf-file user-bge-m3-q4_k_m.gguf -p "The meaning to life and the universe is"
Server
llama-server --hf-repo cm4ker/USER-bge-m3-Q4_K_M-GGUF --hf-file user-bge-m3-q4_k_m.gguf -c 2048
Alternative Usage Steps
You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo:
Step 1: Clone llama.cpp from GitHub
git clone https://github.com/ggerganov/llama.cpp
Step 2: Build llama.cpp
Move into the llama.cpp folder and build it with LLAMA_CURL=1
flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1
for Nvidia GPUs on Linux).
cd llama.cpp && LLAMA_CURL=1 make
Step 3: Run Inference
./llama-cli --hf-repo cm4ker/USER-bge-m3-Q4_K_M-GGUF --hf-file user-bge-m3-q4_k_m.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo cm4ker/USER-bge-m3-Q4_K_M-GGUF --hf-file user-bge-m3-q4_k_m.gguf -c 2048
đ Documentation
This model was converted to GGUF format from deepvk/USER-bge-m3
using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.
đ License
This model is licensed under the apache-2.0
license.
đ Model Information
Property |
Details |
Base Model |
deepvk/USER-bge-m3 |
Datasets |
deepvk/ru-HNP , deepvk/ru-WANLI , Shitao/bge-m3-data , RussianNLP/russian_super_glue , reciTAL/mlsum , Milana/russian_keywords , IlyaGusev/gazeta , d0rj/gsm8k-ru , bragovo/dsum_ru , CarlBrendt/Summ_Dialog_News |
Language |
ru |
Library Name |
sentence-transformers |
Pipeline Tag |
sentence-similarity |
Tags |
sentence-transformers , sentence-similarity , feature-extraction , llama-cpp , gguf-my-repo |