%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
EN
Geitje 7B Chat GPTQ
GEITje-7B-chat is a Dutch conversation model based on the Mistral architecture, specifically optimized for chat and dialogue tasks.
Downloads 21
Release Time : 12/19/2023
Model Overview
This model is a 7B-parameter Dutch conversation model based on the Mistral architecture, trained on the no_robots_nl and ultrachat_10k_nl datasets, suitable for Dutch dialogue generation tasks.
Model Features
Dutch language optimization
Specifically trained and optimized for Dutch dialogue tasks
Conversational ability
Focused on generating natural and fluent dialogue responses
Open-source license
Uses Apache 2.0 license, allowing commercial use
Model Capabilities
Dutch dialogue generation
Chatbot development
Natural language understanding
Use Cases
Customer service
Dutch customer service bot
Automated customer service system for the Dutch market
Provides smooth and natural Dutch customer service conversations
Education
Dutch language learning assistant
Helps learners practice Dutch conversations
Provides a natural Dutch conversation practice environment
đ Geitje 7B Chat - GPTQ
This repository provides GPTQ model files for Geitje 7B Chat, offering multiple quantisation options for different hardware and requirements.
đ Quick Start
Downloading the Model
In text-generation-webui
- To download from the
mainbranch, enterTheBloke/GEITje-7B-chat-GPTQin the "Download model" box. - To download from another branch, add
:branchnameto the end of the download name, e.g.,TheBloke/GEITje-7B-chat-GPTQ:gptq-4bit-32g-actorder_True
From the command line
- Install the
huggingface-hubPython library:
pip3 install huggingface-hub
- To download the
mainbranch to a folder calledGEITje-7B-chat-GPTQ:
mkdir GEITje-7B-chat-GPTQ
huggingface-cli download TheBloke/GEITje-7B-chat-GPTQ --local-dir GEITje-7B-chat-GPTQ --local-dir-use-symlinks False
- To download from a different branch, add the
--revisionparameter:
mkdir GEITje-7B-chat-GPTQ
huggingface-cli download TheBloke/GEITje-7B-chat-GPTQ --revision gptq-4bit-32g-actorder_True --local-dir GEITje-7B-chat-GPTQ --local-dir-use-symlinks False
Using the Model in text-generation-webui
- Ensure you're using the latest version of text-generation-webui.
- Click the Model tab.
- Under Download custom model or LoRA, enter
TheBloke/GEITje-7B-chat-GPTQ.- To download from a specific branch, enter, for example,
TheBloke/GEITje-7B-chat-GPTQ:gptq-4bit-32g-actorder_True - See the "Provided files, and GPTQ parameters" section for the list of branches for each option.
- To download from a specific branch, enter, for example,
- Click Download.
- Once the model has finished downloading, it will say "Done".
- In the top left, click the refresh icon next to Model.
- In the Model dropdown, choose the model you just downloaded:
GEITje-7B-chat-GPTQ - The model will automatically load and is now ready for use!
- If you want any custom settings, set them and then click Save settings for this model followed by Reload the Model in the top right.
- Note that you do not need to and should not set manual GPTQ parameters anymore. These are set automatically from the file
quantize_config.json.
- Note that you do not need to and should not set manual GPTQ parameters anymore. These are set automatically from the file
- Once you're ready, click the Text Generation tab and enter a prompt to get started!
Serving the Model from Text Generation Inference (TGI)
It's recommended to use TGI version 1.1.0 or later. The official Docker container is: ghcr.io/huggingface/text-generation-inference:1.1.0
Example Docker parameters:
--model-id TheBloke/GEITje-7B-chat-GPTQ --port 3000 --quant
⨠Features
- Multiple GPTQ parameter permutations are provided, allowing users to choose the best option for their hardware and requirements.
- The model supports various inference servers/webuis, including text-generation-webui, KoboldAI United, LoLLMS Web UI, and Hugging Face Text Generation Inference (TGI).
đĻ Installation
Prerequisites
- For GPTQ models, Linux (NVidia/AMD) and Windows (NVidia only) are currently supported. macOS users should use GGUF models.
- Install the necessary libraries as described in the "Quick Start" section.
đ Documentation
Model Information
| Property | Details |
|---|---|
| Model Type | mistral |
| Base Model | Rijgersberg/GEITje-7B-chat |
| Training Data | Rijgersberg/no_robots_nl, Rijgersberg/ultrachat_10k_nl |
| Model Creator | Edwin Rijgersberg |
| Quantized By | TheBloke |
| License | apache-2.0 |
Prompt Template
<|user|>
{prompt}
<|assistant|>
Provided files, and GPTQ parameters
Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements.
Each separate quant is in a different branch. See below for instructions on fetching from different branches.
Most GPTQ files are made with AutoGPTQ. Mistral models are currently made with Transformers.
Explanation of GPTQ parameters
- Bits: The bit size of the quantised model.
- GS: GPTQ group size. Higher numbers use less VRAM, but have lower quantisation accuracy. "None" is the lowest possible value.
- Act Order: True or False. Also known as
desc_act. True results in better quantisation accuracy. Some GPTQ clients have had issues with models that use Act Order plus Group Size, but this is generally resolved now. - Damp %: A GPTQ parameter that affects how samples are processed for quantisation. 0.01 is default, but 0.1 results in slightly better accuracy.
- GPTQ dataset: The calibration dataset used during quantisation. Using a dataset more appropriate to the model's training can improve quantisation accuracy. Note that the GPTQ calibration dataset is not the same as the dataset used to train the model - please refer to the original model repo for details of the training dataset(s).
- Sequence Length: The length of the dataset sequences used for quantisation. Ideally this is the same as the model sequence length. For some very long sequence models (16+K), a lower sequence length may have to be used. Note that a lower sequence length does not limit the sequence length of the quantised model. It only impacts the quantisation accuracy on longer inference sequences.
- ExLlama Compatibility: Whether this file can be loaded with ExLlama, which currently only supports Llama and Mistral models in 4-bit.
| Branch | Bits | GS | Act Order | Damp % | GPTQ Dataset | Seq Len | Size | ExLlama | Desc |
|---|---|---|---|---|---|---|---|---|---|
| main | 4 | 128 | Yes | 0.1 | Dolly 15K Dutch | 4096 | 4.16 GB | Yes | 4-bit, with Act Order and group size 128g. Uses even less VRAM than 64g, but with slightly lower accuracy. |
| gptq-4bit-32g-actorder_True | 4 | 32 | Yes | 0.1 | Dolly 15K Dutch | 4096 | 4.57 GB | Yes | 4-bit, with Act Order and group size 32g. Gives highest possible inference quality, with maximum VRAM usage. |
| gptq-8bit--1g-actorder_True | 8 | None | Yes | 0.1 | Dolly 15K Dutch | 4096 | 7.52 GB | No | 8-bit, with Act Order. No group size, to lower VRAM requirements. |
| gptq-8bit-128g-actorder_True | 8 | 128 | Yes | 0.1 | Dolly 15K Dutch | 4096 | 7.68 GB | No | 8-bit, with group size 128g for higher inference quality and with Act Order for even higher accuracy. |
| gptq-8bit-32g-actorder_True | 8 | 32 | Yes | 0.1 | Dolly 15K Dutch | 4096 | 8.17 GB | No | 8-bit, with group size 32g and Act Order for maximum inference quality. |
| gptq-4bit-64g-actorder_True | 4 | 64 | Yes | 0.1 | Dolly 15K Dutch | 4096 | 4.29 GB | Yes | 4-bit, with Act Order and group size 64g. Uses less VRAM than 32g, but with slightly lower accuracy. |
Repositories available
- AWQ model(s) for GPU inference.
- GPTQ models for GPU inference, with multiple quantisation parameter options.
- 2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference
- Edwin Rijgersberg's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions
đ§ Technical Details
Quantisation
These files were quantised using hardware kindly provided by Massed Compute.
Known compatible clients / servers
GPTQ models are currently supported on Linux (NVidia/AMD) and Windows (NVidia only). macOS users: please use GGUF models.
These GPTQ models are known to work in the following inference servers/webuis:
This may not be a complete list; if you know of others, please let me know!
đ License
This model is licensed under the apache-2.0 license.
Phi 2 GGUF
Other
Phi-2 is a small yet powerful language model developed by Microsoft, featuring 2.7 billion parameters, focusing on efficient inference and high-quality text generation.
Large Language Model Supports Multiple Languages
P
TheBloke
41.5M
205
Roberta Large
MIT
A large English language model pre-trained with masked language modeling objectives, using improved BERT training methods
Large Language Model English
R
FacebookAI
19.4M
212
Distilbert Base Uncased
Apache-2.0
DistilBERT is a distilled version of the BERT base model, maintaining similar performance while being more lightweight and efficient, suitable for natural language processing tasks such as sequence classification and token classification.
Large Language Model English
D
distilbert
11.1M
669
Llama 3.1 8B Instruct GGUF
Meta Llama 3.1 8B Instruct is a multilingual large language model optimized for multilingual dialogue use cases, excelling in common industry benchmarks.
Large Language Model English
L
modularai
9.7M
4
Xlm Roberta Base
MIT
XLM-RoBERTa is a multilingual model pretrained on 2.5TB of filtered CommonCrawl data across 100 languages, using masked language modeling as the training objective.
Large Language Model Supports Multiple Languages
X
FacebookAI
9.6M
664
Roberta Base
MIT
An English pre-trained model based on Transformer architecture, trained on massive text through masked language modeling objectives, supporting text feature extraction and downstream task fine-tuning
Large Language Model English
R
FacebookAI
9.3M
488
Opt 125m
Other
OPT is an open pre-trained Transformer language model suite released by Meta AI, with parameter sizes ranging from 125 million to 175 billion, designed to match the performance of the GPT-3 series while promoting open research in large-scale language models.
Large Language Model English
O
facebook
6.3M
198
1
A pretrained model based on the transformers library, suitable for various NLP tasks
Large Language Model
Transformers
Transformers1
unslothai
6.2M
1
Llama 3.1 8B Instruct
Llama 3.1 is Meta's multilingual large language model series, featuring 8B, 70B, and 405B parameter scales, supporting 8 languages and code generation, with optimized multilingual dialogue scenarios.
Large Language Model Transformers Supports Multiple Languages
Transformers Supports Multiple LanguagesL
meta-llama
5.7M
3,898
T5 Base
Apache-2.0
The T5 Base Version is a text-to-text Transformer model developed by Google with 220 million parameters, supporting multilingual NLP tasks.
Large Language Model Supports Multiple Languages
T
google-t5
5.4M
702
Featured Recommended AI Models
Š 2025AIbase