Mixtral 8x22B V0.1 GGUF
Mixtral 8x22B是由MistralAI发布的1760亿参数混合专家模型,支持多语言文本生成任务。
下载量 170.27k
发布时间 : 4/10/2024
模型简介
这是一个基于混合专家架构的大规模语言模型,支持法语、英语、西班牙语、意大利语和德语等多种语言的文本生成。模型采用Apache 2.0许可证发布,支持量化以降低资源需求。
模型特点
大规模混合专家架构
采用1760亿参数的混合专家架构,其中约350亿参数在推理时活跃,平衡了性能与效率
多语言支持
原生支持法语、英语、西班牙语、意大利语和德语等多种语言的文本生成
量化支持
提供从2比特到16比特的多种量化版本,显著降低硬件需求
长上下文处理
支持长达65k tokens的上下文窗口,适合处理长文档和复杂任务
模型能力
多语言文本生成
长文本处理
创意写作
技术文档生成
内容摘要
问答系统
使用案例
内容创作
网站内容生成
自动生成网站建设指南和内容
如示例所示,能生成详细的步骤指南
技术文档编写
自动生成技术文档和教程
商业应用
多语言客服
构建支持多种语言的自动客服系统
🚀 Mixtral-8x22B-v0.1-GGUF
Mixtral-8x22B-v0.1-GGUF是基于特定模型进行量化处理后的版本。它以Mixtral 8x22B模型为基础,具备多方面的特性,如不同的量化比特位数,可在不同硬件条件下使用。该模型适用于文本生成任务,为相关应用提供了有力支持。
🚀 快速开始
下载模型
你可以仅下载所需的量化模型,而无需克隆整个仓库,具体命令如下:
huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'
加载分片模型
llama_load_model_from_file
会自动检测文件数量,并从其余文件中加载额外的张量。
llama.cpp/main -m Mixtral-8x22B-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e
✨ 主要特性
- 模型基础信息:于4月10日,@MistralAI 发布了名为 “Mixtral 8x22B” 的模型,这是一个176B的混合专家模型(MoE)。
- 参数规模:141B MoE,约35B活跃。
- 上下文长度:65k 标记。
- 微调能力:基础模型可进行微调。
- 显存要求:在fp16格式下需要约260GB VRAM,在int4格式下需要73GB。
- 许可协议:根据其Discord信息,该模型采用Apache 2.0许可协议。
- 可用平台:可在 @huggingface(社区)获取。
- 分词器:使用与之前模型类似的分词器。
- 量化模型:这里的GGUF和量化模型基于 v2ray/Mixtral-8x22B-v0.1 模型。支持2-bit、3-bit、4-bit、5-bit、6-bit、8-bit、16-bit等多种量化比特位数。
💻 使用示例
基础用法
huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'
此命令用于从Hugging Face下载指定的量化模型文件。
高级用法
llama.cpp/main -m Mixtral-8x22B-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e
该命令用于加载分片模型并进行推理,输入提示信息为构建网站的步骤,生成1024个标记的输出。
输出示例
system_info: n_threads = 64 / 128 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 |
sampling:
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampling order:
CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature
generate: n_ctx = 512, n_batch = 2048, n_predict = 1024, n_keep = 1
Building a website can be done in 10 simple steps:
Step 1: Pick a domain name
The domain name is your address on the Internet. It’s what people type into the browser to get to your website. It’s important to pick a domain name that is easy to remember and relates to your business. For example, if you were a plumber, you could register a domain like fixitplumbing.com. You can check the availability of a domain name with the WHOIS lookup tool. If your domain is available, you can register it at a domain registrar like GoDaddy.com or Domain.com.
Step 2: Sign up for a web hosting account
Web hosting is the service that stores your website files and makes them available to people on the Internet. It’s important to pick a web hosting provider that is reliable and has good customer service. Some popular web hosting providers include Bluehost, Hostgator, and Dreamhost.
Step 3: Create a website template
A website template is a pre-designed website that you can use as a starting point for your own website. There are many free website templates available online. Once you’ve found a template you like, you can download it and start customizing it to fit your needs.
Step 4: Add your content
Once you’ve chosen a template, you’ll need to add your own content to it. This includes things like your company logo, contact information, and text about your business. You can also add photos and videos to make your website more engaging.
Step 5: Test your website
Before you make your website live, it’s important to test it out. This includes checking for broken links, typos, and making sure that all of your content is correct. You can also ask friends or family to test your website and give you feedback.
Step 6: Launch your website
Once you’re happy with your website, you can make it live on the Internet. This process is called “launching” your website. You’ll need to upload your website files to your web hosting account and then point your domain name to your hosting account. Once you’ve done this, your website will be available to people on the Internet.
Step 7: Promote your website
Just because you’ve built a website doesn’t mean people will automatically find it. You need to promote your website to get people to visit it. This includes things like search engine optimization (SEO) and social media marketing.
Step 8: Track your website’s progress
Once you’ve built your website, you need to track its progress. This includes things like traffic, search engine rankings, and conversion rates. By tracking your website’s progress, you can make sure that it’s working properly and that people are finding it.
Step 9: Keep your website up-to-date
Just because you’ve built your website doesn’t mean you’re done. You need to keep your website up-to-date by adding new content and fixing any errors that occur. By keeping your website up-to-date, you can make sure that it’s always available to people on the Internet.
Step 10: Repeat steps 1-10
Once you’ve built your website, you need to promote it so that people can find it. You can do this by adding your website to search engines, directories, and social media sites. You can also promote your website by word-of-mouth and by giving people your business card.
Once you’ve promoted your website, you need to keep track of how it’s doing. You can do this by using website analytics tools. These tools will help you see how many people are visiting your website, where they’re coming from, and what they’re doing on your site.
If you want to keep your website up-to-date, you need to add new content on a regular basis. You can do this by writing blog posts, creating infographics, or recording videos. You can also add new content by updating your website’s design and by adding new features.
By following these steps, you can build a website that’s available to people on the Internet. You can also keep your website
由于这似乎是一个基础模型,它会持续生成内容。
📄 许可证
该模型采用Apache 2.0许可协议。
🔗 致谢
- MistralAI 开放模型权重。
- v2ray 下载、转换并与社区共享 Mixtral-8x22B-v0.1 模型。
- philschmid 在其Twitter上分享的图片。
Phi 2 GGUF
其他
Phi-2是微软开发的一个小型但强大的语言模型,具有27亿参数,专注于高效推理和高质量文本生成。
大型语言模型 支持多种语言
P
TheBloke
41.5M
205
Roberta Large
MIT
基于掩码语言建模目标预训练的大型英语语言模型,采用改进的BERT训练方法
大型语言模型 英语
R
FacebookAI
19.4M
212
Distilbert Base Uncased
Apache-2.0
DistilBERT是BERT基础模型的蒸馏版本,在保持相近性能的同时更轻量高效,适用于序列分类、标记分类等自然语言处理任务。
大型语言模型 英语
D
distilbert
11.1M
669
Llama 3.1 8B Instruct GGUF
Meta Llama 3.1 8B Instruct 是一个多语言大语言模型,针对多语言对话用例进行了优化,在常见的行业基准测试中表现优异。
大型语言模型 英语
L
modularai
9.7M
4
Xlm Roberta Base
MIT
XLM-RoBERTa是基于100种语言的2.5TB过滤CommonCrawl数据预训练的多语言模型,采用掩码语言建模目标进行训练。
大型语言模型 支持多种语言
X
FacebookAI
9.6M
664
Roberta Base
MIT
基于Transformer架构的英语预训练模型,通过掩码语言建模目标在海量文本上训练,支持文本特征提取和下游任务微调
大型语言模型 英语
R
FacebookAI
9.3M
488
Opt 125m
其他
OPT是由Meta AI发布的开放预训练Transformer语言模型套件,参数量从1.25亿到1750亿,旨在对标GPT-3系列性能,同时促进大规模语言模型的开放研究。
大型语言模型 英语
O
facebook
6.3M
198
1
基于transformers库的预训练模型,适用于多种NLP任务
大型语言模型
Transformers

1
unslothai
6.2M
1
Llama 3.1 8B Instruct
Llama 3.1是Meta推出的多语言大语言模型系列,包含8B、70B和405B参数规模,支持8种语言和代码生成,优化了多语言对话场景。
大型语言模型
Transformers 支持多种语言

L
meta-llama
5.7M
3,898
T5 Base
Apache-2.0
T5基础版是由Google开发的文本到文本转换Transformer模型,参数规模2.2亿,支持多语言NLP任务。
大型语言模型 支持多种语言
T
google-t5
5.4M
702
精选推荐AI模型
Llama 3 Typhoon V1.5x 8b Instruct
专为泰语设计的80亿参数指令模型,性能媲美GPT-3.5-turbo,优化了应用场景、检索增强生成、受限生成和推理任务
大型语言模型
Transformers 支持多种语言

L
scb10x
3,269
16
Cadet Tiny
Openrail
Cadet-Tiny是一个基于SODA数据集训练的超小型对话模型,专为边缘设备推理设计,体积仅为Cosmo-3B模型的2%左右。
对话系统
Transformers 英语

C
ToddGoldfarb
2,691
6
Roberta Base Chinese Extractive Qa
基于RoBERTa架构的中文抽取式问答模型,适用于从给定文本中提取答案的任务。
问答系统 中文
R
uer
2,694
98