Mixtral 8x22B V0.1 GGUF

🚀 Mixtral-8x22B-v0.1-GGUF

Mixtral-8x22B-v0.1-GGUF是基于特定模型进行量化处理后的版本。它以Mixtral 8x22B模型为基础，具备多方面的特性，如不同的量化比特位数，可在不同硬件条件下使用。该模型适用于文本生成任务，为相关应用提供了有力支持。

🚀 快速开始

下载模型

你可以仅下载所需的量化模型，而无需克隆整个仓库，具体命令如下：

huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'

加载分片模型

llama_load_model_from_file 会自动检测文件数量，并从其余文件中加载额外的张量。

llama.cpp/main -m Mixtral-8x22B-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e

✨ 主要特性

• 模型基础信息：于4月10日，@MistralAI 发布了名为 “Mixtral 8x22B” 的模型，这是一个176B的混合专家模型（MoE）。
  • 参数规模：141B MoE，约35B活跃。
  • 上下文长度：65k 标记。
  • 微调能力：基础模型可进行微调。
  • 显存要求：在fp16格式下需要约260GB VRAM，在int4格式下需要73GB。
  • 许可协议：根据其Discord信息，该模型采用Apache 2.0许可协议。
  • 可用平台：可在 @huggingface（社区）获取。
  • 分词器：使用与之前模型类似的分词器。
• 量化模型：这里的GGUF和量化模型基于 v2ray/Mixtral-8x22B-v0.1 模型。支持2-bit、3-bit、4-bit、5-bit、6-bit、8-bit、16-bit等多种量化比特位数。

💻 使用示例

基础用法

huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'

此命令用于从Hugging Face下载指定的量化模型文件。

高级用法

llama.cpp/main -m Mixtral-8x22B-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e

该命令用于加载分片模型并进行推理，输入提示信息为构建网站的步骤，生成1024个标记的输出。

输出示例

system_info: n_threads = 64 / 128 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 |
sampling:
        repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
        top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampling order:
CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature
generate: n_ctx = 512, n_batch = 2048, n_predict = 1024, n_keep = 1


 Building a website can be done in 10 simple steps:
Step 1: Pick a domain name
The domain name is your address on the Internet. It’s what people type into the browser to get to your website. It’s important to pick a domain name that is easy to remember and relates to your business. For example, if you were a plumber, you could register a domain like fixitplumbing.com. You can check the availability of a domain name with the WHOIS lookup tool. If your domain is available, you can register it at a domain registrar like GoDaddy.com or Domain.com.
Step 2: Sign up for a web hosting account
Web hosting is the service that stores your website files and makes them available to people on the Internet. It’s important to pick a web hosting provider that is reliable and has good customer service. Some popular web hosting providers include Bluehost, Hostgator, and Dreamhost.
Step 3: Create a website template
A website template is a pre-designed website that you can use as a starting point for your own website. There are many free website templates available online. Once you’ve found a template you like, you can download it and start customizing it to fit your needs.
Step 4: Add your content
Once you’ve chosen a template, you’ll need to add your own content to it. This includes things like your company logo, contact information, and text about your business. You can also add photos and videos to make your website more engaging.
Step 5: Test your website
Before you make your website live, it’s important to test it out. This includes checking for broken links, typos, and making sure that all of your content is correct. You can also ask friends or family to test your website and give you feedback.
Step 6: Launch your website
Once you’re happy with your website, you can make it live on the Internet. This process is called “launching” your website. You’ll need to upload your website files to your web hosting account and then point your domain name to your hosting account. Once you’ve done this, your website will be available to people on the Internet.
Step 7: Promote your website
Just because you’ve built a website doesn’t mean people will automatically find it. You need to promote your website to get people to visit it. This includes things like search engine optimization (SEO) and social media marketing.
Step 8: Track your website’s progress
Once you’ve built your website, you need to track its progress. This includes things like traffic, search engine rankings, and conversion rates. By tracking your website’s progress, you can make sure that it’s working properly and that people are finding it.
Step 9: Keep your website up-to-date
Just because you’ve built your website doesn’t mean you’re done. You need to keep your website up-to-date by adding new content and fixing any errors that occur. By keeping your website up-to-date, you can make sure that it’s always available to people on the Internet.
Step 10: Repeat steps 1-10

Once you’ve built your website, you need to promote it so that people can find it. You can do this by adding your website to search engines, directories, and social media sites. You can also promote your website by word-of-mouth and by giving people your business card.

Once you’ve promoted your website, you need to keep track of how it’s doing. You can do this by using website analytics tools. These tools will help you see how many people are visiting your website, where they’re coming from, and what they’re doing on your site.

If you want to keep your website up-to-date, you need to add new content on a regular basis. You can do this by writing blog posts, creating infographics, or recording videos. You can also add new content by updating your website’s design and by adding new features.

By following these steps, you can build a website that’s available to people on the Internet. You can also keep your website

由于这似乎是一个基础模型，它会持续生成内容。

📄 许可证

该模型采用Apache 2.0许可协议。

🔗 致谢

• MistralAI 开放模型权重。
• v2ray 下载、转换并与社区共享 Mixtral-8x22B-v0.1 模型。
• philschmid 在其Twitter上分享的图片。