Mixtral 8x22B V0.1 GGUF

🚀 Mixtral-8x22B-v0.1-GGUF

Mixtral-8x22B-v0.1-GGUF是基於特定模型進行量化處理後的版本。它以Mixtral 8x22B模型為基礎，具備多方面的特性，如不同的量化比特位數，可在不同硬件條件下使用。該模型適用於文本生成任務，為相關應用提供了有力支持。

🚀 快速開始

下載模型

你可以僅下載所需的量化模型，而無需克隆整個倉庫，具體命令如下：

huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'

加載分片模型

llama_load_model_from_file 會自動檢測文件數量，並從其餘文件中加載額外的張量。

llama.cpp/main -m Mixtral-8x22B-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e

✨ 主要特性

• 模型基礎信息：於4月10日，@MistralAI 發佈了名為 “Mixtral 8x22B” 的模型，這是一個176B的混合專家模型（MoE）。
  • 參數規模：141B MoE，約35B活躍。
  • 上下文長度：65k 標記。
  • 微調能力：基礎模型可進行微調。
  • 顯存要求：在fp16格式下需要約260GB VRAM，在int4格式下需要73GB。
  • 許可協議：根據其Discord信息，該模型採用Apache 2.0許可協議。
  • 可用平臺：可在 @huggingface（社區）獲取。
  • 分詞器：使用與之前模型類似的分詞器。
• 量化模型：這裡的GGUF和量化模型基於 v2ray/Mixtral-8x22B-v0.1 模型。支持2-bit、3-bit、4-bit、5-bit、6-bit、8-bit、16-bit等多種量化比特位數。

💻 使用示例

基礎用法

huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'

此命令用於從Hugging Face下載指定的量化模型文件。

高級用法

llama.cpp/main -m Mixtral-8x22B-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e

該命令用於加載分片模型並進行推理，輸入提示信息為構建網站的步驟，生成1024個標記的輸出。

輸出示例

system_info: n_threads = 64 / 128 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 |
sampling:
        repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
        top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampling order:
CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature
generate: n_ctx = 512, n_batch = 2048, n_predict = 1024, n_keep = 1


 Building a website can be done in 10 simple steps:
Step 1: Pick a domain name
The domain name is your address on the Internet. It’s what people type into the browser to get to your website. It’s important to pick a domain name that is easy to remember and relates to your business. For example, if you were a plumber, you could register a domain like fixitplumbing.com. You can check the availability of a domain name with the WHOIS lookup tool. If your domain is available, you can register it at a domain registrar like GoDaddy.com or Domain.com.
Step 2: Sign up for a web hosting account
Web hosting is the service that stores your website files and makes them available to people on the Internet. It’s important to pick a web hosting provider that is reliable and has good customer service. Some popular web hosting providers include Bluehost, Hostgator, and Dreamhost.
Step 3: Create a website template
A website template is a pre-designed website that you can use as a starting point for your own website. There are many free website templates available online. Once you’ve found a template you like, you can download it and start customizing it to fit your needs.
Step 4: Add your content
Once you’ve chosen a template, you’ll need to add your own content to it. This includes things like your company logo, contact information, and text about your business. You can also add photos and videos to make your website more engaging.
Step 5: Test your website
Before you make your website live, it’s important to test it out. This includes checking for broken links, typos, and making sure that all of your content is correct. You can also ask friends or family to test your website and give you feedback.
Step 6: Launch your website
Once you’re happy with your website, you can make it live on the Internet. This process is called “launching” your website. You’ll need to upload your website files to your web hosting account and then point your domain name to your hosting account. Once you’ve done this, your website will be available to people on the Internet.
Step 7: Promote your website
Just because you’ve built a website doesn’t mean people will automatically find it. You need to promote your website to get people to visit it. This includes things like search engine optimization (SEO) and social media marketing.
Step 8: Track your website’s progress
Once you’ve built your website, you need to track its progress. This includes things like traffic, search engine rankings, and conversion rates. By tracking your website’s progress, you can make sure that it’s working properly and that people are finding it.
Step 9: Keep your website up-to-date
Just because you’ve built your website doesn’t mean you’re done. You need to keep your website up-to-date by adding new content and fixing any errors that occur. By keeping your website up-to-date, you can make sure that it’s always available to people on the Internet.
Step 10: Repeat steps 1-10

Once you’ve built your website, you need to promote it so that people can find it. You can do this by adding your website to search engines, directories, and social media sites. You can also promote your website by word-of-mouth and by giving people your business card.

Once you’ve promoted your website, you need to keep track of how it’s doing. You can do this by using website analytics tools. These tools will help you see how many people are visiting your website, where they’re coming from, and what they’re doing on your site.

If you want to keep your website up-to-date, you need to add new content on a regular basis. You can do this by writing blog posts, creating infographics, or recording videos. You can also add new content by updating your website’s design and by adding new features.

By following these steps, you can build a website that’s available to people on the Internet. You can also keep your website

由於這似乎是一個基礎模型，它會持續生成內容。

📄 許可證

該模型採用Apache 2.0許可協議。

🔗 致謝

• MistralAI 開放模型權重。
• v2ray 下載、轉換並與社區共享 Mixtral-8x22B-v0.1 模型。
• philschmid 在其Twitter上分享的圖片。