Granite 3.3 8b Instruct GGUF

🚀 Granite-3.3-8B-Instruct

Granite-3.3-8B-Instruct是一款擁有80億參數、上下文長度達128K的語言模型。它經過微調，在推理和指令遵循能力上表現出色。該模型基於Granite-3.3-8B-Base構建，在AlpacaEval-2.0和Arena-Hard等通用性能基準測試中取得顯著提升，在數學、編碼和指令遵循方面也有改進。它支持通過<think></think>和<response></response>標籤進行結構化推理，能清晰區分內部思考和最終輸出。

🚀 快速開始

安裝依賴庫

pip install torch torchvision torchaudio
pip install accelerate
pip install transformers

運行示例代碼

from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
import torch

model_path="ibm-granite/granite-3.3-8b-instruct"
device="cuda"
model = AutoModelForCausalLM.from_pretrained(
        model_path,
        device_map=device,
        torch_dtype=torch.bfloat16,
    )
tokenizer = AutoTokenizer.from_pretrained(
        model_path
)

conv = [{"role": "user", "content":"Redesign a common household item to make it more sustainable and user-friendly. Explain the changes and their benefits."}]

input_ids = tokenizer.apply_chat_template(conv, return_tensors="pt", thinking=True, return_dict=True, add_generation_prompt=True).to(device)

set_seed(42)
output = model.generate(
    **input_ids,
    max_new_tokens=8192,
)

prediction = tokenizer.decode(output[0, input_ids["input_ids"].shape[1]:], skip_special_tokens=True)
print(prediction)

✨ 主要特性

• 多語言支持：支持英語、德語、西班牙語、法語、日語、葡萄牙語、阿拉伯語、捷克語、意大利語、韓語、荷蘭語和中文等多種語言，用戶還可針對其他語言進行微調。
• 豐富的任務能力：具備思考、總結、文本分類、文本提取、問答、檢索增強生成（RAG）、代碼相關任務、函數調用任務、多語言對話以及長上下文任務（如長文檔/會議總結、長文檔問答等）處理能力。
• 結構化推理：支持通過<think></think>和<response></response>標籤進行結構化推理，清晰區分內部思考和最終輸出。

📦 安裝指南

安裝以下必要的庫：

pip install torch torchvision torchaudio
pip install accelerate
pip install transformers

💻 使用示例

基礎用法

from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
import torch

model_path="ibm-granite/granite-3.3-8b-instruct"
device="cuda"
model = AutoModelForCausalLM.from_pretrained(
        model_path,
        device_map=device,
        torch_dtype=torch.bfloat16,
    )
tokenizer = AutoTokenizer.from_pretrained(
        model_path
)

conv = [{"role": "user", "content":"Redesign a common household item to make it more sustainable and user-friendly. Explain the changes and their benefits."}]

input_ids = tokenizer.apply_chat_template(conv, return_tensors="pt", thinking=True, return_dict=True, add_generation_prompt=True).to(device)

set_seed(42)
output = model.generate(
    **input_ids,
    max_new_tokens=8192,
)

prediction = tokenizer.decode(output[0, input_ids["input_ids"].shape[1]:], skip_special_tokens=True)
print(prediction)

示例輸出

thinking=True

<think>To address this prompt, I'll focus on redesigning a common household item: the plastic water bottle. The goal is to make it more sustainable and user-friendly. Here are the proposed changes and their benefits:

1. **Material**: Replace plastic with biodegradable or recyclable materials like glass or plant-based plastics (PLA).
2. **Design**: Introduce a modular design for easy refilling and cleaning.
3. **Functionality**: Incorporate a built-in water filter for purification.
4. **Durability**: Enhance the bottle's lifespan through robust construction.
5. **User-friendliness**: Simplify the cap mechanism for easy opening and closing.

These changes would significantly reduce plastic waste, promote recycling, ensure access to clean drinking water, and enhance the overall user experience.

Now, I will detail each change and its benefits in a structured format.</think>

<response>### Redesign of the Plastic Water Bottle for Sustainability and User-Friendliness

#### 1. Material: Biodegradable or Recyclable Alternatives

**Change**: Transition from traditional PET plastic to glass or plant-based plastics (Polylactic Acid - PLA).

**Benefits**:
- **Sustainability**: Glass and PLA are biodegradable or highly recyclable, drastically reducing plastic pollution.
- **Health**: Unlike PET, these materials do not leach chemicals into the water, ensuring safer consumption.

#### 2. Modular Design for Easy Refilling and Cleaning

**Change**: Implement a detachable, interchangeable inner liner within a reusable outer casing.

**Benefits**:
- **User-Friendliness**: Users can easily remove the inner liner for thorough cleaning, reducing the risk of bacterial growth.
- **Sustainability**: The outer casing remains intact, encouraging reuse, while the inner liner can be replaced or recycled when worn out.

#### 3. Built-in Water Filter

**Change**: Integrate a removable, replaceable water filter into the bottle cap.

**Benefits**:
- **Accessibility**: Provides clean drinking water directly from any source (tap, river, etc.), eliminating the need for additional filtration devices.
- **Health**: Removes contaminants, bacteria, and impurities, ensuring safe hydration.
- **Sustainability**: Reduces the use of single-use plastic water bottles by offering a purification solution within a reusable container.

#### 4. Enhanced Durability

**Change**: Use high-quality, durable materials and robust construction techniques for both the outer casing and inner liner.

**Benefits**:
- **Longevity**: A more robust design increases the bottle's lifespan, reducing the frequency of replacements.
- **Cost-effectiveness**: Although the initial cost might be higher, the long-term savings from reduced replacements make it economical.
- **Sustainability**: Less frequent disposal means lower environmental impact.

#### 5. Simplified Cap Mechanism

**Change**: Adopt a one-touch, easy-to-open and close cap design, possibly inspired by sports bottle caps.

**Benefits**:
- **User-Friendliness**: Especially beneficial for children, elderly, or those with limited hand dexterity.
- **Convenience**: Quick access to water encourages more frequent hydration, promoting healthier habits.

### Conclusion

This redesigned water bottle addresses key sustainability concerns by minimizing plastic use and waste, while also enhancing user-friendliness through improved functionality, durability, and accessibility. By integrating biodegradable materials, a modular design, a built-in filter, robust construction, and a simple cap mechanism, the bottle not only reduces environmental footprint but also offers a superior user experience.</response>

thinking=False

Item for Redesign: Plastic Bottle (e.g., water or beverage bottle)

Redesign Concept: Eco-Friendly, Modular, and Multi-Functional Reusable Bottle

Changes:

1. Material: Replace the conventional plastic with food-grade, BPA-free, and recyclable materials such as stainless steel or glass. These materials are durable, non-toxic, and have a longer lifespan, reducing waste.

2. Modular Design: Introduce a modular design that allows users to customize their bottle according to their needs. This could include interchangeable caps, mouthpiece sizes, and carrying loops. Users can purchase additional modules as needed, promoting a more sustainable approach by reducing the need to buy an entirely new bottle for different purposes.

3. Integrated Filter: Incorporate a built-in, washable, and reusable filter that can remove impurities and improve the taste of water. This eliminates the need for single-use disposable filters or bottled water, further reducing plastic waste.

4. Smart Cap: Develop a smart cap with a built-in digital display and temperature sensor. This feature allows users to track their daily water intake, set hydration goals, and monitor the temperature of their beverage. The smart cap can be synced with a mobile app for additional functionality, such as reminders and progress tracking.

5. Easy-to-Clean Design: Ensure the bottle has a wide mouth and smooth interior surfaces for easy cleaning. Include a brush for hard-to-reach areas, making maintenance simple and encouraging regular use.

6. Collapsible Structure: Implement a collapsible design that reduces the bottle's volume when not in use, making it more portable and convenient for storage.

Benefits:

1. Sustainability: By using recyclable materials and reducing plastic waste, this redesigned bottle significantly contributes to a more sustainable lifestyle. The modular design and reusable filter also minimize single-use plastic consumption.

2. User-Friendly: The smart cap, easy-to-clean design, and collapsible structure make the bottle convenient and user-friendly. Users can customize their bottle to suit their needs, ensuring a better overall experience.

3. Healthier Option: Using food-grade, BPA-free materials and an integrated filter ensures that the beverages consumed are free from harmful chemicals and impurities, promoting a healthier lifestyle.

4. Cost-Effective: Although the initial investment might be higher, the long-term savings from reduced purchases of single-use plastic bottles and disposable filters make this reusable bottle a cost-effective choice.

5. Encourages Hydration: The smart cap's features, such as hydration tracking and temperature monitoring, can motivate users to stay hydrated and develop healthier habits.

By redesigning a common household item like the plastic bottle, we can create a more sustainable, user-friendly, and health-conscious alternative that benefits both individuals and the environment.

📚 詳細文檔

評估結果

多基準測試對比

數學基準測試

訓練數據

訓練數據主要來自兩個關鍵來源：

1. 具有寬鬆許可的公開可用數據集。
2. 用於增強推理能力的內部合成生成數據。

基礎設施

使用IBM的超級計算集群Blue Vela訓練Granite-3.3-8B-Instruct，該集群配備了NVIDIA H100 GPU，為在數千個GPU上訓練模型提供了可擴展且高效的基礎設施。

倫理考量與侷限性

Granite-3.3-8B-Instruct基於Granite-3.3-8B-Base構建，使用了寬鬆許可的開源數據和部分專有數據以提升性能。由於它繼承了前一個模型的基礎，因此適用於Granite-3.3-8B-Base的所有倫理考量和侷限性仍然適用。

資源鏈接

• 瞭解Granite的最新更新：https://www.ibm.com/granite
• 通過教程、最佳實踐和提示工程建議開始使用：https://www.ibm.com/granite/docs/
• 瞭解最新的Granite學習資源：https://github.com/ibm-granite-community/

🔧 技術細節

Granite-3.3-8B-Instruct基於Transformer架構，通過微調提升了推理和指令遵循能力。它使用了IBM的超級計算集群Blue Vela進行訓練，該集群配備了NVIDIA H100 GPU，為模型訓練提供了強大的計算支持。

📄 許可證

本項目採用Apache 2.0許可證。