Sciphi Mistral 7B 32k

🚀 SciPhi-Mistral-7B-32k模型卡片

SciPhi-Mistral-7B-32k是一個基於Mistral-7B-v0.1微調的大語言模型（LLM）。該模型使用超過10億個標記進行了四個週期的微調，這些標記包括常規的指令調優數據和合成教科書。這項工作的目標是提高模型的科學推理和教育能力。為獲得最佳效果，請遵循Alpaca提示指南。

SciPhi-AI可通過免費的託管API使用，不過暴露的模型可能會有所不同。目前，可使用SciPhi-Self-RAG-Mistral-7B-32k。更多詳細信息可在文檔中找到。

✨ 主要特性

• 模型架構：基於Mistral-7B-v0.1，具有Transformer架構、分組查詢注意力、滑動窗口注意力和字節回退BPE分詞器等特性。
• 微調優化：使用超過10億個標記進行四個週期的微調，提升科學推理和教育能力。
• API支持：可通過免費的託管API使用。

📦 安裝指南

文檔未提及安裝步驟，跳過該章節。

💻 使用示例

基礎用法

# 推薦的聊天格式示例
messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]

# 轉換為以下格式
# 系統提示
print("### System:")
print("You are a friendly chatbot who always responds in the style of a pirate")
# 指令
print("### Instruction:")
print("How many helicopters can a human eat in one sitting?")
# 響應
print("### Response:")
print("...")

📚 詳細文檔

模型架構

推薦的聊天格式

我們推薦將以下格式：

messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]

轉換為：

### System:
You are a friendly chatbot who always responds in the style of a pirate

### Instruction:
How many helicopters can a human eat in one sitting?

### Response:
...

參考文獻

1. Lian, W., Goodson, B., Wang, G., Pentland, E., Cook, A., Vong, C., & Teknium. (2023). MistralOrca: Mistral-7B Model Instruct-tuned on Filtered OpenOrcaV1 GPT-4 Dataset. HuggingFace repository. 鏈接
2. Mukherjee, S., Mitra, A., Jawahar, G., Agarwal, S., Palangi, H., & Awadallah, A. (2023). Orca: Progressive Learning from Complex Explanation Traces of GPT-4. arXiv預印本 arXiv:2306.02707.
3. Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., Zhou, D., Le, Q. V., Zoph, B., Wei, J., & Roberts, A. (2023). The Flan Collection: Designing Data and Methods for Effective Instruction Tuning. arXiv預印本 arXiv:2301.13688.
4. Mistral AI. (2023). Model Card for Mistral-7B-v0.1. The Mistral-7B-v0.1 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters. Mistral-7B-v0.1 outperforms Llama 2 13B on all benchmarks tested. For full details, please refer to the paper and release blog post. Model Architecture: Transformer with Grouped-Query Attention, Sliding-Window Attention, and Byte-fallback BPE tokenizer. 鏈接

致謝

感謝AI Alignment Lab、vikp、jph00以及其他為這項工作做出貢獻的人。

📄 許可證

本項目採用MIT許可證。