Cadet Tiny

🚀 Cadet-Tiny 是什麼？

受 Allen AI 的 Cosmo-XL 啟發，Cadet-Tiny 是一個基於 SODA 數據集訓練的 超小型 對話模型。Cadet-Tiny 旨在用於邊緣推理（甚至可以在僅有 2GB 內存的樹莓派上運行）。

Cadet-Tiny 基於谷歌的 t5-small 預訓練模型進行訓練，因此，它的大小約為 Cosmo-3B 模型的 2%。

這是我製作的第一個 SEQ2SEQ 自然語言處理模型！我非常激動能在 HuggingFace 上與大家分享它！😊

如果您有任何問題或改進建議，請通過以下郵箱聯繫我：tcgoldfarb@gmail.com

📦 模型信息

📚 谷歌 Colab 鏈接

以下是谷歌 Colab 文件的鏈接，我在其中詳細介紹了模型的訓練過程以及如何使用 AI2 的 SODA 公共數據集。 點擊訪問

🚀 快速開始

使用以下代碼片段開始使用 Cadet-Tiny！

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import colorful as cf

cf.use_true_colors()
cf.use_style('monokai')
class CadetTinyAgent:
    def __init__(self):
        print(cf.bold | cf.purple("Waking up Cadet-Tiny..."))
        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
        self.tokenizer = AutoTokenizer.from_pretrained("t5-small", model_max_length=512)
        self.model = AutoModelForSeq2SeqLM.from_pretrained("ToddGoldfarb/Cadet-Tiny", low_cpu_mem_usage=True).to(self.device)
        self.conversation_history = ""

    def observe(self, observation):
        self.conversation_history = self.conversation_history + observation
        # The number 400 below is just a truncation safety net. It leaves room for 112 input tokens.
        if len(self.conversation_history) > 400:
            self.conversation_history = self.conversation_history[112:]

    def set_input(self, situation_narrative="", role_instruction=""):
        input_text = "dialogue: "

        if situation_narrative != "":
            input_text = input_text + situation_narrative

        if role_instruction != "":
            input_text = input_text + " <SEP> " + role_instruction

        input_text = input_text + " <TURN> " + self.conversation_history

        # Uncomment the line below to see what is fed to the model.
        # print(input_text)

        return input_text

    def generate(self, situation_narrative, role_instruction, user_response):
        user_response = user_response + " <TURN> "
        self.observe(user_response)

        input_text = self.set_input(situation_narrative, role_instruction)

        inputs = self.tokenizer([input_text], return_tensors="pt").to(self.device)
        
        # I encourage you to change the hyperparameters of the model! Start by trying to modify the temperature.
        outputs = self.model.generate(inputs["input_ids"], max_new_tokens=512, temperature=0.75, top_p=.95,
                                      do_sample=True)
        cadet_response = self.tokenizer.decode(outputs[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)
        added_turn = cadet_response + " <TURN> "
        self.observe(added_turn)

        return cadet_response

    def reset_history(self):
        self.conversation_history = []

    def run(self):
        def get_valid_input(prompt, default):
            while True:
                user_input = input(prompt)
                if user_input in ["Y", "N", "y", "n"]:
                    return user_input
                if user_input == "":
                    return default

        while True:
            continue_chat = ""

            # MODIFY THESE STRINGS TO YOUR LIKING :)
            situation_narrative = "Imagine you are Cadet-Tiny talking to ???."
            role_instruction = "You are Cadet-Tiny, and you are talking to ???."

            self.chat(situation_narrative, role_instruction)
            continue_chat = get_valid_input(cf.purple("Start a new conversation with new setup? [Y/N]:"), "Y")
            if continue_chat in ["N", "n"]:
                break

        print(cf.blue("CT: See you!"))

    def chat(self, situation_narrative, role_instruction):
        print(cf.green(
            "Cadet-Tiny is running! Input [RESET] to reset the conversation history and [END] to end the conversation."))
        while True:
            user_input = input("You: ")
            if user_input == "[RESET]":
                self.reset_history()
                print(cf.green("[Conversation history cleared. Chat with Cadet-Tiny!]"))
                continue
            if user_input == "[END]":
                break
            response = self.generate(situation_narrative, role_instruction, user_input)
            print(cf.blue("CT: " + response))


def main():
    print(cf.bold | cf.blue("LOADING MODEL"))

    CadetTiny = CadetTinyAgent()
    CadetTiny.run()


if __name__ == '__main__':
    main()

📄 引用與特別感謝

特別感謝 Hyunwoo Kim 與我討論使用 SODA 數據集的最佳方法。如果您還沒有了解過他們在 SODA、Prosocial-Dialog 或 COSMO 方面的工作，我建議您去看看！同時，也請閱讀關於 SODA 的論文！論文信息如下：

@article{kim2022soda,
    title={SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization},
    author={Hyunwoo Kim and Jack Hessel and Liwei Jiang and Peter West and Ximing Lu and Youngjae Yu and Pei Zhou and Ronan Le Bras and Malihe Alikhani and Gunhee Kim and Maarten Sap and Yejin Choi},
    journal={ArXiv},
    year={2022},
    volume={abs/2212.10465}
}