Convosensegenerator

🚀 ConvoSenseGeneratorモデルカード

ConvoSenseGeneratorは、対話コンテキストに対する常識的推論を生成する生成モデルです。感情反応、動機、原因、後続の出来事など、10種類の一般的な社会的常識タイプをカバーしています！

このモデルは、ChatGPT3.5を使用して合成的に収集された大規模データセットConvoSenseで学習されています。

ConvoSenseGeneratorが生成する推論は、人間が判断すると、対応する対話コンテキストに対して高い合理性、高い新規情報率、高い詳細度を達成しており、以前の人間が書いたデータセットで学習されたモデルを上回っています。

📚 詳細ドキュメント

• リポジトリ: コード
• 論文: ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI
• 担当者: Sarah E. Finch

🔧 技術詳細

ConvoSenseGeneratorは、最近のデータセットConvoSenseで学習されています。 ConvoSenseGeneratorのバックボーンモデルはT5-3bです。

💻 使用例

基本的な使用法

ConvoSenseGeneratorは、以下の常識タイプを、提供された質問を使用してカバーしています。

commonsense_questions = {
    "cause": 'What could have caused the last thing said to happen?', 
    "prerequisities": 'What prerequisites are required for the last thing said to occur?', 
    "motivation": 'What is an emotion or human drive that motivates Speaker based on what they just said?', 
    "subsequent": 'What might happen after what Speaker just said?', 
    "desire": 'What does Speaker want to do next?',
    "desire_o": 'What will Listener want to do next based on what Speaker just said?',
    "react": 'How is Speaker feeling after what they just said?',
    "react_o": 'How does Listener feel because of what Speaker just said?',
    "attribute": 'What is a likely characteristic of Speaker based on what they just said?',
    "constituents": 'What is a breakdown of the last thing said into a series of required subevents?' 
}

論文中の実験によると、ConvoSenseGeneratorの最良のパフォーマンスを発揮する設定は、以下の生成ハイパーパラメータを使用します。

generation_config = {
    "repetition_penalty": 1.0,
    "num_beams": 10,
    "num_beam_groups": 10,
    "diversity_penalty": 0.5
}

以下は、ConvoSenseGeneratorを実行するための簡単なコードスニペットです。

import torch
from transformers import AutoTokenizer, T5ForConditionalGeneration

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained("sefinch/ConvoSenseGenerator")
model = T5ForConditionalGeneration.from_pretrained("sefinch/ConvoSenseGenerator").to(device)

# ConvoSenseGenerator covers these commonsense types, using the provided questions
commonsense_questions = {
    "cause": 'What could have caused the last thing said to happen?', 
    "prerequisities": 'What prerequisites are required for the last thing said to occur?', 
    "motivation": 'What is an emotion or human drive that motivates Speaker based on what they just said?', 
    "subsequent": 'What might happen after what Speaker just said?', 
    "desire": 'What does Speaker want to do next?',
    "desire_o": 'What will Listener want to do next based on what Speaker just said?',
    "react": 'How is Speaker feeling after what they just said?',
    "react_o": 'How does Listener feel because of what Speaker just said?',
    "attribute": 'What is a likely characteristic of Speaker based on what they just said?',
    "constituents": 'What is a breakdown of the last thing said into a series of required subevents?' 
}

def format_input(conversation_history, commonsense_type):

    # prefix last turn with Speaker, and alternately prefix each previous turn with either Listener or Speaker
    prefixed_turns = list(
        reversed(
            [
                f"{'Speaker' if i % 2 == 0 else 'Listener'}: {u}"
                for i, u in enumerate(reversed(conversation_history))
            ]
        )
    )

    # model expects a maximum of 7 total conversation turns to be given
    truncated_turns = prefixed_turns[-7:]

    # conversation representation separates the turns with newlines
    conversation_string = '\n'.join(truncated_turns)

    # format the full input including the commonsense question
    input_text = f"provide a reasonable answer to the question based on the dialogue:\n{conversation_string}\n\n[Question] {commonsense_questions[commonsense_type]}\n[Answer]"

    return input_text

def generate(conversation_history, commonsense_type):
    # convert the input into the expected format to run the model
    input_text = format_input(conversation_history, commonsense_type) 

    # tokenize the input_text
    inputs = tokenizer([input_text], return_tensors="pt").to(device)

    # get multiple model generations using the best-performing generation configuration (based on experiments detailed in paper)
    outputs = model.generate(
        inputs["input_ids"],
        repetition_penalty=1.0,
        num_beams=10,
        num_beam_groups=10,
        diversity_penalty=0.5,
        num_return_sequences=5,
        max_new_tokens=400
    )

    # decode the generated inferences
    inferences = tokenizer.batch_decode(outputs, skip_special_tokens=True, clean_up_tokenization_spaces=False)

    return inferences

conversation = [
    "Hey, I'm trying to convince my parents to get a dog, but they say it's too much work.",
    "Well, you could offer to do everything for taking care of it. Have you tried that?",
    "But I don't want to have to take the dog out for walks when it is the winter!"
]

inferences = generate(conversation, "cause")
print('\n'.join(inferences))

# Outputs:
# the speaker's fear of the cold and the inconvenience of having to take the dog out in the winter.
# the speaker's preference for indoor activities during winter, such as watching movies or playing video games.
# the speaker's fear of getting sick from taking the dog out in the cold.
# a previous negative experience with taking dogs for walks in the winter.
# the listener's suggestion to offer to help with taking care of the dog, which the speaker may have considered but was not willing to do.

📄 ライセンス

このモデルはApache-2.0ライセンスの下で提供されています。

引用

このリポジトリのリソースが役に立った場合は、以下のように引用してください。

@article{convosense_finch:24,
    author = {Finch, Sarah E. and Choi, Jinho D.},
    title = "{ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI}",
    journal = {Transactions of the Association for Computational Linguistics},
    volume = {12},
    pages = {467-483},
    year = {2024},
    month = {05},
    issn = {2307-387X},
    doi = {10.1162/tacl_a_00659},
    url = {https://doi.org/10.1162/tacl\_a\_00659},
    eprint = {https://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl\_a\_00659/2369521/tacl\_a\_00659.pdf},
}