Simctg Wikitext103

🚀 SimCTGによるGPT - 2言語モデル

このモデルは、論文 A Contrastive Framework for Neural Text Generationに基づき、Wikitext - 103ベンチマーク (Merity et al., 2016) でSimCTGを用いて学習されたGPT - 2言語モデルを提供します。

プロジェクトリポジトリにて、SimCTGとContrastive Searchの適用方法に関する詳細なチュートリアルを提供しています。以下では、当社のアプローチを使用してテキスト生成を行う簡単なチュートリアルを紹介します。

🚀 クイックスタート

このセクションでは、SimCTGを使用してテキスト生成を行う手順を説明します。

📦 インストール

SimCTGのインストール

pip install simctg --upgrade

💻 使用例

1. SimCTGモデルの初期化

import torch
# load SimCTG language model
from simctg.simctggpt import SimCTGGPT
model_name = r'cambridgeltl/simctg_wikitext103'
model = SimCTGGPT(model_name)
model.eval()
tokenizer = model.tokenizer

2. テキストプレフィックスの準備

prefix_text = r"Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles . 
Buchanan also criticized the controls , calling"
print ('Prefix is: {}'.format(prefix_text))
tokens = tokenizer.tokenize(prefix_text)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
input_ids = torch.LongTensor(input_ids).view(1,-1)

3. 対照検索によるテキスト生成

beam_width, alpha, decoding_len = 8, 0.6, 128
output = model.fast_contrastive_search(input_ids=input_ids, beam_width=beam_width, 
                                       alpha=alpha, decoding_len=decoding_len)    
                                           
print("Output:\n" + 100 * '-')
print(tokenizer.decode(output))
'''
  Prefix is: Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles . 
             Buchanan also criticized the controls , calling
  Output:
  ----------------------------------------------------------------------------------------------------
  Butt criticized Donald's controls in certain situations in the game, as well as the difficulty of some levels and puzzles. Buchanan also 
  criticized the controls, calling them " unimpressive " and a " nightmare " of an experience to play with players unfamiliar with Tetris. 
  On the other hand, his opinion was shared by other reviewers, and some were critical of the game's technical design for the Wii version 
  of Tetris. In addition, Tintin's review included a quote from Roger Ebert, who said that Tetris was better than the original game due to 
  its simplicity and ease of play. Ebert's comments were included in the game's DVD commentary, released on March 22, 2010. It is unclear 
  if any of the video commentary was taken from the DVD
'''

📚 ドキュメント

当社の研究の詳細については、メインのプロジェクトリポジトリを参照してください。

📄 ライセンス

もし当社の論文やリソースが役に立った場合は、スターを付けて論文を引用していただけると幸いです。ありがとうございます！

@article{su2022contrastive,
  title={A Contrastive Framework for Neural Text Generation},
  author={Su, Yixuan and Lan, Tian and Wang, Yan and Yogatama, Dani and Kong, Lingpeng and Collier, Nigel},
  journal={arXiv preprint arXiv:2202.06417},
  year={2022}
}