Simctg Wikitext103

🚀 基於SimCTG的GPT - 2語言模型

本模型提供了一個基於GPT - 2的語言模型，該模型在Wikitext - 103基準測試 (Merity et al., 2016) 上使用SimCTG進行訓練，相關研究基於我們的論文 A Contrastive Framework for Neural Text Generation。

我們在 項目倉庫 中提供了關於如何應用SimCTG和對比搜索的詳細教程。接下來，我們將簡要介紹如何使用我們的方法進行文本生成。

🚀 快速開始

📦 安裝指南

pip install simctg --upgrade

💻 使用示例

基礎用法

import torch
# load SimCTG language model
from simctg.simctggpt import SimCTGGPT
model_name = r'cambridgeltl/simctg_wikitext103'
model = SimCTGGPT(model_name)
model.eval()
tokenizer = model.tokenizer

高級用法

prefix_text = r"Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles . 
Buchanan also criticized the controls , calling"
print ('Prefix is: {}'.format(prefix_text))
tokens = tokenizer.tokenize(prefix_text)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
input_ids = torch.LongTensor(input_ids).view(1,-1)

beam_width, alpha, decoding_len = 8, 0.6, 128
output = model.fast_contrastive_search(input_ids=input_ids, beam_width=beam_width, 
                                       alpha=alpha, decoding_len=decoding_len)    
                                           
print("Output:\n" + 100 * '-')
print(tokenizer.decode(output))
'''
  Prefix is: Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles . 
             Buchanan also criticized the controls , calling
  Output:
  ----------------------------------------------------------------------------------------------------
  Butt criticized Donald's controls in certain situations in the game, as well as the difficulty of some levels and puzzles. Buchanan also 
  criticized the controls, calling them " unimpressive " and a " nightmare " of an experience to play with players unfamiliar with Tetris. 
  On the other hand, his opinion was shared by other reviewers, and some were critical of the game's technical design for the Wii version 
  of Tetris. In addition, Tintin's review included a quote from Roger Ebert, who said that Tetris was better than the original game due to 
  its simplicity and ease of play. Ebert's comments were included in the game's DVD commentary, released on March 22, 2010. It is unclear 
  if any of the video commentary was taken from the DVD
'''

如需瞭解我們工作的更多詳細信息，請參考我們的主 項目倉庫。

📄 許可證

引用說明

如果您覺得我們的論文和資源有用，請給我們點個星並引用我們的論文，謝謝！

@article{su2022contrastive,
  title={A Contrastive Framework for Neural Text Generation},
  author={Su, Yixuan and Lan, Tian and Wang, Yan and Yogatama, Dani and Kong, Lingpeng and Collier, Nigel},
  journal={arXiv preprint arXiv:2202.06417},
  year={2022}
}