Simctg Wikitext103

🚀 基于SimCTG的GPT - 2语言模型

本模型提供了一个基于GPT - 2的语言模型，该模型在Wikitext - 103基准测试 (Merity et al., 2016) 上使用SimCTG进行训练，相关研究基于我们的论文 A Contrastive Framework for Neural Text Generation。

我们在 项目仓库 中提供了关于如何应用SimCTG和对比搜索的详细教程。接下来，我们将简要介绍如何使用我们的方法进行文本生成。

🚀 快速开始

📦 安装指南

pip install simctg --upgrade

💻 使用示例

基础用法

import torch
# load SimCTG language model
from simctg.simctggpt import SimCTGGPT
model_name = r'cambridgeltl/simctg_wikitext103'
model = SimCTGGPT(model_name)
model.eval()
tokenizer = model.tokenizer

高级用法

prefix_text = r"Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles . 
Buchanan also criticized the controls , calling"
print ('Prefix is: {}'.format(prefix_text))
tokens = tokenizer.tokenize(prefix_text)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
input_ids = torch.LongTensor(input_ids).view(1,-1)

beam_width, alpha, decoding_len = 8, 0.6, 128
output = model.fast_contrastive_search(input_ids=input_ids, beam_width=beam_width, 
                                       alpha=alpha, decoding_len=decoding_len)    
                                           
print("Output:\n" + 100 * '-')
print(tokenizer.decode(output))
'''
  Prefix is: Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles . 
             Buchanan also criticized the controls , calling
  Output:
  ----------------------------------------------------------------------------------------------------
  Butt criticized Donald's controls in certain situations in the game, as well as the difficulty of some levels and puzzles. Buchanan also 
  criticized the controls, calling them " unimpressive " and a " nightmare " of an experience to play with players unfamiliar with Tetris. 
  On the other hand, his opinion was shared by other reviewers, and some were critical of the game's technical design for the Wii version 
  of Tetris. In addition, Tintin's review included a quote from Roger Ebert, who said that Tetris was better than the original game due to 
  its simplicity and ease of play. Ebert's comments were included in the game's DVD commentary, released on March 22, 2010. It is unclear 
  if any of the video commentary was taken from the DVD
'''

如需了解我们工作的更多详细信息，请参考我们的主 项目仓库。

📄 许可证

引用说明

如果您觉得我们的论文和资源有用，请给我们点个星并引用我们的论文，谢谢！

@article{su2022contrastive,
  title={A Contrastive Framework for Neural Text Generation},
  author={Su, Yixuan and Lan, Tian and Wang, Yan and Yogatama, Dani and Kong, Lingpeng and Collier, Nigel},
  journal={arXiv preprint arXiv:2202.06417},
  year={2022}
}