# OpenWebText Training

DUO is a pre-trained model for text generation, which can be used for masked language modeling tasks. It is trained on the OpenWebText corpus and has good performance.

DUO is a pretrained model based on the Transformers library, focusing on masked language modeling tasks and suitable for the field of natural language processing.

A masked language model based on the diffusion process, using the forward diffusion training method, capable of reconstructing the original text from inputs masked to different degrees.

A masked language model based on the diffusion process, similar in scale to GPT2-medium, trained through a forward diffusion process capable of reconstructing original inputs from different masking levels.