Rugpt Chitchat

🚀 Russian Chit-chat, Deductive and Common Sense reasoning model

This model serves as the core of a prototype dialogue system with two main functions.

🚀 Quick Start

The model has two main functions:

✨ Features

1. Chat Replication Generation: It takes the dialogue history (1 - 10 previous utterances) as input to generate chat responses.

   - Hi, how are you?
   - Hi, not so good.
   - <<< This is the response we expect from the model >>>
   

2. Answer Deduction: It can deduce answers to given questions based on additional facts or common sense. Relevant facts are assumed to be retrieved from an external knowledge base using another model, such as sbert_pq. The model will construct a grammatical and concise answer using the provided facts and question text.

   - Today is September 15th. What month is it now?
   - September
   

   The model can also perform syllogistic reasoning and solve simple arithmetic problems.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM


device = "cuda" if torch.cuda.is_available() else "cpu"
model_name = "inkoziev/rugpt_chitchat"
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.add_special_tokens({'bos_token': '<s>', 'eos_token': '</s>', 'pad_token': '<pad>'})
model = AutoModelForCausalLM.from_pretrained(model_name)
model.to(device)
model.eval()

# Input the last 2 - 3 dialogue utterances. Each utterance starts with "-" on a separate line
input_text = """<s>- Hi! What are you doing?
- Hi :) I'm in a taxi
-"""

encoded_prompt = tokenizer.encode(input_text, add_special_tokens=False, return_tensors="pt").to(device)

output_sequences = model.generate(input_ids=encoded_prompt, max_length=100, num_return_sequences=1, pad_token_id=tokenizer.pad_token_id)

text = tokenizer.decode(output_sequences[0].tolist(), clean_up_tokenization_spaces=True)[len(input_text)+1:]
text = text[: text.find('</s>')]
print(text)

📚 Documentation

Model Variants and Metrics

The currently released model has 760 million parameters, similar to sberbank-ai/rugpt3large_based_on_gpt2. The following table shows the accuracy of solving arithmetic problems on a held - out test set:

📄 License

The model is under the unlicense license.

Contacts

If you have any questions about using this model or suggestions for its improvement, please email mentalcomputing@gmail.com

Citation:

@MISC{rugpt_chitchat,
    author  = {Ilya Koziev},
    title   = {Russian Chit-chat with Common sence Reasoning},
    url     = {https://huggingface.co/inkoziev/rugpt_chitchat},
    year    = 2022
}