Roberta Uz

🚀 Uzbek NER model

This model is a fine - tuned version of [FacebookAI/xlm - roberta - large](https://huggingface.co/FacebookAI/xlm - roberta - large) on the Uzbek Ner dataset. It can be used to achieve high - precision named entity recognition in Uzbek text, providing important support for natural language processing tasks.

🚀 Quick Start

This model is a fine - tuned version of [FacebookAI/xlm - roberta - large](https://huggingface.co/FacebookAI/xlm - roberta - large) on the Uzbek Ner dataset. It achieves the following results on the evaluation set:

• Loss: 0.1754
• Precision: 0.5848
• Recall: 0.6313
• F1: 0.6071
• Accuracy: 0.9386

✨ Features

• Fine - Tuned: Based on the powerful [FacebookAI/xlm - roberta - large](https://huggingface.co/FacebookAI/xlm - roberta - large), fine - tuned on the Uzbek Ner dataset.
• High Performance: Achieves good results in loss, precision, recall, F1 score, and accuracy on the evaluation set.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

# from transformers import AutoTokenizer, AutoModelForTokenClassification
# import torch

# custom_id2label = {
#     0: "O", 1: "B-CARDINAL", 2: "I-CARDINAL", 3: "B-DATE", 4: "I-DATE",
#     5: "B-EVENT", 6: "I-EVENT", 7: "B-GPE", 8: "I-GPE", 9: "B-LOC", 10: "I-LOC",
#     11: "B-MONEY", 12: "I-MONEY", 13: "B-ORDINAL", 14: "B-ORG", 15: "I-ORG",
#     16: "B-PERCENT", 17: "I-PERCENT", 18: "B-PERSON", 19: "I-PERSON",
#     20: "B-TIME", 21: "I-TIME"
# }
# custom_label2id = {v: k for k, v in custom_id2label.items()}

# model_name = "mustafoyev202/roberta-uz"
# tokenizer = AutoTokenizer.from_pretrained(model_name)
# model = AutoModelForTokenClassification.from_pretrained(model_name, num_labels=23)

# model.config.id2label = custom_id2label
# model.config.label2id = custom_label2id

# text = "Tesla kompaniyasi AQSHda joylashgan."

# tokens = tokenizer(text.split(), return_tensors="pt", is_split_into_words=True)

# with torch.no_grad():
#     logits = model(**tokens).logits

# predicted_token_class_ids = logits.argmax(-1).squeeze().tolist()

# word_ids = tokens.word_ids()
# previous_word_id = None
# word_predictions = {}

# for i, word_id in enumerate(word_ids):
#     if word_id is not None:
#         label = custom_id2label[predicted_token_class_ids[i]]
#         if word_id != previous_word_id:  # New word
#             word_predictions[word_id] = label
#         previous_word_id = word_id

# words = text.split()  # Splitting for simplicity
# final_predictions = [(word, word_predictions.get(i, "O")) for i, word in enumerate(words)]

# print("Predictions:")
# for word, label in final_predictions:
#     print(f"{word}: {label}")

# labels = torch.tensor([predicted_token_class_ids]).unsqueeze(0)  # Adjust dimensions
# loss = model(**tokens, labels=labels).loss
# print("\nLoss:", round(loss.item(), 2))

📚 Documentation

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e - 05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon = 1e - 08 and optimizer_args = No additional optimizer arguments
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_warmup_ratio: 0.08
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0

📄 License

This model is released under the MIT license.