Legalbertpt_fp Open-source Language Model - Fine-tuning for Specific Tasks in the Portuguese Legal Domain

Legalbertpt Fp

Developed by raquelsilveira

Legalbert-pt is a language model specialized for the Portuguese legal domain through pre-training, which can be fine-tuned for specific tasks.

Large Language Model

Transformers

Open Source License:Openrail #Portuguese legal texts #Brazilian judicial documents #Domain-specific BERT

Downloads 738

Release Time : 5/2/2023

Model Overview

Legalbert-pt is a language model tailored for the Portuguese legal domain, developed in two versions: one as a supplementary version to the BERTimbau model, and another trained entirely from scratch. Experiments in legal entity recognition and litigation document classification tasks demonstrate that domain-specific language models outperform general-purpose language models in all tasks.

Model Features

Legal Domain Specialization

Pre-trained specifically for the Portuguese legal domain, achieving domain-specific capabilities.

Two Version Options

Offers a supplementary version based on BERTimbau and a version trained entirely from scratch to meet different needs.

Excellent Performance

Outperforms general-purpose language models in legal entity recognition and litigation document classification tasks.

Model Capabilities

Legal text comprehension

Legal entity recognition

Litigation document classification

Use Cases

Legal Document Processing

Legal Entity Recognition

Identify entities such as names of people and organizations in legal documents.

Outperforms general-purpose language models

Litigation Document Classification

Classify legal documents such as complaints, petitions, rulings, and judgments.

Outperforms general-purpose language models

🚀 LegalBert-pt

LegalBert-pt is a specialized language model for the legal domain in Portuguese, offering two versions for enhanced performance in legal tasks.

🚀 Quick Start

LegalBert-pt is a language model tailored for the legal domain in Portuguese. It was pre - trained for domain specialization and can be fine - tuned for specific tasks. Two versions were developed, with the BERTimbau - based one showing effectiveness in perplexity analysis. Experiments in legal entity identification and legal petition classification demonstrated that specialized language models outperform generic ones, highlighting the importance of domain specialization for improving learning algorithm accuracy.

✨ Features

Specialized for the legal domain in Portuguese.
Two versions available: one as a complement to BERTimbau and one from scratch.
Demonstrated better performance than generic language models in legal tasks.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer  # Or BertTokenizer
from transformers import AutoModelForPreTraining  # Or BertForPreTraining for loading pretraining heads
from transformers import AutoModel  # or BertModel, for BERT without pretraining heads
  
model = AutoModelForPreTraining.from_pretrained('raquelsilveira/legalbertpt_fp')
tokenizer = AutoTokenizer.from_pretrained('raquelsilveira/legalbertpt_fp')

📚 Documentation

Available models

Property	Details
Model Type	LegalBert-pt SC, LegalBert-pt FP
Initial model
#Layers	12
#Params	110M

Dataset

To pretrain various versions of the LegalBert-pt language model, a total of 1.5 million legal documents in Portuguese were collected from ten Brazilian courts. These documents included initial petitions, petitions, decisions, and sentences. The data were sourced from the Codex system of the Brazilian National Council of Justice (CNJ) under an agreement for research purposes.

Data source	Number of documents	%
Court of Justice of the State of Ceará	80,504	5.37%
Court of Justice of the State of Piauí	90,514	6.03%
Court of Justice of the State of Rio de Janeiro	33,320	2.22%
Court of Justice of the State of Rondônia	971,615	64.77%
Federal Regional Court of the 3rd Region	70,196	4.68%
Federal Regional Court of the 5th Region	6,767	0.45%
Regional Labor Court of the 9th Region	16,133	1.08%
Regional Labor Court of the 11th Region	5,351	0.36%
Regional Labor Court of the 13th Region	155,567	10.37%
Regional Labor Court of the 23th Region	70,033	4.67%
Total	1,500,000	100.00%

📄 License

The model is released under the OpenRAIL license.

📚 Citation

Raquel Silveira, Caio Ponte, Vitor Almeida, Vládia Pinheiro, and Vasco Furtado. 2023. LegalBert-pt: A Pretrained Language Model for the Brazilian Portuguese Legal Domain. In Intelligent Systems: 12th Brazilian Conference, BRACIS 2023, Belo Horizonte, Brazil, September 25–29, 2023, Proceedings, Part III. Springer-Verlag, Berlin, Heidelberg, 268–282. https://doi.org/10.1007/978-3-031-45392-2_18

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご