Legal BERTimbau Large

🚀 法律版巴西BERT模型（Legal_BERTimbau）

Legal_BERTimbau是一個針對法律領域微調的BERT模型，基於巴西葡萄牙語預訓練模型BERTimbau，能夠在法律文本處理任務中發揮出色的性能。

🚀 快速開始

安裝依賴

確保你已經安裝了transformers庫：

pip install transformers

加載模型和分詞器

from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("rufimelo/Legal-BERTimbau-large")
model = AutoModelForMaskedLM.from_pretrained("rufimelo/Legal-BERTimbau-large")

✨ 主要特性

• 領域適配：基於BERTimbau模型，針對法律領域進行微調，能更好地處理法律文本。
• 多模型可選：提供基礎版和大型版兩種模型，滿足不同需求。

📦 安裝指南

使用pip安裝所需的transformers庫：

pip install transformers

💻 使用示例

基礎用法

掩碼語言建模預測示例

from  transformers  import  pipeline
from  transformers  import  AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("rufimelo/Legal-BERTimbau-large")
model = AutoModelForMaskedLM.from_pretrained("rufimelo/Legal-BERTimbau-large")

pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer)
pipe('O advogado apresentou [MASK] para o juíz')
# [{'score': 0.5034703612327576, 
#'token': 8190, 
#'token_str': 'recurso', 
#'sequence': 'O advogado apresentou recurso para o juíz'}, 
#{'score': 0.07347951829433441, 
#'token': 21973, 
#'token_str': 'petição', 
#'sequence': 'O advogado apresentou petição para o juíz'}, 
#{'score': 0.05165359005331993, 
#'token': 4299, 
#'token_str': 'resposta', 
#'sequence': 'O advogado apresentou resposta para o juíz'}, 
#{'score': 0.04611917585134506,
#'token': 5265, 
#'token_str': 'exposição', 
#'sequence': 'O advogado apresentou exposição para o juíz'}, 
#{'score': 0.04068068787455559, 
#'token': 19737, 'token_str': 
#'alegações', 
#'sequence': 'O advogado apresentou alegações para o juíz'}]

獲取BERT嵌入

import  torch
from  transformers  import  AutoModel

model = AutoModel.from_pretrained('rufimelo/Legal-BERTimbau-large')
input_ids = tokenizer.encode('O advogado apresentou recurso para o juíz', return_tensors='pt')

with  torch.no_grad():
	outs = model(input_ids)
	encoded = outs[0][0, 1:-1]
	
#tensor([[ 0.0328, -0.4292, -0.6230, ..., -0.3048, -0.5674, 0.0157], 
#[-0.3569, 0.3326, 0.7013, ..., -0.7778, 0.2646, 1.1310], 
#[ 0.3169, 0.4333, 0.2026, ..., 1.0517, -0.1951, 0.7050], 
#..., 
#[-0.3648, -0.8137, -0.4764, ..., -0.2725, -0.4879, 0.6264], 
#[-0.2264, -0.1821, -0.3011, ..., -0.5428, 0.1429, 0.0509], 
#[-1.4617, 0.6281, -0.0625, ..., -1.2774, -0.4491, 0.3131]])

📚 詳細文檔

模型介紹

Legal_BERTimbau Large是基於BERTimbau Large微調的BERT模型。BERTimbau Base是針對巴西葡萄牙語的預訓練BERT模型，在命名實體識別、句子文本相似度和文本蘊含識別三項下游NLP任務中達到了最先進的性能。它有Base和Large兩種尺寸可供選擇。

當訓練數據和測試數據之間存在領域差異時，語言模型的性能可能會發生顯著變化。為了創建適應法律領域的葡萄牙語語言模型，原始的BERTimbau模型經過了微調階段，在30000份在線可用的葡萄牙語法律文件上進行了1個“預訓練”週期的訓練（學習率：1e - 5）。

可用模型

📄 許可證

本項目採用MIT許可證。

📖 引用

如果您使用了本項目，請引用BERTimbau的相關工作：

@inproceedings{souza2020bertimbau,
  author    = {F{\'a}bio Souza and
               Rodrigo Nogueira and
               Roberto Lotufo},
  title     = {{BERT}imbau: pretrained {BERT} models for {B}razilian {P}ortuguese},
  booktitle = {9th Brazilian Conference on Intelligent Systems, {BRACIS}, Rio Grande do Sul, Brazil, October 20-23 (to appear)},
  year      = {2020}
}