Legal T5 Small Summ En

🚀 legal_t5_small_summ_en模型

legal_t5_small_summ_en是一個用於英文法律文本摘要的模型。它首次發佈於 此倉庫，該模型在來自jrc - acquis的三個平行語料庫上進行訓練。

🚀 快速開始

模型描述

legal_t5_small_summ_en基於t5 - small模型，在大量平行文本語料庫上進行訓練。這是一個較小的模型，通過使用dmodel = 512、dff = 2,048、8頭注意力機制，並且編碼器和解碼器各只有6層，將t5的基線模型進行了縮小。這個變體大約有6000萬個參數。

預期用途和限制

該模型可用於英文法律文本的摘要提取。

使用方法

以下是在PyTorch中使用此模型對英文法律文本進行摘要提取的示例：

from transformers import AutoTokenizer, AutoModelWithLMHead, TranslationPipeline

pipeline = TranslationPipeline(
model=AutoModelWithLMHead.from_pretrained("SEBIS/legal_t5_small_summ_en"),
tokenizer=AutoTokenizer.from_pretrained(pretrained_model_name_or_path = "SEBIS/legal_t5_small_summ_en", do_lower_case=False, 
                                            skip_special_tokens=True),
    device=0
)

en_text = "THE COMMISSION OF THE EUROPEAN COMMUNITIES, Having regard to the Treaty establishing the European Community, Having regard to Council Regulation (EC) No 1255/1999 of 17 May 1999 on the common organisation of the market in milk and milk products [1], and in particular Article 15 thereof, Whereas: (1) Article 7(1) of Commission Regulation (EC) No 2799/1999 [2] fixes the amount of aid for skimmed milk and skimmed-milk powder intended for animal feed taking into account the factors set out in Article 11(2) of Regulation (EC) No 1255/1999. In view of the developments in the market price of skimmed-milk powder, of the increase in the market prices for competing proteins, and of the reduction of the supply of skimmed-milk powder, the amount of aid should be reduced. (2) Regulation (EC) No 2799/1999 should therefore be amended accordingly. (3) The Management Committee for Milk and Milk Products has not delivered an opinion within the time-limit set by its chairman, HAS ADOPTED THIS REGULATION: Article 1 In Article 7 of Regulation (EC) No 2799/1999, paragraph 1 is replaced by the following: \"1. Aid is fixed at: (a) EUR 1,62 per 100 kg of skimmed milk with a protein content of not less than 35,6 % of the non-fatty dry extract; (b) EUR 1,42 per 100 kg of skimmed milk with a protein content of not less than 31,4 % but less than 35,6 % of the non-fatty dry extract; (c) EUR 20,00 per 100 kg of skimmed-milk powder with a protein content of not less than 35,6 % of the non-fatty dry extract; (d) EUR 17,64 per 100 kg of skimmed-milk powder with a protein content of not less than 31,4 % but less than 35,6 % of the non-fatty dry extract.\" Article 2 This Regulation shall enter into force on the day following its publication in the Official Journal of the European Union. This Regulation shall be binding in its entirety and directly applicable in all Member States. Done at Brussels, 19 April 2006. For the Commission Mariann Fischer Boel Member of the Commission [1] OJ L 160, 26.6.1999, p. 48. Regulation as last amended by Regulation (EC) No 1913/2005 (OJ L 307, 25.11.2005, p. 2). [2] OJ L 340, 31.12.1999, p. 3. Regulation as last amended by Regulation (EC) No 1194/2005 (OJ L 194, 26.7.2005, p. 7). -------------------------------------------------- "

pipeline([en_text], max_length=512)

訓練數據

legal_t5_small_summ_en模型在 JRC - ACQUIS 數據集上進行訓練，該數據集包含2.2萬篇文本。

訓練過程

模型在單個TPU Pod V3 - 8上總共訓練了250K步，使用序列長度為512（批量大小為64）。它總共有大約2.2億個參數，採用編碼器 - 解碼器架構進行訓練。使用的優化器是AdaFactor，並在預訓練時採用逆平方根學習率調度。

預處理

使用從平行語料庫（所有可能的語言對）中8800萬行文本訓練的一元模型來獲取詞彙表（使用字節對編碼），該詞彙表與此模型一起使用。

預訓練

文檔未詳細描述預訓練的具體內容。

評估結果

當模型用於分類測試數據集時，取得了以下結果：

BibTeX引用和引用信息

由 Ahmed Elnaggar/@Elnaggar_AI 創建 | 領英