Legal T5 Small Summ En

🚀 legal_t5_small_summ_en模型

legal_t5_small_summ_en是一个用于英文法律文本摘要的模型。它首次发布于 此仓库，该模型在来自jrc - acquis的三个平行语料库上进行训练。

🚀 快速开始

模型描述

legal_t5_small_summ_en基于t5 - small模型，在大量平行文本语料库上进行训练。这是一个较小的模型，通过使用dmodel = 512、dff = 2,048、8头注意力机制，并且编码器和解码器各只有6层，将t5的基线模型进行了缩小。这个变体大约有6000万个参数。

预期用途和限制

该模型可用于英文法律文本的摘要提取。

使用方法

以下是在PyTorch中使用此模型对英文法律文本进行摘要提取的示例：

from transformers import AutoTokenizer, AutoModelWithLMHead, TranslationPipeline

pipeline = TranslationPipeline(
model=AutoModelWithLMHead.from_pretrained("SEBIS/legal_t5_small_summ_en"),
tokenizer=AutoTokenizer.from_pretrained(pretrained_model_name_or_path = "SEBIS/legal_t5_small_summ_en", do_lower_case=False, 
                                            skip_special_tokens=True),
    device=0
)

en_text = "THE COMMISSION OF THE EUROPEAN COMMUNITIES, Having regard to the Treaty establishing the European Community, Having regard to Council Regulation (EC) No 1255/1999 of 17 May 1999 on the common organisation of the market in milk and milk products [1], and in particular Article 15 thereof, Whereas: (1) Article 7(1) of Commission Regulation (EC) No 2799/1999 [2] fixes the amount of aid for skimmed milk and skimmed-milk powder intended for animal feed taking into account the factors set out in Article 11(2) of Regulation (EC) No 1255/1999. In view of the developments in the market price of skimmed-milk powder, of the increase in the market prices for competing proteins, and of the reduction of the supply of skimmed-milk powder, the amount of aid should be reduced. (2) Regulation (EC) No 2799/1999 should therefore be amended accordingly. (3) The Management Committee for Milk and Milk Products has not delivered an opinion within the time-limit set by its chairman, HAS ADOPTED THIS REGULATION: Article 1 In Article 7 of Regulation (EC) No 2799/1999, paragraph 1 is replaced by the following: \"1. Aid is fixed at: (a) EUR 1,62 per 100 kg of skimmed milk with a protein content of not less than 35,6 % of the non-fatty dry extract; (b) EUR 1,42 per 100 kg of skimmed milk with a protein content of not less than 31,4 % but less than 35,6 % of the non-fatty dry extract; (c) EUR 20,00 per 100 kg of skimmed-milk powder with a protein content of not less than 35,6 % of the non-fatty dry extract; (d) EUR 17,64 per 100 kg of skimmed-milk powder with a protein content of not less than 31,4 % but less than 35,6 % of the non-fatty dry extract.\" Article 2 This Regulation shall enter into force on the day following its publication in the Official Journal of the European Union. This Regulation shall be binding in its entirety and directly applicable in all Member States. Done at Brussels, 19 April 2006. For the Commission Mariann Fischer Boel Member of the Commission [1] OJ L 160, 26.6.1999, p. 48. Regulation as last amended by Regulation (EC) No 1913/2005 (OJ L 307, 25.11.2005, p. 2). [2] OJ L 340, 31.12.1999, p. 3. Regulation as last amended by Regulation (EC) No 1194/2005 (OJ L 194, 26.7.2005, p. 7). -------------------------------------------------- "

pipeline([en_text], max_length=512)

训练数据

legal_t5_small_summ_en模型在 JRC - ACQUIS 数据集上进行训练，该数据集包含2.2万篇文本。

训练过程

模型在单个TPU Pod V3 - 8上总共训练了250K步，使用序列长度为512（批量大小为64）。它总共有大约2.2亿个参数，采用编码器 - 解码器架构进行训练。使用的优化器是AdaFactor，并在预训练时采用逆平方根学习率调度。

预处理

使用从平行语料库（所有可能的语言对）中8800万行文本训练的一元模型来获取词汇表（使用字节对编码），该词汇表与此模型一起使用。

预训练

文档未详细描述预训练的具体内容。

评估结果

当模型用于分类测试数据集时，取得了以下结果：

BibTeX引用和引用信息

由 Ahmed Elnaggar/@Elnaggar_AI 创建 | 领英