Reasoning Bert Ccnews

🚀 基于google-bert/bert-base-uncased的句子转换器模型

本模型是基于 google-bert/bert-base-uncased 在 reason_unfiltered 数据集上微调得到的 sentence-transformers 模型。它能将句子和段落映射到768维的密集向量空间，可用于语义文本相似度计算、语义搜索、释义挖掘、文本分类、聚类等任务。

🚀 快速开始

安装Sentence Transformers库

pip install -U sentence-transformers

加载模型并进行推理

from sentence_transformers import SentenceTransformer

# 从🤗 Hub下载模型
model = SentenceTransformer("bwang0911/reasoning-bert-ccnews")
# 进行推理
sentences = [
    'Energy advocates call for new commitment to renewable growth',
    'The piece below was submitted by CFE, VoteSolar, and Environment Connecticut in response to the latest delay in the shared solar pilot program.\nSolar and environmental advocates are calling for a new community solar program in Connecticut that will expand solar access, energy choices and consumer savings for families, municipalities, and businesses statewide. The demand follows today’s Department of Energy and Environmental Protection (DEEP) technical hearing where attendees reviewed the state’s current Shared Clean Energy Facilities pilot program. The pilot has stalled several times over the last two years, most recently following DEEP’s decision to scrap all the proposals they have received and issue a new request for projects. DEEP heard from many advocates and developers at the hearing who are frustrated with this latest delay and skeptical about the long term success of the pilot.\nThe current pilot program was meant to expand solar access to Connecticut energy customers who can’t put solar on their own roof, but it contained flaws that have prevented any development to date. As set out in the legislation, the program has several poor design elements and a goal too small to draw significant private sector interest. Below are statements from stakeholders in Connecticut’s clean energy economy:\n“For years, Connecticut has missed out on the opportunity to bring solar energy choices to all consumers and more clean energy jobs to the state,” said Sean Garren, Northeast Regional director for Vote Solar. “Connecticut’s lackluster community solar program hasn’t unlocked the benefits of solar access for a single resident to date due to poor design and a lack of ambition at the scale needed, brought about by the electric utilities’ intervention. We’re calling on the legislature to catch up to the rest of New England — and the nation — with a smart, well-structured community solar program designed to serve consumers statewide.”\n“Two years of foot dragging and refusal by the Department of Energy and Environmental Protection to follow the law and implement a community solar program is preventing tens of thousands of Connecticut families from gaining access to clean, affordable, secure solar power,” said Chris Phelps, State Director for Environment Connecticut. “Community solar is helping other states accelerate solar growth, create jobs, and cut pollution. Connecticut policy makers should take action now to create a bold community solar program.”\n“Shared solar programs have been sweeping the nation for the last decade, but Connecticut has been left in the shade — losing out on healthier air, investment dollars, and green jobs that would accompany a full-scale, statewide shared solar program,” said Claire Coleman, Climate and Energy Attorney for Connecticut Fund for the Environment. “DEEP’s decision to start over with the already overly-restrictive shared solar pilot puts Connecticut further in the dark. Our climate and economy cannot wait any longer. Connecticut’s leaders must move quickly to ramp up in-state renewables through a full-scale shared solar program if Connecticut is going to have any chance of meeting its obligations under the Global Warming Solutions Act to reduce greenhouse gas emissions.”\nVote Solar is a nonprofit organization working to foster economic development and energy independence by bringing solar energy to the mainstream nationwide. Learn more at votesolar.org.',
    "The second text elaborates on the first by providing details about the specific context of the energy advocates' call for renewable growth. It identifies the advocates (CFE, VoteSolar, Environment Connecticut), the specific renewable energy program (community solar), and the reasons for their call, including program delays and design flaws.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# 获取嵌入向量的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

✨ 主要特性

• 多任务支持：可用于语义文本相似度计算、语义搜索、释义挖掘、文本分类、聚类等多种自然语言处理任务。
• 高维向量映射：能将句子和段落映射到768维的密集向量空间，便于进行语义分析。
• 微调优化：基于 google-bert/bert-base-uncased 模型在特定数据集上进行微调，提升了模型在相关任务上的性能。

📦 安装指南

安装Sentence Transformers库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import SentenceTransformer

# 从🤗 Hub下载模型
model = SentenceTransformer("bwang0911/reasoning-bert-ccnews")
# 进行推理
sentences = [
    'Energy advocates call for new commitment to renewable growth',
    'The piece below was submitted by CFE, VoteSolar, and Environment Connecticut in response to the latest delay in the shared solar pilot program.\nSolar and environmental advocates are calling for a new community solar program in Connecticut that will expand solar access, energy choices and consumer savings for families, municipalities, and businesses statewide. The demand follows today’s Department of Energy and Environmental Protection (DEEP) technical hearing where attendees reviewed the state’s current Shared Clean Energy Facilities pilot program. The pilot has stalled several times over the last two years, most recently following DEEP’s decision to scrap all the proposals they have received and issue a new request for projects. DEEP heard from many advocates and developers at the hearing who are frustrated with this latest delay and skeptical about the long term success of the pilot.\nThe current pilot program was meant to expand solar access to Connecticut energy customers who can’t put solar on their own roof, but it contained flaws that have prevented any development to date. As set out in the legislation, the program has several poor design elements and a goal too small to draw significant private sector interest. Below are statements from stakeholders in Connecticut’s clean energy economy:\n“For years, Connecticut has missed out on the opportunity to bring solar energy choices to all consumers and more clean energy jobs to the state,” said Sean Garren, Northeast Regional director for VoteSolar. “Connecticut’s lackluster community solar program hasn’t unlocked the benefits of solar access for a single resident to date due to poor design and a lack of ambition at the scale needed, brought about by the electric utilities’ intervention. We’re calling on the legislature to catch up to the rest of New England — and the nation — with a smart, well-structured community solar program designed to serve consumers statewide.”\n“Two years of foot dragging and refusal by the Department of Energy and Environmental Protection to follow the law and implement a community solar program is preventing tens of thousands of Connecticut families from gaining access to clean, affordable, secure solar power,” said Chris Phelps, State Director for Environment Connecticut. “Community solar is helping other states accelerate solar growth, create jobs, and cut pollution. Connecticut policy makers should take action now to create a bold community solar program.”\n“Shared solar programs have been sweeping the nation for the last decade, but Connecticut has been left in the shade — losing out on healthier air, investment dollars, and green jobs that would accompany a full-scale, statewide shared solar program,” said Claire Coleman, Climate and Energy Attorney for Connecticut Fund for the Environment. “DEEP’s decision to start over with the already overly-restrictive shared solar pilot puts Connecticut further in the dark. Our climate and economy cannot wait any longer. Connecticut’s leaders must move quickly to ramp up in-state renewables through a full-scale shared solar program if Connecticut is going to have any chance of meeting its obligations under the Global Warming Solutions Act to reduce greenhouse gas emissions.”\nVote Solar is a nonprofit organization working to foster economic development and energy independence by bringing solar energy to the mainstream nationwide. Learn more at votesolar.org.',
    "The second text elaborates on the first by providing details about the specific context of the energy advocates' call for renewable growth. It identifies the advocates (CFE, VoteSolar, Environment Connecticut), the specific renewable energy program (community solar), and the reasons for their call, including program delays and design flaws.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# 获取嵌入向量的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

高级用法

# 高级场景说明：可根据具体需求对模型进行进一步的微调或与其他模型结合使用，以满足更复杂的任务需求。
# 这里假设我们有一个自定义的数据集，需要对模型进行微调
from sentence_transformers import SentenceTransformer, InputExample, losses
from torch.utils.data import DataLoader

# 加载预训练模型
model = SentenceTransformer("bwang0911/reasoning-bert-ccnews")

# 自定义数据集
train_examples = [
    InputExample(texts=['This is a sample sentence', 'Each sentence is converted'], label=0.8),
    InputExample(texts=['This is another sample sentence', 'Each sentence is transformed'], label=0.7)
]

# 创建数据加载器
train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=2)

# 定义损失函数
train_loss = losses.CosineSimilarityLoss(model)

# 微调模型
model.fit(train_objectives=[(train_dataloader, train_loss)], epochs=1, warmup_steps=100)

# 使用微调后的模型进行推理
sentences = [
    'This is a test sentence',
    'This is a similar test sentence'
]
embeddings = model.encode(sentences)
print(embeddings.shape)

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers Documentation
• 仓库：Sentence Transformers on GitHub
• Hugging Face：Sentence Transformers on Hugging Face

完整模型架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 196, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

评估指标

信息检索

在 mteb/nfcorpus、mteb/trec-covid、mteb/fiqa 和 mteb/quora 数据集上进行评估，使用 InformationRetrievalEvaluator 进行评估。

训练详情

训练数据集

• reason_unfiltered：reason_unfiltered 数据集，版本为 2e4fb05，包含44,978个训练样本。
• 数据集列：包含 title、body 和 reason 列。
• 近似统计信息：基于前1000个样本的统计信息如下：

  	标题	正文	原因
  类型	字符串	字符串	字符串
  详情	最小值：6个词元 平均值：15.34个词元 最大值：42个词元	最小值：21个词元 平均值：178.04个词元 最大值：196个词元	最小值：28个词元 平均值：59.19个词元 最大值：88个词元

• 样本示例：

  标题	正文	原因
  Fight Leaves Wayne Simmonds Shirtless	Reed Saxon/AP Images Kevin Bieksa and Wayne Simmonds dropped the gloves just 95 seconds into last night’s 4-3 Ducks shootout win over the Flyers, and Bieksa immediately yanked his opponent’s jersey over his head, to the delight of the crowd and to grins from Simmonds and the officials. That’s not supposed to happen. NHL players wear something called a fight strap, which binds the back of the jersey to the pants, preventing the jersey from being pulled off. (Losing a jersey is an advantage in a fight, as it gives the shirtless player’s opponent nothing to grab on to. Sabres enforcer Rob Ray was notorious for losing his gear in a fight, occasionally taking it off himself before clinching.) Any player who engaged in a fight without wearing a fight strap is subject to an automatic game misconduct. Advertisement Simmonds wasn’t ejected, though; at the one-minute mark of the video above, you can see he did have his fight strap properly attached. It just broke, which happens on occasion.	The article describes a hockey fight involving Wayne Simmonds, confirming the title's claim. It details the fight, including Simmonds' jersey being pulled off, and explains the rules and context around the incident, directly elaborating on the event suggested by the title.
  Merck CEO Kenneth Frazier ditches Trump over Charlottesville silence	Merck CEO Kenneth C. Frazier resigned from the president’s council on manufacturing Monday in direct protest of President Donald Trump’s lack of condemnation of white nationalist actions in Charlottesville, Va. over the weekend. In a statement, Frazier, who is African-American, said he believes the country’s strength comes from the diversity of its citizens and that he feels personally compelled to stand up for that diversity and against intolerance. “America’s leaders must honor our fundamental values by clearly rejecting expressions of hatred, bigotry and group supremacy, which run counter to the American ideal that all people are created equal,” he wrote. “As CEO of Merck, and as a matter of personal conscience, I feel a responsibility to take a stand against intolerance and extremism.” RELATED: At least one death has been confirmed after a car plowed into a crowd of protesters in Charlottesville Trump immediately fired back at Frazier on Twitter, saying the Merck CEO now “will have...	The second text provides a detailed elaboration of the first. It explains the context of Kenneth Frazier's resignation, the reasons behind it (Trump's silence on Charlottesville), and includes Frazier's statement. It also provides additional background information about Frazier and the President's Manufacturing Council.
  Lightning's Braydon Coburn: Joining road trip	Coburn (lower body) will travel with the team on its upcoming four-game road trip and is hoping to play at some point in the second half of the trip, Bryan Burns of the Lightning's official site reports. The veteran blueliner is yet to play in the month of December, having already missed four games. However, the fact that Coburn is traveling with the team and has been given a chance to play at some point within the next week will be music to the ears of fantasy owners who benefited from Coburn's surprising production -- seven points in 25 games -- earlier in the season. Keep an eye out for updates as the trip progresses.	The second text elaborates on the first by providing details about Braydon Coburn's situation. It specifies that he will join the team on a road trip and offers context about his injury, recovery timeline, and potential for playing, directly expanding on the initial announcement.

• 损失函数：使用 ReasoningGuidedRankingLoss，参数如下：

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

训练超参数

• 非默认超参数：
  • eval_strategy: steps
  • per_device_train_batch_size: 256
  • learning_rate: 1e-05
  • warmup_ratio: 0.1
  • fp16: True
  • batch_sampler: no_duplicates

所有超参数

训练日志

框架版本

• Python: 3.10.12
• Sentence Transformers: 3.5.0.dev0
• Transformers: 4.50.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.4.1
• Tokenizers: 0.21.1

📄 许可证

文档中未提及相关许可证信息。

📖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}