Clinical Trials All MiniLM L6 V2

🚀 基於sentence-transformers/all-MiniLM-L6-v2的句子轉換器

本項目是一個基於sentence-transformers框架，從sentence-transformers/all-MiniLM-L6-v2微調而來的模型。它能夠將句子和段落映射到一個384維的密集向量空間，可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等任務。

🚀 快速開始

安裝依賴

首先，你需要安裝sentence-transformers庫：

pip install -U sentence-transformers

運行推理

安裝完成後，你可以加載該模型並進行推理：

from sentence_transformers import SentenceTransformer

# 從🤗 Hub下載模型
model = SentenceTransformer("sentence_transformers_model_id")
# 定義待編碼的句子
sentences = [
    'open label trial safety efficacy sym001 treatment immune thrombocytopenic purpura itp. If your serious vaccine-induced adverse event has been entered in the CDC Vaccine Adverse Event Reporting System (VAERS) we are interested in enrolling you for this study in order to log your symptoms.\n\nThe primary goal of this study is to create a national database and gather vaccine-associated serious adverse events/injury data from newly vaccinated individuals in the US in order to identify the possible underlying causal relationships and plausible underlying biological mechanisms. The project aims to identify the genetic determinants of vaccine-induced adverse response by studying host genetics. We plan to use whole genome sequencing to identify single nucleotide polymorphisms associated with cardiovascular, neurological, gastrointestinal, musculoskeletal and immunological symptoms induced by vaccine administration.\n\nThe secondary goal is to establish criteria that enable classification of vaccine-induced adverse events/injuries compare data from our database with the official Vaccine Injury Table National Vaccine Injury Compensation Program on or after March 21, 2017.\n\nThe tertiary goal is to establish a database to gather detailed long-term adverse reaction data from subjects enrolled in FDA Emergency Use Authorized vaccine clinical trials.',
    'Immune Thrombocytopenic Purpura inclusion criterion confirm presence thrombocytopenia platelet count   30000mm3 predose visit history isolated itp rhdpositive serology previous treatment response line therapy itpexclusion criterion know clinical picture suggestive cause thrombocytopenia especially systematic lupus erythematosusantiphospholipid syndrome evans syndrome immunodeficiency state lymphoproliferative disorder liver diseaseingestion drug quinidinequinine heparin sulfonamide hereditary thrombocytopenia confirm relevant laboratory finding suspect infection hiv hepatitis c h pylori clinical splenomegaly history abnormal bone marrow examination ongoing haemorrhage correspond grade 3 4 bleeding scale underlie haemolytic condition history splenectomy subject pregnant breast feeding intend pregnant incidence severity adverse event aes include adverse event saes 6 week post dose measurement platelet count day 1 week 6',
    'Multiple System Atrophy inclusion criteriadiagnostic1 participant diagnosis possible probable msa modify gilman et al 2008 diagnostic criteria2 participant onset msa symptom occur 4 year screen assess investigator3 evidence msa specific symptom deficit measure umsars scaleexclusion criteriamedical history1 participant contraindication study proceduresdiagnostic assessments1 presence confound diagnosis andor condition affect participant safety study investigator judgement2 participant participation previous study diseasemodifye therapy prove receipt active treatment compromise interpretability datum present study consultation medical monitor designeeother1 participant participate study investigate active passive immunization αsynuclein αsyn progressive disease pd msa immunoglobulin g therapy 6 month screen change baseline modify unified multiple system atrophy rating scale umsar week 52 umsar historical review 11item scale adapt unify parkinson disease rating scale updrs assess activity relate motor disability relate autonomic dysfunction item score 0 normal 3 severe total score sum score domain range 0 33 high score mean poor health 52 week change baseline 11item umsar week 52 11 item umsar include 11 item ii assess motor autonomic disability umsar historical review assess activity relate motor disability autonomic dysfunction umsar ii motor examination measure functional impairment specific parkinsonian cerebellar feature item score 0 normal 4 severe total score sum score domain range 0 44 high score mean poor health 52 weekschange baseline umsar total score umsar   ii week 52 umsar total scale consist item umsars part ii umsar historical review 12item scale assess activity relate motor disability autonomic dysfunction item score 0 normal 4 severe umsar ii motor examination 14item scale measure functional impairment eg speech rapid alternate movement hand finger tap leg agility select complex movement specific parkinsonian tremor rest cerebellar ocular motor dysfunction heelshin test feature item score 0 normal 4 severe 52 weekschange baseline umsars week 52 umsar historical review modified 11item scale adapt updrs assess activity relate motor disability 8 item 4 novel item relate autonomic dysfunction item score 0 normal 4 severe total score sum score item range 0 44 high score mean poor health 52 weekschange baseline umsars ii week 52 umsar ii motor examination 14item scale item eg speech rapid alternate movement hand finger tap leg agility measure functional impairment select complex movement item directly refer specific parkinsonian tremor rest cerebellar ocular motor dysfunction heelshin test feature motor examination section umsar base modify updrsiii item addition novel item heelkneeshin ataxia item score 0 normal 4 severe total score sum score item range 0 56 high score mean poor health 52 weeksclinical global impressionseverity cgis score cgis assess clinicians impression participant clinical condition clinician use total clinical experience participant population rate current severity participant illness 7point scale range 1 normal ill 7 extremely ill participant high score mean well health 52 weekschange baseline scale outcome parkinson disease   autonomic dysfunction scopaaut total score scopaaut patientreported outcome assess autonomic function autonomic function critical symptom domain msa scale selfcomplete participant consist 25 item assess follow domain gastrointestinal 7 item urinary 6 item cardiovascular 3 item thermoregulatory 4 item pupillomotor 1 item sexual 2 item man 2 item woman score item range 0 experience symptom 3 experience symptom total composite score include domain report score range 0 symptom 69 high burden symptom 52 weeksoverall survival os os define time day study drug administration death cause 52 weekschange baseline level cerebrospinal fluid csf free alphasynuclein αsyn 52 weekscmax maximum observe serum concentration tak341 predose day 1 29 57 85 169 253 337 multiple timepoint 24 hour postdose day 1 57 85 169 337 anytime day 365 427 early termination day 57 applicable early pk cohortstmax time occurrence cmax serum tak341 predose day 1 29 57 85 169 253 337 multiple timepoint 24 hour postdose day 1 57 85 169 337 anytime day 365 427 early termination day 57 applicable early pk cohortsaucτ area concentrationtime curve dose interval serum tak341 predose day 1 29 57 85 169 253 337 multiple timepoint 24 hour postdose day 1 57 85 169 337 anytime day 365 427 early termination day 57 applicable early pk cohortscsf concentration tak341 lumbar puncture csf sampling perform predose day 1 85 applicable early pk cohort 365number participant adverse event ae adverse event ae define untoward medical occurrence participant administer pharmaceutical product untoward medical occurrence necessarily causal relationship treatment datum report number participant analyze safety parameter include clinically significant abnormal value clinical laboratory evaluation vital sign ecg parameters physical examination neurological examination columbiasuicide severity rating scale cssrs 52 weeksnumber participant antidrug antibody 52 week',
]
# 對句子進行編碼
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# 計算嵌入向量之間的相似度分數
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

✨ 主要特性

• 語義表示能力：能夠將句子和段落映射到一個384維的密集向量空間，從而捕捉文本的語義信息。
• 多任務適用性：可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等多種自然語言處理任務。
• 微調靈活性：基於sentence-transformers框架，可在自定義數據集上進行微調，以適應特定的任務需求。

📦 安裝指南

安裝本模型所需的依賴庫，可使用以下命令：

pip install -U sentence-transformers

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers Documentation
• 倉庫：Sentence Transformers on GitHub
• Hugging Face：Sentence Transformers on Hugging Face

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

訓練詳情

訓練數據集

未命名數據集

• 規模：92,934個訓練樣本
• 列信息：包含<code>Text1</code>和<code>Text2</code>兩列
• 基於前1000個樣本的近似統計信息：

  	Text1	Text2
  類型	字符串	字符串
  詳情	最小值：29個詞元 平均值：104.36個詞元 最大值：256個詞元	最小值：8個詞元 平均值：227.98個詞元 最大值：256個詞元

• 樣本示例：

  Text1	Text2
  study people normal kidney function people reduce kidney function test bi 1467335 process body. The primary objective of the current study is to investigate the influence of moderate renal impairment on the pharmacokinetics of multiple doses in comparison to a matched control group with normal renal function.	Renal Insufficiency
  16w interventional study titration doseefficacy assessment exelon chinese alzheimers disease patient. To investigate the efficacy of Exelon capsule at maximal tolerated dose in mild to moderate Chinese AD patients via dosage titration from 3mg/d to 12mg/d in a 16 weeks duration	Alzheimer's Disease key inclusion criterion diagnosis dementia alzheimers type accord dsmiv criterion clinical diagnosis probable ad accord nincdsadrda criteria mmse score 10 26 treatment naïve patient stop donepezil galantamine huperzine memantine 2 week stable medical condition sign inform consent form patient hisher legal guardiankey exclusion criterion severe ad patient history cerebrovascular disease active uncontrolled epilepsy active hypothyroidism asthma cns infection neurodegenerative disorder advanced severe progressive unstable medical condition attend clinical trial take clinical trial drug score 4 modify hachinski ischemic scale mhis patient achei memantine mean change baseline alzheimer disease assessment scale cognitive subscale adascog alzheimer disease assessment scale cognitive subscale adascog measure change cognitive function alzheimer disease assessment scale adas scale measure specific cognitive behavior disorder alzheimer disease ad patient alzheimer di...
  case series saneso 360 gastroscope. To confirm the procedural performance of the Saneso 360° gastroscope in Esophago-gastro-duodenoscopy (EGD) procedures.	EGD Procedure inclusion criterion 18 74 year age willing able comply study procedure provide write inform consent participate study schedule clinically indicate routine egd procedure asa class 13exclusion criterion alter esophageal gastric duodenal anatomy pregnant woman child 18 year age adult 75 year age subject routine endoscopic procedure contraindicate comorbid medical condition patient currently enrol investigational study directly interfere current study prior write approval sponsor asa class 45 successful egd procedure success assess end procedure 1 procedure success define successful intubation portion duodenum photograph portion duodenum take 24 hour study day endoscopist qualitative rating saneso 360 gastroscope endoscopist rate experience saneso 360 gastroscope immediately follow completion study procedure 1 fivepoint likert scale 5 excellent 4 good 3 acceptable 2 difficult 1unacceptable 24 hour study dayendoscopist qualitative rating saneso 360 gastroscope compare past...

• 損失函數：使用MatryoshkaLoss，參數如下：

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          384,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

訓練超參數

非默認超參數

• per_device_train_batch_size：16
• learning_rate：2e-05
• num_train_epochs：1
• warmup_ratio：0.1
• fp16：True
• batch_sampler：no_duplicates

所有超參數

訓練日誌

框架版本

• Python：3.10.12
• Sentence Transformers：3.3.1
• Transformers：4.48.1
• PyTorch：2.1.0a0+32f93b1
• Accelerate：1.3.0
• Datasets：3.2.0
• Tokenizers：0.21.0

📄 許可證

文檔中未提及許可證相關信息。

🔧 技術細節

本模型基於sentence-transformers框架，從sentence-transformers/all-MiniLM-L6-v2微調而來。模型結構包含一個Transformer層、一個池化層和一個歸一化層。在訓練過程中，使用了MatryoshkaLoss損失函數，並設置了一系列超參數進行優化。

📖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}