Cnmoro TinyLlama ContextQuestionPair Classifier Reranker Gguf

🚀 TinyLlama-ContextQuestionPair-Classifier-Reranker - GGUF

本项目提供了TinyLlama-ContextQuestionPair-Classifier-Reranker模型的量化版本（GGUF格式），可用于文本排序任务。该模型由Richard Erkhov进行量化处理，原始模型由https://huggingface.co/cnmoro/ 创建。

项目链接

• Github
• Discord
• Request more models

✨ 主要特性

• 多语言支持：支持英语（en）和葡萄牙语（pt）。
• 分类与重排序：适用于分类和重排序任务，特别是在RAG（检索增强生成）场景中。

📦 模型信息

量化模型列表

💻 使用示例

基础用法

template = """<s><|system|>
You are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>
<|user|>
Context:
{Text}

Question:
{Prompt}</s>
<|assistant|>
"""

# Output should be:

{"relevant": true}

# or

{"relevant": false}

示例输入输出

<s><|system|>
You are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>
<|user|>
Context:
old. NFT were observed in almost all patients over 60 years of age, but the incidence was low.
Many ubiquitin-positive small-sized granules were observed in the second and third layer of the parahippocampal gyrus of aged patients,
and the incidence rose with increasing age. On the other hand, few of these granules were in patients with Alzheimer\'s type dementia.
Granulovacuolar degeneration was examined. Many centrally-located granules were positive for ubiquitin. Based on electron microscopic
observation of these granules at several stages, the granules were thought to be a type of autophagosome. During the first stage of
granulovacuolar degeneration, electron-dense materials appeared in the cytoplasm, following which they were surrounded by smooth cytoplasm,
following which they were surrounded by smooth endoplasmic reticulum. Analytical electron microscopy disclosed that the granules contained
some aluminium. Several senile changes in the central nervous system in cadavers were examined. The pattern of extension of Alzheimer\'s
neurofibrillary tangles (NFT) and senile plaques (SP) in the olfactory bulbs of 100 specimens was examined during routine autopsy by
immunohistochemical staining. NFT were first observed in the anterior olfactory nucleus after the age of 60, and incidence rose with
increasing age. Senile plaques were found in the nucleus when there were many SP in the cerebral cortex. Of 25 non-demented amyotrophic
lateral sclerosis patients, SP were found in the cerebral cortices of 10, and 9 of 10 were over 60 years old. NFT were observed in almost
all patients over

Question:
What is granulovacuolar degeneration and what was its observation on electron microscopy?</s>
<|assistant|>
{"relevant": true}</s>

vLLM推荐请求参数

prompt = "<s><|system|>\nYou are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>\n<|user|>\nContext:\nConhecida como missão de imagem de raios-x e espectroscopia (da sigla em inglês XRISM), a estratégia é utilizar o telescópio para ampliar os estudos da humanidade a níveis celestiais com uma fração dos pixels da tela de um Gameboy original, lançado em 1989. Isso é possível por meio de uma ferramenta chamada “Resolve”. Apesar de utilizar a medição em pixels, a tecnologia é bastante diferente de uma câmera. Com um conjunto de microcalorímetros de seis pixels quadrados que mede 0,5 cm², ela detecta a temperatura de cada raio-x que o atinge. Como funciona o Resolve do telescópio XRISM? Cientista do projeto XRISM da NASA, Brian Williams explicou em um comunicado o funcionamento do telescópio. “Chamamos o Resolve de espectrômetro de microcalorímetros porque cada um de seus 36 pixels está medindo pequenas quantidades de calor entregues por cada raio-x recebido, nos permitindo ver as impressões digitais químicas dos elementos que compõem as fontes com detalhes sem precedentes”.\n\nQuestion:\nQual é a sigla em alemão mencionada?</s>\n<|assistant|>\n{\"relevant\":"

headers = {
    "Accept": "text/event-stream",
    "Authorization": "Bearer EMPTY"
}

body = {
    "model": model,
    "prompt": [prompt],
    "best_of": 5,
    "max_tokens": 1,
    "temperature": 0,
    "top_p": 1,
    "use_beam_search": True,
    "top_k": -1,
    "min_p": 0,
    "repetition_penalty": 1,
    "length_penalty": 1,
    "min_tokens": 1,
    "logprobs": 1
}

result = requests.post(base_uri, headers=headers, json=body)
result = result.json()

boolean_response = bool(eval(json_result['choices'][0]['text'].strip().title()))
print(boolean_response)

📄 许可证

本项目原始模型采用 cc-by-nc-2.0 许可证。