Promptclue Base

🚀 PromptCLUE: Zero-shot Learning Model for All Chinese Tasks

PromptCLUE is a zero-shot learning model for all Chinese tasks. This model is pre-trained on 100 billion Chinese tokens, with a cumulative learning of 1.5 trillion Chinese tokens. It is also trained on hundreds of tasks using the Prompt approach. For understanding tasks such as classification, sentiment analysis, and extraction, it supports custom label systems. For various generation tasks, it can perform free sampling generation.

• Online Demo
• Use the clueai Toolkit and API (Large Version)
• Github Project
• Try on Colab

📦 Installation

Load the model

# Load the model
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("ClueAI/PromptCLUE-base")
model = T5ForConditionalGeneration.from_pretrained("ClueAI/PromptCLUE-base")

Use the model for prediction and inference

import torch
#device = torch.device('cpu')
device = torch.device('cuda')
model.to(device)
def preprocess(text):
  return text.replace("\n", "_")

def postprocess(text):
  return text.replace("_", "\n")

def answer(text, sample=False, top_p=0.8):
  '''sample: Whether to sample. For generation tasks, set it to True;
  top_p: Between 0-1, the more diverse the generated content'''
  text = preprocess(text)
  encoding = tokenizer(text=[text], truncation=True, padding=True, max_length=768, return_tensors="pt").to(device) 
  if not sample:
    out = model.generate(**encoding, return_dict_in_generate=True, output_scores=False, max_length=128, num_beams=4, length_penalty=0.6)
  else:
    out = model.generate(**encoding, return_dict_in_generate=True, output_scores=False, max_length=64, do_sample=True, top_p=top_p)
  out_text = tokenizer.batch_decode(out["sequences"], skip_special_tokens=True)
  return postprocess(out_text[0])

💻 Usage Examples

Basic Usage

The following are some example inputs and outputs:

News Classification (classify)

Input:
Classification Task:
Fund Taihe dropped 7.15% due to a too-low discount rate. Reporter Zhu Jingfeng from Securities Times reported that because its discount rate was the lowest among closed-end funds in the market, Fund Taihe was heavily sold by investors yesterday, with a decline of 7.15%, far exceeding the market. The market showed that Fund Taihe opened higher with the market, then began to fluctuate downward, and accelerated its decline in the afternoon with almost no decent rebound. As of the close, while the CSI 300 Index only fell 2.56%, Fund Taihe closed down 7.15%, the largest decline among all closed-end funds. Most closed-end funds declined about 2% yesterday.
Options: Finance, Entertainment, Politics, Stocks
Answer:

Model output:
Finance

Intention Classification (classify)

Input:
Intention Classification:
Help me book a room in Pudong, Shanghai on Sunday.
Options: Alarm, Literature, Hotel, Art, Sports, Health, Weather, Others
Answer:

Model output:
Hotel

Sentiment Analysis (classify)

Input:
Sentiment Analysis:
This looks okay, but actually I don't like it.
Options: Positive, Negative
Answer:

Model output:
Negative

Reasoning (generate)

Input:
Please infer the relationship between the context:
Premise: Sorry, that's how things are.
Hypothesis: That's how things are, no need to apologize.
Options: Contradictory, Entailed, Neutral
Answer:

Model output:
Contradictory

Reading Comprehension (generate)

Input:
Read the passage and give the answer:
The Hong Kong Dollar Effective Exchange Rate Index (HKDEERI), also known as the Hong Kong Dollar Exchange Rate Index, is an index compiled by the Hong Kong Government Statistics Department to reflect the weighted average change in the nominal effective exchange rate of the Hong Kong Dollar against the currencies of Hong Kong's major trading partners. The weighting is based on the average trade pattern from 1999 to 2000, but the government has not公布 the detailed calculation formula. The base date of the old HKDEERI was January 1, 2000, with a base value of 100 points. Since January 3, 2012, the new series of HKDEERI (including 15 currencies and with January 2010 = 100) has replaced the old series. The main function of the HKDEERI is to reflect the change in the prices of Hong Kong's goods and services relative to its major trading partners and is usually regarded as an indicator of Hong Kong's price competitiveness.
Question: How is the weighting of the HKDEERI determined?
Answer:

Model output:
Based on the average trade pattern from 1999 to 2000

Reading Comprehension - Free Style (generate)

Input:
Read the following dialogue and answer the question.
Man: Why are you so late for work today? Woman: I worked late yesterday and I'm also sick. Man: Then go back and rest. I'll help you ask for leave. Woman: Thank you.
Question: How is the woman?
Options: Working, Sick, On the phone, Going on a business trip.
Answer:

Model output:
Sick

Summarization (generate)

Input:
Generate a summary for the following article:
At 12:52 on September 5th, Beijing time, a 6.8-magnitude earthquake occurred in Luding County, Ganzi Tibetan Autonomous Prefecture, Sichuan Province. After the earthquake, the leaders attached great importance and gave important instructions, requiring that saving lives be the top priority, making every effort to rescue the affected people, and minimizing casualties.
Answer:

Model output:
A 6.8-magnitude earthquake occurred in Ganzi, Sichuan

Translation - Chinese to English (generate)

Input:
Translate into English:
The speaker went to Taiwan, and the Chinese people are very angry.
Answer:

Model output:
The secretary went to Taiwan and the Chinese people were angry.

Translation - English to Chinese (generate)

Input:
Translate into Chinese:
This is a dialogue robot that can talk to people.
Answer:

Model output:
这是一台可以与人交谈的对话机器人。

General Information Extraction (generate)

Input:
Information Extraction:
According to Xinhua News Agency, the Qingcheng District Government of Qingyuan City, Guangdong Province announced yesterday that Chen Bai, the director of the Qingcheng District Environmental Protection Bureau who was recently reported for allegedly extorting enterprises and saying "I can shut down a factory in a minute", has been removed from his position as a member of the Qingcheng District Party Committee.
Question: Institution Name, Person Name, Position
Answer:

Model output:
Institution Name: Xinhua News Agency, Qingcheng District Government, Qingcheng District Environmental Protection Bureau, Qingcheng District Party Committee
Person Name: Chen Bai
Position: Director, Member of the Party Committee

Resume Information Extraction (generate)

Input:
Read the text and extract key information:
Zhang Xuanwu was born in 2000, has Chinese nationality, no overseas residency rights, a doctoral degree, and is currently the technical director of Hangzhou Xiansuo Technology.
Question: Institution, Person Name, Position, Hometown, Major, Nationality, Education, Ethnicity
Answer:

Model output:
Person Name: Zhang Xuanwu
Position: Technical Director of Hangzhou Xiansuo Technology
Nationality: Chinese Nationality
Education: Doctoral Degree

Medical Information Extraction (generate)

Input:
Extract information from the text:
The patient's mental state is okay, diet is okay, sleep is okay, and bowel movements are normal. After comprehensive treatment, the patient's dizziness symptoms have been alleviated. The patient continues to take drugs to improve cerebral blood vessels and adjust blood pressure.
Question: Symptom, Treatment, Examination, Body Part, Disease
Answer:

Model output:
Symptom: Dizziness
Treatment: Drugs to improve cerebral blood vessels and adjust blood pressure
Body Part: Bowel Movements

E-commerce Customer Demand Analysis (classify)

Input:
E-commerce Customer Appeal Classification:
I received it, but it doesn't fit well. Can I return or exchange it?
Options: Buyer asks if the product supports Huabei payment, Buyer indicates to bookmark and follow the store, Buyer asks about return and exchange rules, Buyer needs product recommendations
Answer:

Model output:
Buyer asks about return and exchange rules

Medical Semantic Similarity (classify)

Input:
Do the following sentences have the same semantics?
Text 1: What should I do if I have numbness in my legs due to diabetes?
Text 2: How to control the lifestyle for diabetes?
Options: Similar, Not Similar
Answer:

Model output:
Not Similar

Question Generation (generate)

Input:
Question Generation:
On September 22, 2022, according to China News Service, the Ministry of Commerce held a regular press conference. Spokesperson Shu Jueting said that from January to August this year, China's actual use of foreign capital was 138.4 billion US dollars, a year-on-year increase of 20.2%. Among them, EU investment in China increased by 123.7% (including investment data through free ports). This fully shows that foreign investors, including those from the EU, continue to be optimistic about the Chinese market and hope to continue to deepen investment cooperation with China.
Answer:

Model output:
1. According to the Ministry of Commerce spokesperson Shu Jueting, from January to August last year, China's actual use of foreign capital was 138.4 billion US dollars, a year-on-year increase of 20.2%. How much did EU investment in China increase?
2. Did the increase in EU investment in China meet expectations?
3. What achievements has the EU made in investment cooperation with China?

Anaphora Resolution (generate)

Input:
Anaphora Resolution:
Passage:
Shaoping and Runye entered the yard of her second uncle's house. Runsheng came over and said to him (pronoun), "I went to your dormitory to look for you twice. Where did you go?"
Question: Who does the pronoun "him" refer to?
Answer:

Model output:
Shaoping

Keyword Extraction (generate)

Input:
Extract keywords:
On the 21st local time, the US Federal Reserve announced a 75-basis-point interest rate hike, raising the target range for the federal funds rate to between 3.00% and 3.25%, in line with market expectations. This is the Fed's fifth rate hike this year and the third consecutive increase, setting the largest intensive rate-hiking pace since 1981.
Keywords:

Model output:
Federal Reserve, Interest Rate Target Range, Interest Rate Hike, Basis Point

Sentiment Tendency (classify)

What kind of emotion is contained in the text:
Such a cute handsome guy, I'm in love...
Options: Disgust, Like, Happy, Sad, Surprised, Angry, Afraid
Answer:

Model output:
Like

For more example functions and the model, please refer to ClueAI.

📄 License

The license of this project is creativeml-openrail-m.