đ FastChat-T5 Model Card
FastChat-T5 is an open - source chatbot model. It is fine - tuned from Flan - t5 - xl on user - shared conversations, offering capabilities for commercial use and research in the field of language processing.
đ Quick Start
For more information about using the FastChat - T5 model, please refer to the official repository: [FastChat](https://github.com/lm - sys/FastChat#FastChat - T5).
⨠Features
- Open - source: Based on the Flan - t5 - xl architecture, it is trained on user - shared conversations from ShareGPT.
- Encoder - decoder architecture: Can autoregressively generate responses to user inputs.
- Versatile use cases: Suitable for both commercial applications of large language models and chatbots, as well as research purposes.
đĻ Installation
No specific installation steps are provided in the original document.
đ Documentation
Model Details
| Property |
Details |
| Model Type |
FastChat - T5 is an open - source chatbot trained by fine - tuning Flan - t5 - xl (3B parameters) on user - shared conversations collected from ShareGPT. It is based on an encoder - decoder transformer architecture and can autoregressively generate responses to users' inputs. |
| Model Date |
FastChat - T5 was trained on April 2023. |
| Organizations Developing the Model |
The FastChat developers, primarily Dacheng Li, Lianmin Zheng and Hao Zhang. |
| Paper or Resources for More Information |
[FastChat Repository](https://github.com/lm - sys/FastChat#FastChat - T5) |
| License |
Apache License 2.0 |
| Where to Send Questions or Comments about the Model |
[FastChat Issues](https://github.com/lm - sys/FastChat/issues) |
Intended Use
- Primary Intended Uses: The primary use of FastChat - T5 is the commercial usage of large language models and chatbots. It can also be used for research purposes.
- Primary Intended Users: The primary intended users of the model are entrepreneurs and researchers in natural language processing, machine learning, and artificial intelligence.
Training Dataset
70K conversations collected from ShareGPT.com.
Training Details
It processes the ShareGPT data in the form of question - answering. Each ChatGPT response is processed as an answer, and previous conversations between the user and the ChatGPT are processed as the question.
The encoder bi - directionally encodes a question into a hidden representation. The decoder uses cross - attention to attend to this representation while generating an answer uni - directionally from a start token.
This model is fine - tuned for 3 epochs, with a max learning rate 2e - 5, warmup ratio 0.03, and a cosine learning rate schedule.
Evaluation Dataset
A preliminary evaluation of the model quality is conducted by creating a set of 80 diverse questions and utilizing GPT - 4 to judge the model outputs. See Vicuna for more details.
đ License
This model is released under the Apache License 2.0.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
