đ li-14b-v0.4-slerp0.1
This is a merged pre - trained language model, created by combining multiple base models using the mergekit tool, aiming to provide better performance in text - generation tasks.
đ Quick Start
This README provides detailed information about the merged model li-14b-v0.4-slerp0.1, including its base models, merge details, evaluation results, and information about the company behind it.
⨠Features
- Model Merging: Utilizes the mergekit tool to merge pre - trained language models.
- Diverse Evaluation: Evaluated on multiple datasets such as IFEval, BBH, MATH, GPQA, MuSR, and MMLU - PRO, providing a comprehensive view of its performance.
- Company Backing: Supported by Century Innovation, a leading "Industrial Internet" printing enterprise with a focus on technological innovation.
đĻ Installation
No installation steps are provided in the original document, so this section is skipped.
đģ Usage Examples
No code examples are provided in the original document, so this section is skipped.
đ Documentation
Merge Details
Merge Method
This model was merged using the SLERP merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
base_model: wanlige/li-14b-v0.4
merge_method: slerp
tokenizer_source: base
dtype: float32
out_dtype: bfloat16
parameters:
t:
- filter: self_attn
value: [ 0.00, 0.50, 0.30, 0.70, 1.00 ]
- filter: mlp
value: [ 1.00, 0.50, 0.70, 0.30, 0.00 ]
- value: [ 0.00, 0.00, 0.00, 0.00, 0.04, 0.08, 0.12, 0.16, 0.24, 0.32, 0.40, 0.48, 0.56, 0.64, 0.72, 0.72, 0.72, 0.72, 0.72, 0.72, 0.72, 0.72, 0.64, 0.56, 0.48 ]
slices:
- sources:
- model: wanlige/li-14b-v0.4
layer_range: [ 0, 48 ]
- model: sthenno-com/miscii-14b-0218
layer_range: [ 0, 48 ]
Detailed results can be found here
| Metric |
Value |
| Avg. |
42.91 |
| IFEval (0 - Shot) |
79.23 |
| BBH (3 - Shot) |
50.88 |
| MATH Lvl 5 (4 - Shot) |
53.32 |
| GPQA (0 - shot) |
14.54 |
| MuSR (0 - shot) |
11.75 |
| MMLU - PRO (5 - shot) |
47.71 |
Company Information
Established on March 9, 2001, and headquartered in Jinan, Shandong Province, Century Innovation has grown over the past two decades by focusing on technological innovation. The company has achieved deep integration of the Internet with the traditional printing industry, pioneering new models and business formats distinct from conventional printing practices.
Century Innovation specializes in the research, design, production, and sales of customized imaging, commercial printing, and packaging products. By combining the Internet, digitalization, automation, and intelligent technologies with the printing industry, the company enables relatively standardized and scalable production for small - batch personalized custom orders. This approach aims to meet the needs of individual consumers and various enterprise users for small - batch customization, providing users with one - stop, scenario - based custom printing services and achieving full - process intelligent manufacturing. As a result, Century Innovation has become a leading "Industrial Internet" printing enterprise.
In the future, Century Innovation will continue to increase investment in technology R & D, deeply integrate the Internet, big data, artificial intelligence, and other next - generation information technologies, and focus on cultivating specialized technical talent. The company will actively adopt digital and intelligent means to optimize innovative business processes and enhance user experience. Through multi - dimensional development, it aims to drive industry collaboration, promote the transformation of old and new drivers in the printing industry, and explore new directions for its growth.
To learn more, visit our official website: Century Innovation
đ§ Technical Details
No specific technical implementation details (more than 50 words) are provided in the original document, so this section is skipped.
đ License
No license information is provided in the original document, so this section is skipped.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
