Firefly Bloom 2b6

🚀 Firefly: A Chinese Conversational Large Language Model

Firefly is an open-source Chinese conversational large language model. It leverages Instruction Tuning on Chinese datasets for optimization. Techniques such as vocabulary pruning, ZeRO, and tensor parallelism are employed to effectively reduce GPU memory consumption and enhance training efficiency. During training, it uses a smaller number of model parameters and less computational resources.

We've constructed a wealth of data related to Chinese culture to boost the model's performance in this area, including couplets, poetry creation, classical Chinese translation, essays, and Jin Yong's novels.

The name "Firefly" is inspired by Du Mu's poem Autumn Eve: "By candlelight in autumn chill the painted screen grows cold; With silken fan she chases fireflies on and on." We hope this project can shine a gentle light, just like a firefly, and contribute to the Chinese NLP open-source community.

Ode to Fireflies
By Li Bai (Tang Dynasty)
The rain can't put out its light like a lamp;
The wind makes its color shine all the more.
If it could fly up into the sky,
It'd surely be a star beside the moon.

Project address: Firefly(流萤): A Chinese Conversational Large Language Model. If you find this model helpful, please like it and star our GitHub project.

✨ Features

• Dataset: firefly-train-1.1M, a high-quality Chinese multi-task instruction tuning dataset containing 1.1M samples. It covers 23 common Chinese NLP tasks, with manual writing of instruction templates for each task to ensure data quality and richness.
• Model Pruning: LLMPruner: A Large Language Model Pruning Tool. It uses vocabulary pruning technology to prune the weights of multilingual large language models. While retaining pre - trained knowledge, it effectively reduces the number of model parameters, improves training efficiency, and shares the weights of Bloom models with various parameter scales after pruning.
• Weight Sharing: Based on bloom-1b4-zh and bloom-2b6-zh, we conducted instruction tuning to obtain two Chinese models with different parameter scales: firefly-1b4 and firefly-2b6.
• Training Code: The training code is open - source, supporting large - model training strategies such as tensor parallelism, ZeRO, and Gemini heterogeneous memory space management. It can train models with 1B - 2B parameters using only one GPU (to be open - sourced after organization).

🚀 Quick Start

💻 Usage Examples

Basic Usage

from transformers import BloomTokenizerFast, BloomForCausalLM
device = 'cuda'
path = 'YeungNLP/firefly-2b6'

tokenizer = BloomTokenizerFast.from_pretrained(path)
model = BloomForCausalLM.from_pretrained(path)
model.eval()
model = model.to(device)
text = input('User：')
while True:
    text = '<s>{}</s></s>'.format(text)
    input_ids = tokenizer(text, return_tensors="pt").input_ids
    input_ids = input_ids.to(device)
    outputs = model.generate(input_ids, max_new_tokens=200, do_sample=True, top_p=0.7, temperature=0.35,
                             repetition_penalty=1.2, eos_token_id=tokenizer.eos_token_id)
    rets = tokenizer.batch_decode(outputs)
    output = rets[0].strip().replace(text, "").replace('</s>', "")
    print("Firefly：{}".format(output))
    text = input('User：')

🔧 Technical Details

During training, for each sample, we process the input and target into the following format and then feed it into the model:

<s>input</s></s>target</s>

We set the maximum input length of the model to 512. The input part does not participate in the loss calculation; only the loss function of the target part is calculated. The training hyperparameters are set as shown in the following table:

⚠️ Limitations and Usage Restrictions

After vocabulary pruning, the number of parameters in our model is only 1.4B and 2.6B, far less than models like ChatGPT and LLaMA, which have tens or hundreds of billions of parameters, and even far less than current mainstream models like Belle and ChatGLM, which have around 7B parameters. Therefore, there are still the following issues in performance:

• It may produce incorrect responses for factual knowledge.
• Since it has not undergone harmless fine - tuning, it may generate discriminatory, harmful, or unethical remarks.
• It still lacks capabilities in code generation and reasoning.

Due to the above limitations, we require that the code, data, and models of this project be used only for academic research, not for commercial purposes, and not for any activities that may cause harm to society.

📄 License

If you use the data, code, or models from this project, please cite it as follows:

@misc{Firefly,
  author = {Jianxin Yang},
  title = {Firefly(流萤): A Chinese Conversational Large Language Model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/yangjianxin1/Firefly}},
}

👀 Follow Us

🎉 Generation Results

Code Generation

firefly-2b6 has certain code generation capabilities.

• Write a bubble sort algorithm in Python.

def bubble_sort(arr):
    n = len(arr)
    for i in range(n):
        for j in range(0, n-i-1):
            if arr[j] > arr[j+1] :
                arr[j], arr[j+1] = arr[j+1], arr[j]
    return arr
# 测试
arr = [64, 34, 25, 12]
sorted_arr = bubble_sort(arr)
print("Original array：", arr)
print("Sorted array：", sorted_arr)

The time complexity of this algorithm is O(n^2).

• Implement a quick sort algorithm in Python. The input is an array, and the output is the sorted array.

def quick_sort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[len(arr) // 2]
    left = [x for x in arr if x < pivot]
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]
    return quick_sort(left) + middle + quick_sort(right)

This function takes a list of integers as input and sorts the list in ascending order. If the length of the list is less than or equal to 1, it returns the original list directly; otherwise, it uses the middle element as the pivot and recursively calls the quick sort algorithm on the remaining elements until the entire list is sorted.

• Write a binary search algorithm in Python.

def binary_search(arr, x):
    left = 0
    right = len(arr) - 1
    while left <= right:
        mid = (left + right) // 2
        if arr[mid] == x:
            return mid
        elif arr[mid] < x:
            left = mid + 1
        else:
            right = mid - 1
    return -1

This function takes two parameters, arr and the element x to be searched, and returns the index of the element closest to the target value in the array. If the target value does not exist, it returns -1.

• Write a function to calculate the number of even numbers in an array. The input is an array, and the output is the number of even numbers.

def count_even_numbers(nums):
    even_count = 0
    for num in nums:
        if num % 2 == 0:
            even_count += 1
    return even_count

This function first uses a for loop to iterate through the entire array and then checks if each element is an even number. If it is, it adds it to the even_count variable. Finally, it returns the value of even_count to get the number of even numbers in the array.

• Generate a login interface for an educational administration system using HTML. It should include username and password input boxes and a login button.

<html>
  <head>
    <title>Educational Administration System</title>
  </head>
  <body>
    <h1>Welcome to use this system!</h1>
    <form action='set_username' method='POST'>
      <label for="username">Please enter your username：</label>
      <input type="text" id="username" name="username"><br><br>
      <label for="password">Please enter your password：</label>
      <input type="password" id="password" name="password"><br><br>
      <button type="submit" value="Login">Login</button>
    </form>
  </body>
</html>

Other Examples

Compared to firefly-1b4, firefly-2b6 has also improved in other aspects, including classical Chinese translation, couplets, and ancient poetry creation.