Yinglong 110m

🚀 YingLong

YingLong is a model designed for time - series forecasting. It offers pre - trained capabilities on a vast dataset of 78B time points, providing powerful support for various time - series prediction tasks. You can find more details about the model in the paper, on our github, and on our project page.

🚀 Quick Start

Installation

pip install xformers transformers
pip install flash-attn --no-build-isolation
git clone https://github.com/Dao-AILab/flash-attention && cd flash-attention
cd csrc/rotary && pip install .
cd ../layer_norm && pip install .

⚠️ Important Note

Flash attention is not required. If you use a V100 or other GPU that doesn't support flash attention, just change FlashAttention2Available = RequirementCache("flash-attn>=2.0.0.post1") to FlashAttention2Available = False in the model.py file. The model should still be able to run.

Usage Examples

Basic Usage

import torch
from transformers import AutoModelForCausalLM

# load pretrain model
model = AutoModelForCausalLM.from_pretrained('qcw2333/YingLong_110m', trust_remote_code=True,torch_dtype=torch.bfloat16).cuda()

# prepare input
batch_size, lookback_length = 1, 2880
seqs = torch.randn(batch_size, lookback_length).bfloat16().cuda()

# generate forecast
prediction_length = 96
output = model.generate(seqs, future_token=prediction_length)

print(output.shape)

A notebook example is also provided here. The sample codes for long - term forecasting tasks and gift - eval tasks are provided at link.

📄 License

This model is licensed under the cc - by - 4.0 License.

📚 Documentation

The YingLong model is introduced in this paper. This version is pre - trained on 78B time points.

📧 Contact

If you have any questions or want to use the code, feel free to contact:

• Xue Wang (xue.w@alibaba - inc.com)
• Tian Zhou (tian.zt@alibaba - inc.com)