🚀 Granite-TimeSeries-TTM-R1 Model Card
TinyTimeMixers (TTMs) are compact pre - trained models for Multivariate Time - Series Forecasting, open - sourced by IBM Research. With less than 1 Million parameters, TTM (accepted in NeurIPS 24) introduces the notion of the first - ever “tiny” pre - trained models for Time - Series Forecasting. TTM outperforms several popular benchmarks demanding billions of parameters in zero - shot and few - shot forecasting. It provides state - of - the - art zero - shot forecasts and can be easily fine - tuned for multi - variate forecasts with just 5% of the training data to be competitive.
🚀 Quick Start
You can start using the TTM model by following these steps. First, load the model from the HF Model Hub, specifying the branch name in the revision
field. Then, you can perform zeroshot forecasting or fine - tune the model for better results.
model = TinyTimeMixerForPrediction.from_pretrained(
"https://huggingface.co/ibm/TTM", revision="main"
)
zeroshot_trainer = Trainer(
model=model,
args=zeroshot_forecast_args,
)
)
zeroshot_output = zeroshot_trainer.evaluate(dset_test)
for param in model.backbone.parameters():
param.requires_grad = False
finetune_forecast_trainer = Trainer(
model=model,
args=finetune_forecast_args,
train_dataset=dset_train,
eval_dataset=dset_val,
callbacks=[early_stopping_callback, tracking_callback],
optimizers=(optimizer, scheduler),
)
finetune_forecast_trainer.train()
fewshot_output = finetune_forecast_trainer.evaluate(dset_test)
✨ Features
- Compact Pre - trained Models: With less than 1 million parameters, TTM offers “tiny” pre - trained models for time - series forecasting.
- High Performance: Outperforms benchmarks with billions of parameters in zero - shot and few - shot forecasting.
- Lightweight and Fast: Can be executed on CPU - only machines and easily fine - tuned in minutes.
- Multiple Forecasting Modes: Supports zeroshot and fine - tuned forecasting, as well as multivariate forecasting via different approaches.
- Exogenous and Categorical Data Support: Allows infusion of exogenous and categorical data.
📦 Installation
The installation process is not explicitly mentioned in the original document, so this section is skipped.
💻 Usage Examples
Basic Usage
model = TinyTimeMixerForPrediction.from_pretrained(
"https://huggingface.co/ibm/TTM", revision="main"
)
zeroshot_trainer = Trainer(
model=model,
args=zeroshot_forecast_args,
)
)
zeroshot_output = zeroshot_trainer.evaluate(dset_test)
Advanced Usage
for param in model.backbone.parameters():
param.requires_grad = False
finetune_forecast_trainer = Trainer(
model=model,
args=finetune_forecast_args,
train_dataset=dset_train,
eval_dataset=dset_val,
callbacks=[early_stopping_callback, tracking_callback],
optimizers=(optimizer, scheduler),
)
finetune_forecast_trainer.train()
fewshot_output = finetune_forecast_trainer.evaluate(dset_test)
📚 Documentation
Model Description
TTM belongs to the “focused pre - trained models” category. Instead of a single large model for all forecasting settings, we create smaller pre - trained models for specific settings, resulting in more accurate forecasts. These models are small, fast, and easy to deploy.
Model Releases
- 512 - 96: Given the last 512 time - points, it can forecast up to the next 96 time - points. Recommended for hourly and minutely resolutions. (branch name: main) [[Benchmark Scripts]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm - r1_benchmarking_512_96.ipynb)
- 1024 - 96: Given the last 1024 time - points, it can forecast up to the next 96 time - points. Targeted for long - forecasting settings. (branch name: 1024 - 96 - v1) [[Benchmark Scripts]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm - r1_benchmarking_1024_96.ipynb)
You can also use the [[get_model]](https://github.com/ibm - granite/granite - tsfm/blob/main/tsfm_public/toolkit/get_model.py) utility to select the required model based on your input context length and forecast length.
Model Capabilities
- Getting Started: [[colab]](https://colab.research.google.com/github/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb)
- Zeroshot Multivariate Forecasting: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb)
- Finetuned Multivariate Forecasting:
- Channel - Independent Finetuning: [[Example 1]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb) [[Example 2]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb)
- Channel - Mix Finetuning: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/tutorial/ttm_channel_mix_finetuning.ipynb)
- New Releases (October 2024):
- Finetuning and Forecasting with Exogenous/Control Variables: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
- Finetuning and Forecasting with static categorical features: [Example: To be added soon]
- Rolling Forecasts: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_rolling_prediction_getting_started.ipynb)
- Helper scripts for optimal Learning Rate suggestions for Finetuning: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
Benchmarks
TTM outperforms popular benchmarks like TimesFM, Moirai, Chronos, etc. in zero/few - shot forecasting and reduces computational requirements. It can run on CPU - only machines. For more details, refer to our paper.
Recommended Use
- Standard scale your data independently for each channel before feeding it to the model. Refer to TSP for data scaling.
- The current open - source version supports only minutely and hourly resolutions. Other lower resolutions are not supported as the model needs a minimum context length of 512 or 1024.
- Avoid upsampling or prepending zeros to increase the context length for shorter - length datasets as it will impact model performance.
Model Details
For more details on TTM architecture and benchmarks, refer to our paper. TTM - 1 supports two modes:
- Zeroshot forecasting: Apply the pre - trained model directly to get an initial forecast.
- Finetuned forecasting: Finetune the pre - trained model with a subset of your target data for better forecasts.
The current release supports multivariate forecasting via channel independence and channel - mixing approaches. It also supports exogenous and categorical data infusion.
Model Sources
- Repository: https://github.com/ibm - granite/granite - tsfm/tree/main/tsfm_public/models/tinytimemixer
- Paper: https://arxiv.org/pdf/2401.03955.pdf
Blogs and articles on TTM
Refer to our [wiki](https://github.com/ibm - granite/granite - tsfm/wiki)
🔧 Technical Details
TTM models are based on the concept of creating focused pre - trained models for specific time - series forecasting settings. They are pre - trained on publicly available time - series data with various augmentations. The models are extremely small and fast, which allows for quick fine - tuning on target data. For more in - depth technical information, please refer to our paper.
📄 License
The model is released under the Apache 2.0 license.
📦 Training Data
The original r1 TTM models were trained on a collection of datasets from the Monash Time Series Forecasting repository:
Property |
Details |
Australian Electricity Demand |
https://zenodo.org/records/4659727 |
Australian Weather |
https://zenodo.org/records/4654822 |
Bitcoin dataset |
https://zenodo.org/records/5122101 |
KDD Cup 2018 dataset |
https://zenodo.org/records/4656756 |
London Smart Meters |
https://zenodo.org/records/4656091 |
Saugeen River Flow |
https://zenodo.org/records/4656058 |
Solar Power |
https://zenodo.org/records/4656027 |
Sunspots |
https://zenodo.org/records/4654722 |
Solar |
https://zenodo.org/records/4656144 |
US Births |
https://zenodo.org/records/4656049 |
Wind Farms Production data |
https://zenodo.org/records/4654858 |
Wind Power |
https://zenodo.org/records/4656032 |
📖 Citation
If you use our model or its associated architectures/approaches in your work, kindly cite the following paper:
BibTeX:
@inproceedings{ekambaram2024tinytimemixersttms,
title={Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series},
author={Vijay Ekambaram and Arindam Jati and Pankaj Dayama and Sumanta Mukherjee and Nam H. Nguyen and Wesley M. Gifford and Chandra Reddy and Jayant Kalagnanam},
booktitle={Advances in Neural Information Processing Systems (NeurIPS 2024)},
year={2024},
}
👥 Model Card Authors
Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Wesley M. Gifford, Sumanta Mukherjee, Chandra Reddy and Jayant Kalagnanam
⚠️ IBM Public Repository Disclosure
All content in this repository including code has been provided by IBM under the associated open source software license and IBM is under no obligation to provide enhancements, updates, or support. IBM developers produced this code as an open source project (not as an IBM product), and IBM makes no assertions as to the level of quality nor security, and will not be maintaining this code going forward.