Granite Timeseries Ttm R1

🚀 Granite-TimeSeries-TTM-R1 Model Card

TinyTimeMixers (TTMs) are compact pre - trained models for Multivariate Time - Series Forecasting, open - sourced by IBM Research. With less than 1 Million parameters, TTM (accepted in NeurIPS 24) introduces the notion of the first - ever “tiny” pre - trained models for Time - Series Forecasting. TTM outperforms several popular benchmarks demanding billions of parameters in zero - shot and few - shot forecasting. It provides state - of - the - art zero - shot forecasts and can be easily fine - tuned for multi - variate forecasts with just 5% of the training data to be competitive.

🚀 Quick Start

You can start using the TTM model by following these steps. First, load the model from the HF Model Hub, specifying the branch name in the revision field. Then, you can perform zeroshot forecasting or fine - tune the model for better results.

# Load Model from HF Model Hub mentioning the branch name in revision field
model = TinyTimeMixerForPrediction.from_pretrained(
                "https://huggingface.co/ibm/TTM", revision="main"
            ) 

# Do zeroshot
zeroshot_trainer = Trainer(
        model=model,
        args=zeroshot_forecast_args,
        )
    )

zeroshot_output = zeroshot_trainer.evaluate(dset_test)


# Freeze backbone and enable few - shot or finetuning:

# freeze backbone
for param in model.backbone.parameters():
  param.requires_grad = False

finetune_forecast_trainer = Trainer(
        model=model,
        args=finetune_forecast_args,
        train_dataset=dset_train,
        eval_dataset=dset_val,
        callbacks=[early_stopping_callback, tracking_callback],
        optimizers=(optimizer, scheduler),
    )
finetune_forecast_trainer.train()
fewshot_output = finetune_forecast_trainer.evaluate(dset_test)

✨ Features

• Compact Pre - trained Models: With less than 1 million parameters, TTM offers “tiny” pre - trained models for time - series forecasting.
• High Performance: Outperforms benchmarks with billions of parameters in zero - shot and few - shot forecasting.
• Lightweight and Fast: Can be executed on CPU - only machines and easily fine - tuned in minutes.
• Multiple Forecasting Modes: Supports zeroshot and fine - tuned forecasting, as well as multivariate forecasting via different approaches.
• Exogenous and Categorical Data Support: Allows infusion of exogenous and categorical data.

📦 Installation

The installation process is not explicitly mentioned in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

# Load Model from HF Model Hub mentioning the branch name in revision field
model = TinyTimeMixerForPrediction.from_pretrained(
                "https://huggingface.co/ibm/TTM", revision="main"
            ) 

# Do zeroshot
zeroshot_trainer = Trainer(
        model=model,
        args=zeroshot_forecast_args,
        )
    )

zeroshot_output = zeroshot_trainer.evaluate(dset_test)

Advanced Usage

# Freeze backbone and enable few - shot or finetuning:

# freeze backbone
for param in model.backbone.parameters():
  param.requires_grad = False

finetune_forecast_trainer = Trainer(
        model=model,
        args=finetune_forecast_args,
        train_dataset=dset_train,
        eval_dataset=dset_val,
        callbacks=[early_stopping_callback, tracking_callback],
        optimizers=(optimizer, scheduler),
    )
finetune_forecast_trainer.train()
fewshot_output = finetune_forecast_trainer.evaluate(dset_test)

📚 Documentation

Model Description

TTM belongs to the “focused pre - trained models” category. Instead of a single large model for all forecasting settings, we create smaller pre - trained models for specific settings, resulting in more accurate forecasts. These models are small, fast, and easy to deploy.

Model Releases

• 512 - 96: Given the last 512 time - points, it can forecast up to the next 96 time - points. Recommended for hourly and minutely resolutions. (branch name: main) [[Benchmark Scripts]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm - r1_benchmarking_512_96.ipynb)
• 1024 - 96: Given the last 1024 time - points, it can forecast up to the next 96 time - points. Targeted for long - forecasting settings. (branch name: 1024 - 96 - v1) [[Benchmark Scripts]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm - r1_benchmarking_1024_96.ipynb)

You can also use the [[get_model]](https://github.com/ibm - granite/granite - tsfm/blob/main/tsfm_public/toolkit/get_model.py) utility to select the required model based on your input context length and forecast length.

Model Capabilities

• Getting Started: [[colab]](https://colab.research.google.com/github/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb)
• Zeroshot Multivariate Forecasting: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb)
• Finetuned Multivariate Forecasting:
  • Channel - Independent Finetuning: [[Example 1]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb) [[Example 2]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb)
  • Channel - Mix Finetuning: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/tutorial/ttm_channel_mix_finetuning.ipynb)
• New Releases (October 2024):
  • Finetuning and Forecasting with Exogenous/Control Variables: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
  • Finetuning and Forecasting with static categorical features: [Example: To be added soon]
  • Rolling Forecasts: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/hfdemo/ttm_rolling_prediction_getting_started.ipynb)
  • Helper scripts for optimal Learning Rate suggestions for Finetuning: [[Example]](https://github.com/ibm - granite/granite - tsfm/blob/main/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)

Benchmarks

TTM outperforms popular benchmarks like TimesFM, Moirai, Chronos, etc. in zero/few - shot forecasting and reduces computational requirements. It can run on CPU - only machines. For more details, refer to our paper.

Recommended Use

1. Standard scale your data independently for each channel before feeding it to the model. Refer to TSP for data scaling.
2. The current open - source version supports only minutely and hourly resolutions. Other lower resolutions are not supported as the model needs a minimum context length of 512 or 1024.
3. Avoid upsampling or prepending zeros to increase the context length for shorter - length datasets as it will impact model performance.

Model Details

For more details on TTM architecture and benchmarks, refer to our paper. TTM - 1 supports two modes:

• Zeroshot forecasting: Apply the pre - trained model directly to get an initial forecast.
• Finetuned forecasting: Finetune the pre - trained model with a subset of your target data for better forecasts.

The current release supports multivariate forecasting via channel independence and channel - mixing approaches. It also supports exogenous and categorical data infusion.

Model Sources

• Repository: https://github.com/ibm - granite/granite - tsfm/tree/main/tsfm_public/models/tinytimemixer
• Paper: https://arxiv.org/pdf/2401.03955.pdf

Blogs and articles on TTM

Refer to our [wiki](https://github.com/ibm - granite/granite - tsfm/wiki)

🔧 Technical Details

TTM models are based on the concept of creating focused pre - trained models for specific time - series forecasting settings. They are pre - trained on publicly available time - series data with various augmentations. The models are extremely small and fast, which allows for quick fine - tuning on target data. For more in - depth technical information, please refer to our paper.

📄 License

The model is released under the Apache 2.0 license.

📦 Training Data

The original r1 TTM models were trained on a collection of datasets from the Monash Time Series Forecasting repository:

📖 Citation

If you use our model or its associated architectures/approaches in your work, kindly cite the following paper:

BibTeX:

@inproceedings{ekambaram2024tinytimemixersttms,
      title={Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series},
      author={Vijay Ekambaram and Arindam Jati and Pankaj Dayama and Sumanta Mukherjee and Nam H. Nguyen and Wesley M. Gifford and Chandra Reddy and Jayant Kalagnanam},
      booktitle={Advances in Neural Information Processing Systems (NeurIPS 2024)},
      year={2024},
}

👥 Model Card Authors

Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Wesley M. Gifford, Sumanta Mukherjee, Chandra Reddy and Jayant Kalagnanam

⚠️ IBM Public Repository Disclosure

All content in this repository including code has been provided by IBM under the associated open source software license and IBM is under no obligation to provide enhancements, updates, or support. IBM developers produced this code as an open source project (not as an IBM product), and IBM makes no assertions as to the level of quality nor security, and will not be maintaining this code going forward.