Riffusion

🚀 Riffusion

Riffusion is an app for real-time music generation with stable diffusion. It empowers users to generate music in real - time. You can read about it at https://www.riffusion.com/about and try it at https://www.riffusion.com/.

🚀 Quick Start

To get started with Riffusion, you can access the following resources:

• Web app: https://github.com/hmartiro/riffusion-app
• Inference server: https://github.com/hmartiro/riffusion-inference
• Model checkpoint: https://huggingface.co/riffusion/riffusion-model-v1

✨ Features

This repository contains the model files, including:

• A diffusers formated library
• A compiled checkpoint file
• A traced unet for improved inference speed
• A seed image library for use with riffusion - app

📚 Documentation

Riffusion v1 Model

Riffusion is a latent text - to - image diffusion model capable of generating spectrogram images given any text input. These spectrograms can be converted into audio clips.

The model was created by Seth Forsgren and Hayk Martiros as a hobby project.

You can use the Riffusion model directly, or try the Riffusion web app.

The Riffusion model was created by fine - tuning the Stable - Diffusion - v1 - 5 checkpoint. Read about Stable Diffusion here 🤗's Stable Diffusion blog.

Model Details

Direct Use

The model is intended for research purposes only. Possible research areas and tasks include:

• Generation of artworks, audio, and use in creative processes.
• Applications in educational or creative tools.
• Research on generative models.

Citation

If you build on this work, please cite it as follows:

@software{Forsgren_Martiros_2022,
  author = {Forsgren, Seth* and Martiros, Hayk*},
  title = {{Riffusion - Stable diffusion for real - time music generation}},
  url = {https://riffusion.com/about},
  year = {2022}
}

📄 License

This model is open access and available to all, with a CreativeML OpenRAIL - M license further specifying rights and usage.

The CreativeML OpenRAIL License specifies:

1. You can't use the model to deliberately produce nor share illegal or harmful outputs or content.
2. Riffusion claims no rights on the outputs you generate, you are free to use them and are accountable for their use which must not go against the provisions set in the license.
3. You may re - distribute the weights and use the model commercially and/or as a service. If you do, please be aware you have to include the same use restrictions as the ones in the license and share a copy of the CreativeML OpenRAIL - M to all your users (please read the license entirely and carefully).

Please read the full license carefully here: https://huggingface.co/spaces/CompVis/stable-diffusion-license

⚠️ Important Note

Please read the LICENSE to access this model.