Sonics Spectttra Gamma 5s

🚀 SONICS: Synthetic Or Not - Identifying Counterfeit Songs

SONICS is a novel dataset for end - to - end Synthetic Song Detection (SSD), aiming to distinguish between human - composed and synthetic songs. It also introduces the SpecTTTra architecture, which significantly improves time and memory efficiency for song authenticity detection.

🚀 Quick Start

To quickly start using the SONICS project, you can follow these steps:

# Install from GitHub
!pip install git+https://github.com/awsaf49/sonics.git

# Load model
from sonics import HFAudioClassifier
model = HFAudioClassifier.from_pretrained("awsaf49/sonics-spectttra-gamma-5s")

✨ Features

• Novel Dataset: SONICS is a novel dataset for end - to - end Synthetic Song Detection (SSD), containing over 97k songs (4,751 hours) with over 49k synthetic songs from popular platforms like Suno and Udio.
• Long - Range Temporal Modeling: Highlights the importance of modeling long - range temporal dependencies in songs for effective authenticity detection, an aspect overlooked in existing methods.
• Efficient Architecture: Introduces SpecTTTra, a novel architecture that significantly improves time and memory efficiency over conventional CNN and Transformer - based models.

📦 Installation

You can install the SONICS project from GitHub using the following command:

!pip install git+https://github.com/awsaf49/sonics.git

💻 Usage Examples

Basic Usage

# Install from GitHub
!pip install git+https://github.com/awsaf49/sonics.git

# Load model
from sonics import HFAudioClassifier
model = HFAudioClassifier.from_pretrained("awsaf49/sonics-spectttra-gamma-5s")

📚 Documentation

📌 Abstract

The recent surge in AI - generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human - composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI - generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end - to - end artificial songs where all components (vocals, music, lyrics, and style) could be AI - generated. Additionally, existing datasets lack music - lyrics diversity, long - duration songs, and open - access fake songs. To address these gaps, we introduce SONICS, a novel dataset for end - to - end Synthetic Song Detection (SSD), comprising over 97k songs (4,751 hours) with over 49k synthetic songs from popular platforms like Suno and Udio. Furthermore, we highlight the importance of modeling long - range temporal dependencies in songs for effective authenticity detection, an aspect entirely overlooked in existing methods. To utilize long - range patterns, we introduce SpecTTTra, a novel architecture that significantly improves time and memory efficiency over conventional CNN and Transformer - based models. For long songs, our top - performing variant outperforms ViT by 8% in F1 score, is 38% faster, and uses 26% less memory, while also surpassing ConvNeXt with a 1% F1 score gain, 20% speed boost, and 67% memory reduction.

🔗 Links

• 📄 Paper
• 🎵 Dataset
• 🔬 ArXiv
• 💻 GitHub

🏆 Model Performance

Model Name	HF Link	Variant	Duration	f_clip	t_clip	F1	Sensitivity	Specificity	Speed (A/S)	FLOPs (G)	Mem. (GB)	# Act. (M)	# Param. (M)
sonics-spectttra-alpha-5s	HF	SpecTTTra-α	5s	1	3	0.78	0.69	0.94	148	2.9	0.5	6	17
sonics-spectttra-beta-5s	HF	SpecTTTra-β	5s	3	5	0.78	0.69	0.94	152	1.1	0.2	5	17
sonics-spectttra-gamma-5s	HF	SpecTTTra-γ	5s	5	7	0.76	0.66	0.92	154	0.7	0.1	2	17
sonics-spectttra-alpha-120s	HF	SpecTTTra-α	120s	1	3	0.97	0.96	0.99	47	23.7	3.9	50	19
sonics-spectttra-beta-120s	HF	SpecTTTra-β	120s	3	5	0.92	0.86	0.99	80	14.0	2.3	29	21
sonics-spectttra-gamma-120s	HF	SpecTTTra-γ	120s	5	7	0.88	0.79	0.99	97	10.1	1.6	20	24

📐 Model Architecture

• Base Model: SpectTTTra (Spectro - Temporal Tokens Transformer)
• Embedding Dimension: 384
• Number of Heads: 6
• Number of Layers: 12
• MLP Ratio: 2.67

🎶 Audio Processing

• Sample Rate: 16kHz
• FFT Size: 2048
• Hop Length: 512
• Mel Bands: 128
• Frequency Range: 20Hz - 8kHz
• Normalization: Mean - std normalization

🔧 Technical Details

The project addresses the limitations of existing fake song detection methods and datasets. It emphasizes the need for detecting end - to - end artificial songs and the importance of long - range temporal modeling. The SpecTTTra architecture is designed to handle long - range patterns more efficiently in terms of time and memory compared to traditional CNN and Transformer - based models.

📄 License

This project is licensed under the MIT license.

📝 Citation

@inproceedings{rahman2024sonics,
        title={SONICS: Synthetic Or Not - Identifying Counterfeit Songs},
        author={Rahman, Md Awsafur and Hakim, Zaber Ibn Abdul and Sarker, Najibul Haque and Paul, Bishmoy and Fattah, Shaikh Anowarul},
        booktitle={International Conference on Learning Representations (ICLR)},
        year={2025},
      }