wav2vec2_ASV Deepfake Audio Detection Model - Open-source Identification of Synthesized or Tampered Voice Content

Wav2vec2 ASV Deepfake Audio Detection

Developed by Bisher

A deepfake audio detection model fine-tuned based on facebook/wav2vec2-base, used to identify synthetic or tampered speech content

Speaker Analysis

Transformers

Open Source License:Apache-2.0 #Audio forgery detection #High accuracy #ASV protection

Downloads 106

Release Time : 8/20/2024

Model Overview

This model detects deepfake speech by analyzing audio features, achieving 89.99% accuracy and 93.72% ROC AUC on the test set

Model Features

High accuracy detection

Achieves 89.99% accuracy and 93.72% ROC AUC on the test set, effectively identifying forged audio

Low false positive rate

Only 8 false positives (Fp), indicating extremely low false alarm rate while maintaining high recall

Based on wav2vec2 architecture

Utilizes the powerful audio feature extraction capability of facebook/wav2vec2-base through fine-tuning

Model Capabilities

Audio authenticity detection

Deepfake recognition

Speech content verification

Use Cases

Security verification

Voice identity authentication

Detects forged voice attacks in voice identity authentication systems

Can identify over 90% of forged voice samples

Content moderation

Audio content review

Identifies tampered or synthetic audio content

89.99% accuracy, 90.57% precision

🚀 wav2vec2_ASV_deepfake_audio_detection

A fine - tuned model for deepfake audio detection based on wav2vec2 architecture

🚀 Quick Start

This model is a fine - tuned version of facebook/wav2vec2-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5628
Accuracy: 0.8999
Precision: 0.9057
F1: 0.8612
Tp: 181
Tn: 16068
Fn: 1800
Fp: 8
Auc Roc: 0.9372

📚 Documentation

Model Information

Property	Details
Library Name	transformers
Base Model	facebook/wav2vec2-base
Model Type	wav2vec2_ASV_deepfake_audio_detection
Metrics	accuracy, precision, f1
License	apache - 2.0

Training Procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 100
eval_batch_size: 100
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 400
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 5
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	F1	Tp	Tn	Fn	Fp	Auc Roc
0.693	0.1143	10	0.6628	0.8854	0.8117	0.8385	23	15964	1958	112	0.5001
0.6589	0.2286	20	0.4915	0.8903	0.7926	0.8386	0	16076	1981	0	0.5030
0.5546	0.3429	30	0.3825	0.8865	0.8231	0.8406	39	15969	1942	107	0.5748
0.3566	0.4571	40	0.3403	0.8909	0.8620	0.8419	28	16059	1953	17	0.6201
0.2115	0.5714	50	0.3617	0.8923	0.8908	0.8442	43	16070	1938	6	0.7028
0.1636	0.6857	60	0.3428	0.8958	0.8756	0.8586	182	15993	1799	83	0.7968
0.1415	0.8	70	0.3899	0.8925	0.9015	0.8440	41	16075	1940	1	0.6722
0.11	0.9143	80	0.3756	0.8930	0.9024	0.8452	50	16075	1931	1	0.7490
0.1041	1.0286	90	0.3885	0.8960	0.9006	0.8526	110	16069	1871	7	0.6362
0.0888	1.1429	100	0.3484	0.8995	0.8936	0.8630	207	16036	1774	40	0.8231
0.0669	1.2571	110	0.3386	0.9049	0.9040	0.8734	299	16041	1682	35	0.8354
0.0552	1.3714	120	0.4530	0.8942	0.9055	0.8480	71	16076	1910	0	0.8554
0.071	1.4857	130	0.4327	0.8963	0.8937	0.8545	128	16057	1853	19	0.8543
0.0665	1.6	140	0.4547	0.8947	0.9045	0.8491	80	16075	1901	1	0.8065
0.054	1.7143	150	0.3210	0.9148	0.9064	0.8970	592	15926	1389	150	0.8851
0.0575	1.8286	160	0.4901	0.8934	0.9012	0.8462	58	16074	1923	2	0.7591
0.0437	1.9429	170	0.4849	0.8979	0.9036	0.8568	144	16069	1837	7	0.6435
0.0471	2.0571	180	0.3822	0.9071	0.9103	0.8767	324	16056	1657	20	0.9277
0.0377	2.1714	190	0.5301	0.8928	0.8962	0.8450	49	16072	1932	4	0.9112
0.0327	2.2857	200	0.5534	0.8920	0.9036	0.8426	30	16076	1951	0	0.8755
0.0522	2.4	210	0.2332	0.9260	0.9192	0.9162	865	15856	1116	220	0.9448
0.0449	2.5143	220	0.3034	0.9102	0.9104	0.8835	397	16038	1584	38	0.9453
0.0338	2.6286	230	0.4001	0.9018	0.9072	0.8654	218	16066	1763	10	0.9153
0.0337	2.7429	240	0.4761	0.8973	0.9056	0.8552	130	16073	1851	3	0.8789
0.0347	2.8571	250	0.5613	0.8921	0.9037	0.8429	32	16076	1949	0	0.9068
0.0301	2.9714	260	0.4896	0.8967	0.9025	0.8540	121	16070	1860	6	0.9480
0.0208	3.0857	270	0.5223	0.8983	0.9053	0.8575	149	16071	1832	5	0.9471
0.0197	3.2	280	0.5003	0.9024	0.9068	0.8669	232	16063	1749	13	0.9445
0.0167	3.3143	290	0.4328	0.9087	0.9123	0.8796	351	16057	1630	19	0.9561
0.0235	3.4286	300	0.3612	0.9097	0.9115	0.8821	380	16047	1601	29	0.9596
0.0207	3.5429	310	0.3538	0.9158	0.9169	0.8934	498	16038	1483	38	0.9591
0.0192	3.6571	320	0.4185	0.9145	0.9171	0.8907	465	16049	1516	27	0.9404
0.0176	3.7714	330	0.6594	0.8926	0.9017	0.8443	43	16075	1938	1	0.8734
0.0174	3.8857	340	0.5727	0.8995	0.9073	0.8600	170	16072	1811	4	0.9276
0.021	4.0	350	0.5943	0.8937	0.8988	0.8471	65	16072	1916	4	0.9460
0.02	4.1143	360	0.5183	0.8982	0.9040	0.8574	149	16069	1832	7	0.9507
0.015	4.2286	370	0.5329	0.8980	0.9037	0.8570	146	16069	1835	7	0.9477
0.0139	4.3429	380	0.5545	0.8967	0.9017	0.8541	122	16069	1859	7	0.9438
0.0103	4.4571	390	0.5638	0.8969	0.9021	0.8546	126	16069	1855	7	0.9403
0.0099	4.5714	400	0.5094	0.9030	0.9078	0.8679	241	16064	1740	12	0.9419
0.0121	4.6857	410	0.5066	0.9049	0.9099	0.8717	275	16064	1706	12	0.9406
0.0122	4.8	420	0.5700	0.8992	0.9047	0.8596	168	16068	1813	8	0.9326
0.0155	4.9143	430	0.5628	0.8999	0.9057	0.8612	181	16068	1800	8	0.9372

Framework versions

Transformers 4.44.1
Pytorch 2.2.1+cu121
Datasets 2.21.0
Tokenizers 0.19.1

📄 License

This model is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご