whisper_tflite_models Open Source Model - F-Droid Platform Supports Voice Transcription and Translation

Whisper Tflite Models

Developed by DocWolle

A TFLite model suitable for the Whisper application on the F-Droid platform, supporting speech transcription and translation functions.

Speech Recognition Open Source License:MIT #Multilingual speech transcription #Real-time speech translation #TFLite deployment

Downloads 11.20k

Release Time : 12/27/2024

Model Overview

This model is specifically designed for the Whisper application, providing speech transcription and translation functions, supporting multiple languages, and allowing specific language operations to be specified through the forced decoder ID.

Model Features

Forced decoder ID

By setting the forced decoder ID, the model can be forced to perform transcription or translation operations in a specific language.

Multilingual support

Supports transcription and translation of multiple languages. Specific language codes can be referred to in the provided link.

TFLite optimization

The model is optimized by TFLite and is suitable for running on mobile devices.

Model Capabilities

Speech transcription

Speech translation

Multilingual processing

Use Cases

Speech processing

Speech transcription

Transcribe speech content into text, supporting multiple languages.

Generate accurate text transcription results.

Speech translation

Translate speech content into text in the specified language.

Generate translated text results.

🚀 Whisper TFLite Models for Whisper App on F-Droid

These Whisper TFLite models are designed for use in the Whisper app on F-Droid. The "transcribe-translate" models come with signatures for "serving_transcribe" and "serving_translate", which can force the model to perform specific actions.

💻 Usage Examples

Basic Usage

@tf.function(
    input_signature=[
        tf.TensorSpec((1, 80, 3000), tf.float32, name="input_features"),
    ],
)
def transcribe(self, input_features):
    outputs = self.model.generate(
        input_features,
        max_new_tokens=450,  # change as needed
        return_dict_in_generate=True,
        forced_decoder_ids=[[2, 50359], [3, 50363]],  # forced to transcribe any language with no timestamps
    )
    return {"sequences": outputs["sequences"]}

@tf.function(
    input_signature=[
        tf.TensorSpec((1, 80, 3000), tf.float32, name="input_features"),
    ],
)
def translate(self, input_features):
    outputs = self.model.generate(
        input_features,
        max_new_tokens=450,  # change as needed
        return_dict_in_generate=True,
        forced_decoder_ids=[[2, 50358], [3, 50363]],  # forced to translate any language with no timestamps
    )
    return {"sequences": outputs["sequences"]}

Advanced Usage

To force transcription for a certain language, set the 1st decoder id as follows:

def transcribe(self, input_features):
    outputs = self.model.generate(
        input_features,
        max_new_tokens=450,  # change as needed
        return_dict_in_generate=True,
        forced_decoder_ids=[[1, 50261], [2, 50359], [3, 50363]],  # forced to transcribe (50359) German (50261) with no timestamps (50363)
    )
    return {"sequences": outputs["sequences"]}

def translate(self, input_features):
    outputs = self.model.generate(
        input_features,
        max_new_tokens=450,  # change as needed
        return_dict_in_generate=True,
        forced_decoder_ids=[[1, 50261], [2, 50358], [3, 50363]],  # different forced_decoder_ids
    )
    return {"sequences": outputs["sequences"]}

(language codes from here: https://github.com/woheller69/whisperIME/blob/master/app/src/main/java/com/whispertflite/utils/InputLang.java)

📄 License

This project is licensed under the MIT License.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご