Parler-TTS Mini v1 is a lightweight text-to-speech model, while Retrieva-JP T5 Base Long is a Japanese text processing model. They are combined for Japanese text-to-speech tasks.
This model combination focuses on Japanese text-to-speech tasks, with Parler-TTS Mini v1 handling speech synthesis and Retrieva-JP T5 Base Long managing text processing.
Model Features
Lightweight Speech Synthesis
Parler-TTS Mini v1 is a lightweight speech synthesis model suitable for resource-constrained environments.
Japanese Text Processing
Retrieva-JP T5 Base Long is optimized specifically for Japanese text, providing high-quality text processing capabilities.
Efficient Combination
The combination of these two models offers an efficient solution for Japanese text-to-speech tasks.
Model Capabilities
Japanese Text-to-Speech
Speech Synthesis
Japanese Text Processing
Use Cases
Voice Assistants
Japanese Voice Assistant
Used to build Japanese voice assistants that convert text into natural speech.
Generates fluent Japanese speech output
Education
Japanese Learning Tool
Used in Japanese learning applications to help learners hear correct pronunciations.
Provides accurate Japanese pronunciation
๐ Japanese Parler-TTS Mini
This repository publishes a model retrained based on parler-tts/parler-tts-mini-v1 to enable text-to-speech in Japanese. This model provides high-quality voice generation while being lightweight.
โ ๏ธ Important Note
It is not compatible with the tokenizer used in the original Parler-TTS. A unique tokenizer is adopted for this model.
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
from rubyinserter import add_ruby
device = "cuda:0"if torch.cuda.is_available() else"cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained("2121-8/japanese-parler-tts-mini").to(device)
prompt_tokenizer = AutoTokenizer.from_pretrained("2121-8/japanese-parler-tts-mini", subfolder="prompt_tokenizer")
description_tokenizer = AutoTokenizer.from_pretrained("2121-8/japanese-parler-tts-mini", subfolder="description_tokenizer")
prompt = "ใใใซใกใฏใไปๆฅใฏใฉใฎใใใซใ้ใใใงใใ๏ผ"
description = "A female speaker with a slightly high-pitched voice delivers her words at a moderate speed with a quite monotone tone in a confined environment, resulting in a quite clear audio recording."
prompt = add_ruby(prompt)
input_ids = description_tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = prompt_tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("parler_tts_japanese_out.wav", audio_arr, model.config.sampling_rate)
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
from rubyinserter import add_ruby
device = "cuda:0"if torch.cuda.is_available() else"cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained("2121-8/japanese-parler-tts-mini").to(device)
prompt_tokenizer = AutoTokenizer.from_pretrained("2121-8/japanese-parler-tts-mini", subfolder="prompt_tokenizer")
description_tokenizer = AutoTokenizer.from_pretrained("2121-8/japanese-parler-tts-mini", subfolder="description_tokenizer")
prompt = "ใใใซใกใฏใไปๆฅใฏใฉใฎใใใซใ้ใใใงใใ๏ผ"
description = "JSUT speaks with an expressive and animated tone in an excellent recording, with a very close-sounding proximity that suggests a private and intimate setting, and delivers her words at a rapid pace."
prompt = add_ruby(prompt)
input_ids = description_tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = prompt_tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("parler_tts_japanese_out.wav", audio_arr, model.config.sampling_rate)
Sample Audio
๐ License
This model and repository are permitted for use for a wide range of purposes, including research, education, and commercial use. However, please comply with the following conditions:
Commercial Use Conditions
You can use the voices and outputs generated by this model for commercial purposes, but selling the model itself (files, weight data, etc.) is prohibited.
Disclaimer of Appropriateness
The creator does not guarantee the accuracy, legality, or appropriateness of the results obtained from using this model.
User's Responsibility
When using this model, please comply with all applicable laws and regulations. All responsibilities arising from the generated content belong to the user.
Creator's Disclaimer
The creator of this repository and model is not responsible for any copyright infringement or other legal issues.
Response to Deletion Requests
In the event of a copyright issue, the problematic resources or data will be promptly deleted.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Safetensors