Monday, June 2, 2025
News PouroverAI
Visit PourOver.AI
No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
News PouroverAI
No Result
View All Result

MyShell Open-Sources OpenVoice: An Instant Voice Cloning AI Library that Takes a Short Audio Clip from the Reference Speaker and Generate Speech in Multiple Language

December 27, 2023
in Data Science & ML
Reading Time: 4 mins read
0 0
A A
0
Share on FacebookShare on Twitter


There are two challenges in voice cloning: 1) Flexible Voice Style Control- Many Instant Voice Cloning (IVC) approaches cannot manipulate voice styles after cloning flexibly. Numerous methods need to be revised to influence various aspects of voice styles precisely. This includes emotions, accents, rhythm, pauses, and intonation, along with accurately reproducing the distinctive tone characteristics of a reference speaker. 2) Zero-Shot Cross-Lingual Voice Cloning- Many IVC approaches require extensive massive-speaker multi-lingual (MSML) datasets for all languages.

A team of MIT, MyShell.ai, and Tsinghua University researchers have proposed OpenVoice, an open-source method for instant voice cloning. This approach can replicate their voice and generate speech in various languages with just a short audio sample from the reference speaker. OpenVoice can clone the tone color. OpenVoice provides adaptable manipulation of critical style elements such as emotion, accent, rhythm, pauses, and intonation. These features are vital in crafting contextually authentic speech and dynamic conversations, steering away from a monotonous narration of input text.

OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive speaker training set without requiring extensive training data for those languages. The technical approach of OpenVoice involves:

Decoupling the components in a voice as much as possible.

Independently generating language.

Tone color.

Other voice features.

The tone color cloning in OpenVoice is achieved through a tone color converter structurally similar to flow-based TTS methods but has different functionalities and training objectives.

The base speaker TTS model in OpenVoice is trained using audio samples from English, Chinese, and Japanese speakers, with the ability to change accent, language, and emotions. OpenVoice is computationally efficient, costing tens of times less than commercially available APIs.

OpenVoice achieves versatile instant voice cloning by replicating the voice of a reference speaker and generating speech in multiple languages. The approach enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, while accurately cloning the tone color of the reference speaker. The model can accurately clone the tone color of the reference speaker even when the language of the reference speaker or the generated speech is unseen in the training dataset. OpenVoice demonstrates superior performance compared to commercially available APIs while being computationally efficient.

In conclusion, OpenVoice showcases impressive capabilities in instant voice cloning, surpassing prior methods in flexibility regarding voice styles and languages. The fundamental idea behind this approach is rooted in the notion that training a base speaker TTS model to handle voice styles and languages is relatively straightforward, as long as the model isn’t tasked with cloning the exact tone color of the reference speaker. As a result, OpenVoice introduces a remarkable design principle by separating the cloning of tone color from other voice styles and language components, enhancing its overall versatility.

Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 35k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.

🚀 Boost your LinkedIn presence with Taplio: AI-driven content creation, easy scheduling, in-depth analytics, and networking with top creators – Try it free now!.



Source link

Tags: AudioclipCloninggenerateInstantlanguageLibraryMultipleMyShellOpenSourcesOpenVoiceReferenceshortspeakerSPEECHtakesVoice
Previous Post

Content Marketing 101 for Manufacturing Companies

Next Post

This AI Paper Explores How Vision-Language Models Enhance Autonomous Driving Systems for Better Decision-Making and Interactivity

Related Posts

AI Compared: Which Assistant Is the Best?
Data Science & ML

AI Compared: Which Assistant Is the Best?

June 10, 2024
5 Machine Learning Models Explained in 5 Minutes
Data Science & ML

5 Machine Learning Models Explained in 5 Minutes

June 7, 2024
Cohere Picks Enterprise AI Needs Over ‘Abstract Concepts Like AGI’
Data Science & ML

Cohere Picks Enterprise AI Needs Over ‘Abstract Concepts Like AGI’

June 7, 2024
How to Learn Data Analytics – Dataquest
Data Science & ML

How to Learn Data Analytics – Dataquest

June 6, 2024
Adobe Terms Of Service Update Privacy Concerns
Data Science & ML

Adobe Terms Of Service Update Privacy Concerns

June 6, 2024
Build RAG applications using Jina Embeddings v2 on Amazon SageMaker JumpStart
Data Science & ML

Build RAG applications using Jina Embeddings v2 on Amazon SageMaker JumpStart

June 6, 2024
Next Post
This AI Paper Explores How Vision-Language Models Enhance Autonomous Driving Systems for Better Decision-Making and Interactivity

This AI Paper Explores How Vision-Language Models Enhance Autonomous Driving Systems for Better Decision-Making and Interactivity

7 Steps to Developing an SEO Process in B2B Marketing

7 Steps to Developing an SEO Process in B2B Marketing

Adapting to the Changing IT Landscape

Adapting to the Changing IT Landscape

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Trending
  • Comments
  • Latest
Is C.AI Down? Here Is What To Do Now

Is C.AI Down? Here Is What To Do Now

January 10, 2024
Accenture creates a regulatory document authoring solution using AWS generative AI services

Accenture creates a regulatory document authoring solution using AWS generative AI services

February 6, 2024
Managing PDFs in Node.js with pdf-lib

Managing PDFs in Node.js with pdf-lib

November 16, 2023
23 Plagiarism Facts and Statistics to Analyze Latest Trends

23 Plagiarism Facts and Statistics to Analyze Latest Trends

June 4, 2024
Azul cloud service spots dead code in Java apps

Azul cloud service spots dead code in Java apps

October 7, 2023
The 15 Best Python Courses Online in 2024 [Free + Paid]

The 15 Best Python Courses Online in 2024 [Free + Paid]

April 13, 2024
Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

June 10, 2024
AI Compared: Which Assistant Is the Best?

AI Compared: Which Assistant Is the Best?

June 10, 2024
How insurance companies can use synthetic data to fight bias

How insurance companies can use synthetic data to fight bias

June 10, 2024
5 SLA metrics you should be monitoring

5 SLA metrics you should be monitoring

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

June 10, 2024
Facebook Twitter LinkedIn Pinterest RSS
News PouroverAI

The latest news and updates about the AI Technology and Latest Tech Updates around the world... PouroverAI keeps you in the loop.

CATEGORIES

  • AI Technology
  • Automation
  • Blockchain
  • Business
  • Cloud & Programming
  • Data Science & ML
  • Digital Marketing
  • Front-Tech
  • Uncategorized

SITEMAP

  • Disclaimer
  • Privacy Policy
  • DMCA
  • Cookie Privacy Policy
  • Terms and Conditions
  • Contact us

Copyright © 2023 PouroverAI News.
PouroverAI News

No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing

Copyright © 2023 PouroverAI News.
PouroverAI News

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In