Audio Cloning

This is an extension of the Video Translator tool that generates cloned audio for short subtitles, improving the quality of audio output. The tool uses the OpenVoice and Melo TTS libraries to clone the audio output generated by the video translator tool.

Challenges faced while integrating audio cloning:

1. Library Compatibility: The libraries were not functional on Python 3.12, which I was using. They worked on Python 3.10, so including the TTS logic in the same script was not possible.
2. Short Subtitles: The libraries needed a longer sample audio to clone the voice better. Some of the subtitles were short, leading to errors in generating the audio for those subtitles. I cloned the OpenVoice repository and updated the checks that required longer audio samples. I also added a check to skip the subtitles that were too short.

Fixes:

Video Translator

I always wished to see the Japanese anime in english. But the audio was not available in English. I had to rely on subtitles. It was the same experience while watching other foreign language movies. Constantly watching the movie and subtitles was too much hassle.

During my break recently, I thought of exploring the possibility of translating the videos to English. With the advancement in AI and open source libraries, I decided to build a basic tool/script to achieve the same. I wanted the script to do the following: