ZOO Digital offers comprehensive localization and media services for adapting original TV and movie content to various languages, regions, and cultures. This facilitates globalization for top content creators worldwide. Renowned in the entertainment industry, ZOO Digital provides high-quality localization and media services on a large scale, including dubbing, subtitling, scripting, and compliance. Traditional localization processes involve manual speaker diarization, where audio streams are segmented based on the speaker’s identity before dubbing into another language can occur. This manual process can be time-consuming, taking 1–3 hours to localize a 30-minute episode. ZOO Digital aims to streamline localization to under 30 minutes through automation.
To achieve this goal, ZOO Digital collaborated with AWS Prototyping to deploy scalable machine learning (ML) models for diarizing media content using Amazon SageMaker, focusing on the WhisperX model. By leveraging automation, ZOO Digital seeks to accelerate content localization workflows to meet the growing demand for localized content.
The collaboration involved storing original media files in an Amazon Simple Storage Service (Amazon S3) bucket, triggering an AWS Lambda function when new files are detected. The Lambda function then invoked the SageMaker endpoint for inference using the WhisperX model, which combines transcription, alignment, and diarization for media assets. WhisperX utilizes models from Hugging Face, including the Whisper model for transcriptions, the Wav2Vec2 model for timestamp alignment, and the pyannote model for diarization.
To host the WhisperX model on SageMaker, model artifacts were pre-downloaded and saved in the serving container during initiation. An inference script was created to load the models and run the transcription, alignment, and diarization processes during inference. The collaboration demonstrated the potential of deploying WhisperX on SageMaker for efficient and cost-effective processing of large media files, such as movies and TV series.
Overall, ZOO Digital’s collaboration with AWS Prototyping showcased the benefits of leveraging machine learning and automation to enhance the localization process for content creators and entertainment industry professionals.
Source link
