Yandex stikes to provide its users with new features enhancing content perception.
The users of Yandex Browser have access to polyphonic video translation which is available in 12 voices in total, 6 male and 6 female ones.
One neural network translates voices in a video into text, sets up punctuation accordingly, and defines boundaries in the sentences. The other neural network is set to analyse the voice’s spectrogram and notes the fragments on the audio tracks which are spoken by different people. It helps to understand which sentences in the text belong to which speaker.
The head of Yandex and Yandex Browser application Dmitry Timko commented that the team constantly works on providing the best quality of translation and dubbing, aiming to make any content from around the world accessible to their users. According to Yandex estimates, approximately half of videos on YouTube are in foreign languages with three or more speakers. Watching these types of videos with 2 dubbing voices is not pleasant which is the reason that the Yandex team created 10 more dubbing voices.
In September last year, off-screen translation of commercial videos in English became publicly available. Later, French, German, Spanish, and Italian languages were added.
Yandex states that it also plans to include new languages such as Chinese. Currently, automatic voice-over translation of live broadcasts on YouTube is in the real-time testing stage.
