Dubbing is a lucrative market, and Verified Market Research predicts that by 2027, film dubbing services alone could generate $3.6 billion a year. But it is also a time-consuming and costly process. On the average, five minutes of narration can take an hour of a recording studio; one calculator sets a price of $75 per minute even for a simple video.
The prospects for AI in this area, in particular natural language processing, speeds up the task by creating human-sounding dubs in multiple languages. One British startup, Papercup, claims its technology is used by media giants Sky News, Discovery and Business Insider and has been used to translate 30 seasons of the cult Bob Ross show. The is the joy of painting.
CEO Jesse Shemen estimates that more than 300 million people have watched Papercup translated videos in the past 12 months.
“There is a significant mismatch between the demand for localization and translation and the ability to meet the demand,” Shemen said. “Shows likes [Netflix’s] “Squid Game” confirms the thesis that people will watch content created anywhere and in any language if it is entertaining and interesting. That’s why the sector is so growth-oriented.”
For example, Papercup today announced that it has raised $20 million in a Series A funding round led by Octopus Ventures, involving Local Globe, Sands Capital, Sky and Guardian Media Ventures, Entrepreneur First and BDMI. To date, the London-based company’s total fundraising is approximately $30.5 million, most of which will go towards researching expressive AI-generated voices and expanding Papercup support for foreign languages, Shemen told TechCrunch via email.
Founded in 2017 by Shemen and Jiamen Gao, Papercup provides an AI dubbing solution that identifies human voices in a target movie or show and creates dubbing in a new language. Video content producers upload their videos, specify the language, wait for the Papercup native-speaker teams to check the audio quality, and receive a translation with a synthetic voice-over.
Shemen claims that the Papercup platform can generate duplicates at a scale and pace that cannot be achieved by manual methods. In addition to the custom translations it creates for clients, Papercup offers a catalog of voices with “realistic” tones and emotions. Many of them have been used in internal communications, corporate announcements and educational materials in addition to film and television, Shemen said.
“Our human-in-the-loop approach means that human translators provide quality control and guarantee accuracy, but must be much less hands-on than if they were doing the entire translation, which means they can work faster and with more translations.” Shemen said. . “During the pandemic, people have been watching more video content, which has greatly increased the demand for our services.”
The market for “synthetic media” created by artificial intelligence is growing. Video and voice firms including Synthesia, Respeecher, Resemble AI and Deepdub have launched AI-assisted dubbing tools for shows and movies. In addition to startups, Nvidia is developing technology which changes the video in such a way that the actor’s expression is consistent with the new language.
But there may be downsides. As Washington Post newspaper Steven Zeichik indicates thatdubbed AI content without attention to detail can lose its “local flavor”. Expressions in one language may not mean the same thing in another. Moreover, AI dubbing raises ethical questions, such as whether recreate the voice of a dead person.
Also nebulous are the ramifications of the voices generated by the performances of acting actors. Wall Street Magazine reports that more than one company has attempted to reproduce the voice of Morgan Freeman in private demos, and that studios are increasingly adding clauses to contracts that aim to use synthetic voices instead of performers “when necessary” – for example, to adjust lines of dialogue during post-production. .
Shemen positions Papercup as a largely neutral platform, albeit one that monitors the use of its platform for potential abuse (such as creating deepfakes). According to Shemen, work is underway on real-time translation of content such as news and sports events, as well as the ability to more precisely control and improve the expressiveness of voices generated by artificial intelligence.
“Meaning [dubbing] It’s clear: people retain 41% of information when watching a short video that is not in their language – with subtitles, they retain 50%, and when dubbed through Papercup, 70%. Only due to subtitles it is 40% more,” said Shemen. “With AI-assisted emotional duplication in multiple languages, truly emotional, Papercup handles all forms of content, making video and audio more accessible and enjoyable for everyone.”
Papercup currently employs 38 people in London and has a network of translators on three continents. The company expects this figure to double by the end of the year.
Credit: techcrunch.com /