Rise of Robo Voices

If we can insert voices, and are starting to insert believable synthetic 'acting', it may not be long before the entire experience will be fabricated?  

ACM NEWS

The Rise of the Robo-Voices

 The next time you see a movie or TV show that was dubbed from a foreign language, the voices you hear may not belong to actors who rerecorded dialogue in a sound booth. In fact, they may not belong to actors at all.

Highly sophisticated digital voice manufacturing is coming, and entertainment executives say it could bring a revolution in sound as industry-changing as computer graphics were for visuals. New companies are using artificial intelligence to create humanlike voices from samples of a living actor's voice—models that not only can sound like specific performers, but can speak any language, cry, scream, laugh, even talk with their mouths full. At the same time, companies are refining the visual technology so actors look like they are really speaking.

As streaming services export American fare globally and foreign markets send their hits to the U.S., dubbing is a bigger business than ever. But the uses of synthetic voices extend well beyond localizing foreign films. AI models can provide youthful voices for aging actors. The technology can resurrect audio from celebrities who have died or lost the ability to speak. And it can tweak dialogue in postproduction without the need for actors.

All the tinkering raises thorny ethical questions. Where is the line between creating an engrossing screen experience and fabricating an effect that leaves audiences feeling duped?

From The Wall Street Journal  

Generating Videos

Video synthesis to supplement with real world data.

IBM’s AI generates new footage from video stills
Kyle Wiggers @KYLE_L_WIGGERS in VentureBeat

A paper coauthored by researchers at IBM describes an AI system — Navsynth — that generates videos seen during training as well as unseen videos. While this in and of itself isn’t novel — it’s an acute area of interest for Alphabet’s DeepMind and others — the researchers say the approach produces superior quality videos compared with existing methods. If the claim holds water, their system could be used to synthesize videos on which other AI systems train, supplementing real-world data sets that are incomplete or marred by corrupted samples.

As the researchers explain, the bulk of work in the video synthesis domain leverages GANs, or two-part neural networks consisting of generators that produce samples and discriminators that attempt to distinguish between the generated samples and real-world samples. They’re highly capable but suffer from a phenomenon called mode collapse, where the generator generates a limited diversity of samples (or even the same sample) regardless of the input. ... "