Hollywood is infamous for celebrity excess, but Tinseltown strictly controls one scandalous indulgence: swearing.
Director Scott Mann encountered these constraints after shooting the thrillerFall.
But the studio had big problems.
Thirty-six of them, to be precise.
They said it had too many f*cks, Mann tells TNW on a video call from LA.
It’s free, every week, in your inbox.
Easier said than done.
Reshoots would cost a bomb and post-production magic couldnt scrub the dirty words.
Thankfully, Mann had another trick up his sleeve.
Mann asked the cast to record cleaner verbiage.
Once the audio was ready, the Flawless system went to work.
The software first converted the actors facesinto 3D models.
Neural networksthen analysed and reconstructed the performances.
Facial expressions and lip movements were synchronised with the new dialogue.
The experiment proved successful.
All 36 f-bombs were replaced without a trace.
Well, nearly all of them.
I did one f*ck in the end, Mann says.
Im allowed one f*ck, apparently.
Satisfied by his restraint, the ratings board gaveFallthe coveted PG-13.
A sequel is now shooting in Thailand.
The studio system
Early access applications for the tool are now open.
A full product release is slated for the first half of this year.
Its already altering where people are shooting, says Mann.
And as it extends out, I think its going to completely transform how we make movies.
Mann relished the experience.
It was a complete privilege.
We were very close on the movie and really happy with the English language version.
But then I saw a foreign translation of the movie.
Mann was horrified by the dubbing.
His script had been rewritten and the actors gestures had mutated.
The culprit, he discovered, pervaded across the industry.
The problem stemmed from Hollywoods established translation process.
When films are dubbed, the scripts are typically rewritten to fit the original mouth movements.
The results range fromamusing to infuriating.
And as an experience, youre not immersed if its not in synchronicity.
Mann began investigating novel dubbing techniques.
He explored head scans, but the rendering lacked realism.
The dubbing merely moved from one uncanny valley to another.
Losing faith in established VFX, Mann started searching beyond the film industry.
He soon stumbled upon a promising alternative:Deep Video Portraits.
The technique enables photo-realistic reanimation of faces using just an input video.
Each facial gesture and lip movement can then be synchronised with speech.
The life-like results stunned observers including Mann.
It blew my mind, he says.
Mann reached out to the research team.
They agreed to collaborate on a new technical test: making De Niros character speak German.
The transformation, Mann says, was like magic.
Mann believed the technique was ideal for Hollywood.
To build the idea into a business, he sought advice from Nick Lynes, a tech industry veteran.
Together, the duo co-founded Flawless in 2018.
The startups first product was TrueSync, a dubbing tool that studios are applying to Hollywood movies.
Among them isVenom: The Last Dance, a Marvel blockbuster released last year.
Flawless also showcased asizzle reelof AI-translated trailers at this years Cannes Film Festival.
Still, not every client is ready to brag about the results.
Threatening acts
As the premiere ofFallapproached, Lionsgate became anxious.
GenAI was still a novel term back then, but unions were already concerned about the threats to performers.
The studio feared the films visual dubbing would spark a backlash.
But luckily, we had planned for the consent workflows and [rights protections] early on.
Flawless built the plan on several pillars.
All thedatawould be legitimately sourced rather than scraped without permission like so many GenAI firms do.
Every output would be fully rights-cleared.
The acting would remain true to the original performances.
Any significant changes would require additional consent.
Thestartupalso restricted the systems operations.
We often call our models narrow models, says Mann.
Theyre large, but theyre focused on a specific aspect and curated for purpose.
Theyre very targeted and based on clean data that can be used for that purpose.
Flawless presented the plan to the Screen Actors Guild (SAG).
They gave it the thumbs up, says Mann.
In August 2022,Fallwas released theatrically in the US.
The film and the dubbing were big successes.
Just a few months later, GenAI exploded into the mainstream.
The trigger was the November launch ofChatGPT.
A wave of image, text and audio generatorsfollowed closelyin its wake.
Suddenly, AIs threats to actors, artists and copyrights had become public concerns.
Another take
In July 2023, the SAG-AFTRA actors union began the longest strike in its history.
One of the guilds prime concerns was the threat posed by AI.
After months of intense negotiations, the union reached a deal with Hollywoods top companies.
Mann welcomed the terms.
The new rules presented business opportunities for Flawless.
By supporting union regulations, the startup hopes to rapidly gain traction in Hollywood.
A month after the SAG-AFTRA strike began, Flawless unveiled a new rights management platform.
Named the Artistic Rights Treasury (A.R.T.
), the system shares AI-generated edits with performers.
If the actor approves the changes, they can consent within the app.
If they dont like the new versions, they can submit their own takes.
A better future for Hollywood?
Over time, Mann expects GenAI to unleash endless opportunities for filmmakers.
He envisions shrinking costs, less drudgery, and lower barriers to entry.
If all goes to plan, Hollywood will regain an appetite for originality.
The key to this industry thriving is innovating and embracing innovation responsibly, Mann says.
Yet even he has lessons to learn about working responsibly.
During production for theFallsequel, Mann has run into a familiar problem.
I accidentally have written far too many f*cks again, he sighs.
We had to have a conversation: were allowed one f*k so lets use it wisely.
Story byThomas Macaulay
Thomas is the managing editor of TNW.
He leads our coverage of European tech and oversees our talented team of writers.
Away from work, he e(show all)Thomas is the managing editor of TNW.
He leads our coverage of European tech and oversees our talented team of writers.
Away from work, he enjoys playing chess (badly) and the guitar (even worse).
