How Are Movie Studios Using AI To Predict Movie Audiences’ Interests?

Movie studios use AI to predict movie audiences by scanning and labeling the objects and scenes in a film's trailer, then comparing those labels with other trailers to find ‘similar movies’ and the moviegoers likely to watch them. Studios now also use AI to forecast box office returns and inform greenlighting, casting, and marketing decisions.

Patterns are what lie at the heart of artificial intelligence (AI), which is why businesses love it. Patterns help to predict the future, and predicting the future is a great way to mint money. Although at times it’s ambiguous how these things work together, film studios like 20^th Century Fox (now Disney's 20th Century Studios) have used AI to predict what films audiences want to see by partnering with AI tech giants like Google.

Now, to be successful in the movie industry, movie studios need to attract moviegoers, but attracting huge audiences is no easy task. The movie-going audience is a diverse group of people with a wide range of tastes and interests. Conventionally, studios rely on intuition and experience when deciding to invest in a particular script. This is a risky approach, especially when the story is unique or unprecedented. Movie studios have relied in the past on high-level data processes to map out customer segments and make forecasts for the future film’s box office earnings. Also, until recently, granular predictions at the segment level have remained elusive due to the dearth of requisite technological tools and algorithms. However, with the advent of machine learning, these barriers have been overcome. Artificial intelligence (AI) is such a tool helping in this iterative and complex process of matching scripts and audiences much more accurately.

Project Merlin

20^th Century Fox, one of the most reputable movie studios in the world, used advanced machine vision systems powered by sophisticated AI algorithms to examine trailer footage and predict the likelihood of audiences watching their movie. The investigation happens frame by frame, labeling important objects and events, and then comparing them to data generated from other film trailers. The core idea is that movies with similar sets of labels/scenes will attract similar sets of audiences. Back in 2018, 20^th Century Fox partnered with Google to build Merlin (the codename for its experimental movie attendance prediction and recommendation system).

How Does It Work?

The first step for the team working on this project was identifying the right technology to power the system. The Cloud Machine Learning Engine (Cloud ML Engine), in conjunction with the TensorFlow deep learning framework, appeared to be the best choice after brainstorming by the team members. Most of the experts on the team voted for it because Cloud ML Engine is a managed service, and can automate all resource provisioning and monitoring. This implies that the team just needs to focus on building the deep learning model for Merlin, rather than devoting time in the arduous process of configuring its infrastructure.

Scanning Movie Trailer

The first step of project Merlin involves scanning the movie trailer. Consider the example of the movie Logan. Merlin scans the trailer and labels the objects, such as “tree”, “facial hair”, “car”,  “forest” etc.

While the first stage involves only scanning for labels and recording the frequency of these labels, the final data generated is much more complex, because the final data also considers how long these objects appear onscreen and specifically when they show up the video.

Labelling The Scanned Data

The temporal information obtained by the process of scanning and labeling is particularly rich, as it correlates with the film’s genre. For example, a trailer with a long close-up shot of a protagonist is more likely for a drama/romantic movie than for an action blockbuster. Similarly, a trailer with quick and furious shots is more common for an action movie. This surely holds true for Logan, with its trailer featuring plenty of slow shots of Hugh Jackman drenched in blood.

Comparing With Other Labelled Dataset

After completing the label analysis of a movie trailer (Logan in the considered example), the next stage is to compare that new analysis with the labels generated from other movie trailers to identify ‘similar movies’. There’s a presumption that there must be some overlap between the audiences for Logan and other action movies. In other words, moviegoers who have previously seen an action-packed film with a “rugged” male protagonist are inherently more likely to see Logan.

What Were The Results?

By comparing this data with analyses of other movie trailers, Merlin tries to predict what films might interest those moviegoers who saw Logan.

The table below shows the top 20 films that moviegoers who went to see Logan also watched. The right column shows Merlin’s predictions, while the left column shows the actual data (collected, based on the survey, i.e., literally asking the people directly).

The Good

As seen in the table, Merlin guesses quite a few movies correctly, including other superhero movies like Batman Vs. Superman: Dawn of Justice, X Men: Apocalypse, and Doctor Strange. It even aptly recognizes John Wick: Chapter 2 as a bedfellow of Logan. That’s an impressive result, as John Wick is certainly not a typical superhero movie. However, it does feature a similarly weary protagonist with a rugged look. Merlin identifies all five of the top choices, albeit not in the same order of importance. Overall, it was successful in identifying more than half of the films.

The Bad

That being said, there were places where Merlin erred. Merlin predicted that The Legend of Tarzan would be a hit with ‘Logan-liking’ audiences, for example. Interestingly, neither Google nor 20th Century Fox offered any explanation for this. My conjecture is that it could have something to do with the “tree,” “forest,” and “light” found in Logan, elements that the Tarzan trailer also shows in a high concentration.

Movie studios are keen on adopting AI for these sorts of analytics, and several companies are striving hard to design systems that can predict the likelihood of a movie’s success through data mining and processing. With more data and further advances in the field of machine learning, we can expect these systems to get more accurate/efficient, since the accuracy remains a challenge… for now!

How Has AI In Hollywood Changed Since Then?

Project Merlin was an early experiment, and the years since have turned AI into a standard tool across the film business. Walt Disney completed its acquisition of 21st Century Fox in March 2019, and the studio behind Merlin was rebranded 20th Century Studios in January 2020, so the "Fox" name on the original project is now part of Disney.

The bigger shift has been from prediction to decision-making. In January 2020, Warner Bros. signed a deal with the startup Cinelytic to use its AI platform during greenlighting, helping executives estimate a star’s value in different territories and a film’s likely returns across theatrical and ancillary markets (the studio stressed it was a guide for marketing and packaging, not a replacement for human judgment). Other firms attack the problem from the script itself: Belgium’s ScriptBook and Switzerland’s Largo.ai read a screenplay and forecast its box office, target audience, and even its likely age rating before a single frame is shot. Netflix, meanwhile, leans on the viewing data of its hundreds of millions of subscribers to help decide which original shows and films to commission in the first place.

And then generative AI arrived. By 2023 the technology had moved from analyzing trailers to creating images, video, and voices, and Hollywood took notice. The simultaneous WGA writers’ and SAG-AFTRA actors’ strikes of 2023 were driven in large part by AI: the deals that ended them established that AI-written material cannot earn a writing credit and that studios need a performer’s consent (and must pay) to create or use a digital replica. Studios are now pairing the analytics with generative tools too. In September 2024, Lionsgate partnered with the AI video company Runway to train a model on its own film and TV catalog for behind-the-scenes work like storyboarding and visual effects. The throughline is the same one Merlin started with: AI is very good at spotting patterns in what audiences have already watched, but predicting the next surprise hit, and replacing human creativity, remains well beyond it.