95th Academy Awards Ceremony

On Sunday, the 95th Academy Awards ceremony came to an end without a major incident. This normally runs against the notion of “there’s no bad publicity”, but we won’t complain about the lack of violence or foot-in-your-mouth moments like announcing the wrong winner. Per Jimmy Kimmel, the Academy listened to its members and made the controversial decision of presenting all the awards on stage in one go — even documentary shorts. This meant the ceremony was nearly four hours long. Despite that, the viewership rose 12% from last year to 18.7 million — the best since 2018. Good on them!

Oscar Winners

With seven Oscars out of eleven nominations, the indisputable king of the night was the fan-favorite small-budget movie picked up by the independent studio, A24: Everything Everywhere All at Once. Having cost $25 million, it grossed over $106 million worldwide. Everything Everywhere was released much earlier in the year than a typical Oscar winner but it carried its momentum admirably as it gained more and more fans. In the process, Michelle Yeoh became the first Asian woman to receive the Best Actress statuette for her leading performance. The other notable moments of the night included great comeback stories in Brendan Fraser winning Best Actor, Ke Hu Quan (Best Supporting Actor), and Jamie Lee Curtis (Best Supporting Actress) after long periods of inactivity. America does love second chances.

Prediction Results and Analysis

So how did we do with our predictions? The short answer is we got six of the eight major categories right including one of the two screenplay categories that proved trickier over the years. In the secondary categories, which we successfully added last year, we had six out of eleven correct this time for a total of 12 on-target predictions out of 19 categories for a 63% overall hit rate.

All Quiet on the Western Front did better than our models expected through the night adding to Netflix’s award circuit success in recent years. All in all, the German production took four awards and Netflix reached a grand total of six including Best Animated Feature (Guillermo del Toro’s Pinnochio) and the Documentary Short Oscar. With that said, three of the five secondary category misses for us ended up in the hands of the second-best nominee according to our models. We will continue polishing those newer models for next year and share the predictions with you moving forward.

On the other hand, if done randomly, getting 12 out of 19 predictions right with anywhere between 5 to 10 nominees for each category is still equivalent to finding the one correct combination out of 244,140,625 possible combinations. While not perfect, our models do pick up many valuable signals pointing to eventual winners. The tables below summarize the prediction outcomes per category. Of the two major award category misses, we already knew the Best Actress Oscar would be a close call and it went to our second pick, Michelle Yeoh. However, Best Original Screenplay also going to Everything Everywhere All at Once was more of a remote outcome per the corresponding model.

Evaluation and Methodology

This year we relied on Fusion models combining the Top 20 models our OptiML automatic classification model searches picked as best performing. Next time, we can look into different combinations such as Top 5 or Top 10 models as one possible variation as far as the methodology is concerned. Another angle for us to consider in the next iteration is coming up with a particular evaluation harness that overweights more recent years. This is not necessarily standard practice, but if we have reason to believe things have changed significantly in the Oscars context it’s worth exploring, e.g., changes to the Academy’s voting member body, the inclusion of more commercially successful movies like Top Gun: Maverick and Avatar: The Way of Water as major category nominees, streaming services like Netflix making headway with their original productions at the expense of traditional studios.

Note that our method for making predictions is completely transparent and documented for all the categories we predicted so we always welcome our users to come up with creative ideas of their own including adding new data points to further enrich our public dataset. This is consistent with our experience since 2011 which has shown investing in data quality and data engineering are more often than not the smartest choices to overcome bottlenecks. Plus relying on the distinguished set of best-in-class methods the Machine Learning community has been able to collectively produce over the decades.

History to Date Predictions Performance

We’ve also updated the cumulative table below that compiles all our predictions between the 2018 and 2023 Oscars and the corresponding hit rates for the major categories. In addition to the Top Picks that we annually shared in our past blogs, this table lists how the accuracy metric improves if we also consider the movies that received the highest two (Top 2) or three (Top 3) scores. The Top Picks alone had an average 70% hit rate, whereas the coverage reaches 93% with the Top 3 taken into account.

As the pioneers of ML-as-a-Service here at BigML, we invite many more of you to put your Machine Learning skills to the test quickly with this very approachable skills practice use case and do so without the overhead of having to download and install many open-source packages worrying about compatibility issues or hard-to-decipher error messages. It takes just 1 minute to create a FREE account and about as much time to clone the movies dataset to your account. As always, let us know how your results turn out at [email protected]!