Slumdog Millionaire is one my favorite movies of all time. And I have followed the career of A.R. Rahman, who composed the movie's music, for several years ever since his debut in 1992. So I was quite thrilled when Slumdog was nominated for 10 academy awards -- and Rahman in two categories, Original Score and Original Song. Thrilled, and a little surprised: while I like Rahman's work in Slumdog, I don't think it's his best work. There is of course nothing wrong with that, as long as Rahman's work is better than that of his competitors this year.

But it got me to thinking: if Rahman had composed the same music for an obscure film this year, rather than for Slumdog Millionaire, would he have been nominated? And even if he had been nominated, what are his chances of winning? In other words, is there a Matthew Effect in Oscar nominations -- to them that have, more shall be given? And, once nominated, is there a halo surrounding movies with many nominations that improves the odds of winning across many award categories? I thought it might be fun to run the numbers based on past years' nominees and winners to see if I could find answers to these questions; it turned out to be somewhat instructive as well, since it required an extension of the standard Market Basket analysis from the world of data mining.

To get the data, I went straight to the source: the official Academy Awards database , which lists all the nominations and winners for the past 80 years. Unfortunately there is not a single page that lists all this information, but it was fairly straightforward to write python scripts that queried the website a few times and collated the data in tabular form. The result: a table that lists every nomination and winner in every category beteen 1927 and 2007. There were 8616 nominations in the period, representing 4215 distinct movies; so each movie was nominated on average for 2 award categories.

Let's start first with the nominations, to see if there is any evidence of the Matthew Effect. Let's say N(k) is the number of movies with exactly k nominations. The table below shows k and N(k) for k between 1 and 10. If we ignore two outliers (k=1 and k=7), it appears that N(k+1)/N(k) is close to 0.6 for k between 2 and 10; the decay is certainly much slower than exponential. This indicates that the number of nominations roughly follows a power-law; and a power-law is the classic embodiment of of the Matthew Effect, arising in contexts such as income and wealth distribution. The table below summarizes the data.

Nominations | Movies |

1 | 2796 |

2 | 513 |

3 | 260 |

4 | 195 |

5 | 128 |

6 | 81 |

7 | 87 |

8 | 50 |

9 | 31 |

10 | 29 |

The next step is to enquire whether there are Oscar categories for
which the effect is much stronger than for other categories. To study
this, we divide the nominated movies into two groups: movies with 4 or
fewer nominations (the "poor" group) and movies with 5 or more
nominations (the "rich" group). Overall, 5382 nominations, or
62.5%, went to movies in the poor group and 3234 nominations, or 37.5%,
went to movies in the rich group. Now, let's look at the major Oscar
categories. The major outliers are Best Picture and Best Director -- both
nominations went overwhelmingly to movies in the rich category (70% and
73%, respectively, compared to the average of 37.5%). This is not
surprising, because the best picture is typically one that is strong in
many disciplines. There is some bias in the acting categories as well,
but the big surprise is Film Editing: 68% of the nominations in this
category are "rich" movies. At other extreme are Music and Special
Effects: approximately 70% of the nominated movies are in the "poor"
category. So it appears that in these categories at least, talent gets
its due without help from Matthew.

Moving from nominations to actual winners, the obvious question is:
does being nominated in many categories boost the chances of winning in
a disproportionate manner? To study this, I used the Market Baskets
approach from Data Mining. In a classic Market Baskets scenario, we ask
which items are often purchased together: such as milk and eggs. In
this case, we model each movie as a basket: the contents of a movie's
basket are its nominations and wins. Do movies with many nominations in
their baskets have a disproportionate number of wins?

We must first deal with a technicality. In a normal
market basket scenario, the contents of each basket are independent of
every other basket, but in this case there are dependencies. Consider
the set of market baskets of the movies that have all been nominated in
a single award category in a particular year; clearly, one of these has
to be the winner in that category, and so the basket of that movie will
also contain a win in that category.

It's easy to extend the Market Baskets model to capture
this idea. I'll call the new model Constrained Market Baskets. Consider
a subset S of market baskets; say, the set of market baskets
corresponding to the "rich" movies with 5 or more nominations. Suppose
movie M is in this set, and has been nominated in award category C. If
there are (say) a total of 5 nominees in this category, then the prior
probability of movie M's basket containing a win is 1/5 or 0.2. We can
repeat this for all the categories M is nominated in, and add up the
priors; this gives the "prior expected value" of the number of wins in
M's basket. We add up the expected wins for all the movies in set S to
get the total number of wins we expect the set S of movies to have;
call this EW. Now, if OW is the actual number of "Observed Wins" across
the movies in set S, we want to see if there is a discrepancy between
EW and OW. In particular, we define the "win boost" of set S to be
OW/EW. If the win boost is higher than 1, then the set S of market
baskets has a disproportionate number of wins, and if it's much less
than 1, then it has fewer wins than expected.

When we do the analysis, the set of "poor" movies, with 4 or fewer nominations, had a total of 5382 nominations, with 1143 "expected wins" but only 840 "observed wins"; a win boost of 0.73. The "rich" movies, by contrast, with 3234 nominations, were expected to win 657 Oscars but actually won 958, a win boost of 1.46. **In
other words: the rich movies, which represent only 37.5% of all
nominations, actually won more than half of all the actual Oscar awards!
**Matthew!

Once again, we can break up the results by category, and look at the
win boosts for specific categories of awards. For most major award
categories, the win boosts for the rich and poor categories are in line
with the overall average boosts. As in the case of nominations, the
effect is very significant in the best picture and best director
categories: in these categories, the "poor" movies have a win boost of
just 0.30! We noted that the Music category seemed resilient to Matthew
in the case of nominations; but in the case of wins, this category has
a win boost of 1.7 for the rich movies, in line with the overall
average. The surprising and significant outlier in this case is the
Best Supporting Actor category, with win boosts very close to 1.0 for
both the rich and the poor movies. It appears that the Best Supporting
Actor award shows no evidence of Matthew; the other acting categories,
however, are in line with the overall averages.

I don't have a deep enough understanding of the movie industry and the
Academy Awards process to speculate on the reasons for these effects.
Perhaps great talent attracts other great talent, and the Awards
reflect that reality. And perhaps the difference between the behavior
of wins and of nominations has to do with the fact that the former uses
simple plurality voting while the latter uses a preferential voting
scheme. In any case, I'm happy on two counts. The statistics on the
Music category say that the Matthew effect likely did not help Mr
Rahman in securing his nominations; but now that he has been nominated,
his chances of winning are greatly boosted because he is associated
with Slumdog's 10 nominations. Jai Ho!

**Update:** A big night for Slumdog, winning 8 awards, including both the music and song awards for A. R. Rahman. While 8 awards is not the best Oscar performance ever, it is the most number of awards won by a movie with 10 nominations (the ones that won more awards had more nominations). Matthew must be pleased.

Fascinating. Does it also hold for Golden Globes?

Posted by: mailman | February 21, 2009 at 06:35 PM

Calling it the halo effect implies that it is the act of judging which is colored by cross-category success, but I think the biggest underlying cause is simply that top talent works together!

Posted by: Q dub | February 21, 2009 at 07:34 PM

Good one ! that's one way to look at things

Posted by: Nikhil Verma | February 22, 2009 at 09:52 AM

excellent analysis!

Posted by: 5m1t | February 22, 2009 at 10:42 AM

Anand - Indeed the halo effect may be operating at several levels, and compounding at each. As you write "Perhaps great talent attracts other great talent" -- e.g. the Matthew effect is at work in movie-making, in addition to award-giving.

We all know a good wine tastes far better when paired with a great meal -- movies are no different.

MD

Posted by: Michael E Driscoll | February 22, 2009 at 03:15 PM

Mr A.R. Rahman Won! Another data point to support the analysis. However, he is a genius and truly deserves it. I am so excited and proud :)

Posted by: Abhishek | February 22, 2009 at 07:59 PM

two oscars for rehman tonight. As the world recognizes a humble genius ..

Anand the academy is a panel of members and withing each of their heads they make which one is a better movie than the other and might prefer to vote for all the nominations of that particular film over choosing for each category.

Posted by: mailman | February 22, 2009 at 09:01 PM

Rumor has it that Subhash Ghai rejected Jai Ho, originally meant for the movie Yuvraj. The song that was on the B-sides of a B-movie turns out to be the best original song at the Oscars. Surely smells like Mathew?

Posted by: Righthalf | February 22, 2009 at 11:14 PM

awesome analysis !

Posted by: Balu | February 23, 2009 at 01:55 AM

Anand,

Great stuff. But I was wondering if you could share the Oscar list? I am a movie buff and curious to see how many of those movies I have seen.

Is it possible to email me (raonikhilesh (at) gmail )or upload it your blog.

Nik

Posted by: Nik | February 23, 2009 at 03:56 AM

I typed up a longish comment. Then I decided I should just blog my comment: http://justlanded.wordpress.com/2009/02/23/oscars-and-the-matthew-effect/

Posted by: anand r | February 23, 2009 at 09:38 AM

Interesting point is that unlike milk and eggs example, the awards are voted on by independent groups of people (I hope!) with the results a secret.

It is true, though, that the nominations were public and that could've led to a bias even among disparate groups.

Posted by: Siva | February 24, 2009 at 09:44 AM

RightHalf: That's an interesting piece of information, if true -- was Jai Ho really a Subhash Ghai reject? Is there some evidence of this?

Nik: Always happy to share data. Please email me (see About page for email address).

Anand R: Nice post! My favorite line: "You have to find the right combination of Hollywood liberal guilt, Hollywood elitist condescension, and Hollywood self-preening and then make it work in your movie’s favor. If all of those are pointing in your direction, you win."

Posted by: anand_rajaraman | February 24, 2009 at 01:20 PM