Saturday, November 22, 2008

The Netflix Recommendation Problem

I have been a Netflix user off and on for well over five years. I briefly cheated on Netflix when Blockbuster started their online service, but I found Blockbuster to be lacking for at least three reasons.

1. I often got the wrong movie.
2. I often received a damaged movie.
3. Their website was TERRIBLE.

One of the things I hated about their website was its recommendations. Even after I rated numerous movies, it would still recommend movies I KNEW I would hate...Judge Dredd, anyone?

Netflix has created a recommendation engine that is top notch. I can nearly guarantee I will like a film Netflix recommends to me. On the other hand, I have, a few times, had friends and acquaintances recommend a movie and, when I looked on Netflix, the website suggested I wouldn't like it. What did I do? I rented the movie, of course. I can trust my friend over a computer right?

Wrong. Netflix is inside my head. They know my movie predilections better than my friends do. Only once, that I can remember, has the site ever steered me very wrong. It said I would really like "Secretary," and I didn't like it much at all. Other than that, Netflix's recommendations have been stellar.

Right now, I am fighting the urge to rent "American Gangster." Many people have recommended it to me, but Netflix says raters like me have only given it 2.9 stars. I've rented 2.9 star-type movies before, and, while I don't hate them, I have decided not to waste my valuable viewing time on movies that make me go "meh". Sorry, friends, I gotta go with Netflix.

The NY Times recently published a story about Netflix's offer of a one million dollar prize to the person/team who can improve Netflix's recommendation algorithm (called Cinematch) by ten percent. Many people are trying now, and the team in the lead has improved it by 9.44%. What keeps all the programmers from the 10% grail is Napoleon Dynamite...and movies like it. These films are polarizing films, quirkly films, ones that tend to get only a one star or a five star rating. According to the Netflix message board, following are the films that produce the most difficulty for the programmers (notice "Secretary" is on the list).

"Fahrenheit 9/11"
"Napoleon Dynamite"
"What the #$*! Do We Know!?"
"Punch-Drunk Love"
"Lost in Translation"
"I Heart Huckabees"
"The Royal Tenenbaums"
"The Life Aquatic with Steve Zissou"
"The Passion of the Christ"
"Kill Bill: Vol. 1"
"Anchorman: The Legend of Ron Burgundy"
"The Office: Series 1"
"Bad Santa"
"Before Sunset"
"Sin City"
"Open Water"

I notice a few patterns among the films.

1. There is a notable lack of blockbusters. The only one I see is Armageddon.
2. Many are more "contemplative" films, like Lost in Translation and Before Sunset.
3. There's only two films that might even be considered political: Fahrenheit 9/11 and The Passion of the Christ. The rest of the films divide us for aesthetic reasons, it seems.
4. The only director I see with two movies on the list is Wes Anderson, with The Life Aquatic (which I didn't like) and The Royal Tenenbaums (which I did like).

I could probably look for more, but I'd like to hear what you think. Any surprises? Patterns?


Anonymous said...

Obviously, Fahrenheit 9/11 and Passion are polarizing because of their one-sided views. If you're a conservative you won't like F-9/11 and vice versa with the Passion. But then wouldn't that make it easier to recommend those movies? If you watch the Ten Commandments you'll probably like the Passion. If you liked Inconvenient Truth you'll like F-9/11. I would think those two movies would be easy to recommend.

Average movie goers don't seek out contemplative quirky films which make up the bulk of this list in my view. These film seems to be viewed by avid film buffs whose critical analysis skills are able to distinguish fine nuances and therefore a wider range of likes and dislikes making the blanket recommendations difficult to categorize. Interesting article.

Eric said...

After further discussion with my wife, she points out that I am the one presenting Netflix a problem (not me personally, but people like me).

She says that because people like me look at films very deeply, we don't have any particular genre we like or hate--we can be picky WITHIN a genre. For instance, my mother in law would hate all Wes Anderson movies. Many of my students LOVE all Wes Anderson movies. However, I like Wes Anderson movies in varying degrees: I love Rushmore, like The Royal Tenenbaums and disliked The Life Aquatic. Another film buff might love Wes Anderson and put his movies in a completely different order. How is the algorithm going to account for that?

Anonymous said...

My two cents:
I would say that I heart Huckabees is an attempt at a Wes Anderson movie, so it could be included in that list.
Also, what number of these would you consider having cult appeal? Don't you think that if films like Repo Man or Cry Baby were more recent then they would make this list. The ones that jump out as culty are Napolean, the Andersons and Huckabees, Kill Bill, Sin City, Secretary, and Memento. Those seem to appeal to different audiences but in similar ways _ the types of movies that would be at the top of some peoples lists from when they are in high school until they actually watch them again when they are in their thirties and say - "meh."

MC said...

Well, I actually liked American Gangster and recommend it, despite the fact that Denzell Washington was totally miscast-he always brings too much dignity to a role to play a believable gangster. The gritty New York of the 70s(reminds me of films like Serpico and The French Connection) and excellent supporting performances by people like Josh Brolin made it worthwhile for me.

I agree that not being consistent by genre or director causes problems. I also agree with your assessment of Wes Anderson-Dharjeeling Express was pretty lame as well in my opinion.

I see a lot of movies where you have to buy into the schitickt or premise of the film to enjoy it. Meta-films like Memento and Adaptation, comic book-like ultra violence in Oldboy, Kill Bill: Vol. 1, and Sin City, dialogue driven films in which very little happens like Lost In Translation, Before Sunset, Sideways, etc..., comedies of a certain love-it-or-hate-it style: (one trick pony: Anchorman, cringe inducing: The Office: Series 1, Napoleon Dynamite, ultra ironic hipsterism: Wes Anderson movies and I Heart Huckabees, dubious/bad taste humor: Bad Santa), a film that takes place in water with a steady cam: Open Water, a musical: Chicago.

Just my two cents.

Nick S. said...

Another problem with NetFlix is that it doesn't differentiate between multiple users on a single account. For example, my wife, kids and I rent under the same account, so Netflix is getting a variety of combined opinions that may or may not be representative of any of us individually. So the movies it recommends are often useless to ALL of us. What I would recommend, is to be able to have multiple log-ins under the same account.

That way Netflix doesn't have to make sense of me giving "Reservoir Dogs" 5 stars right after my daughter 5 stars "Dora and the Snow Princess." Netflix must think I am a sociopath.

Eric said...

OK Pat, I'll rent American Gangster, but if we disagree, I'm leaving you for Netflix.

Nick, you bring up a really good point. I'll bet the recommendation system would get a lot better if they stopped giving accounts to sociopaths. Seriously, though, I bet they could solve many problems with the recommendation software by taking your recommendation. I hope the people at Netflix have been paying close attention to my blog comments.

Jake D said...

As I read through the list of films, I did notice a pattern, of sorts. Several of these films stand alone in style or cinimatic "technique". These films have some kind of uniqueness about them. Take Memento. The use of reversed chronology is pretty rare in film. I loved it, but I've have talked to others that hated it. They found it confusing and hard to follow.
Looking at the list, I find it almost impossible to think of OTHER films that are "like" the films on the list. I imagine that it would make these films difficult to catagorize. In that respect, it makes me want to like these movies even more. (except Armageddon... that shit was rediculous)

BS said...

I'm going to drop a couple of non-sequiturs, because that's what blog comment fields are for. Right?

First, I usually find myself reacting rather indifferently to movies that people market as "You'll either love it or hate it".

Of the 20 or so movies on the list I've seen, I've rated 11 of them a 3.

Secondly, you should check out 'Kiss Kiss Bang Bang' and 'Brick' if you haven't. Let's see cinematch slander those gems.


Lane said...

I understand that you are probably pickier than the average Netflix user, and won't like all the movies in a given genre, by a given director, or which have recieved favorable reviews from any one critic. Does that mean you are more likely to give movies 1's or 5's, or does that mean you are even more selective about what you truly love and truly despise? Because, I would think that giving a lot of movies three stars would fail to register a big change in their algorithm.

Also, out of curiosity, how many movies have you rated? I talked to Aaron about this (since he actually uses Netflix--I just use the Seattle library), and he said he had rated well over a thousand, but that his recommendations were still pretty hit or miss.

Eric said...

I have rated 1247 movies so far.

I don't give many movies a '1' because I am pretty sure, when I rent a movie, that I will like something about it. Even the egregious 'Paranoid Park' got a 2 from me. I don't give a lot of 5s either.

As for Aaron, does he recognize a pattern about the movies Netlix is wrong about? For instance, because I have liked a few French movies, Netflix recommends A LOT of them, so I'm sure there would be some of those I wouldn't like.

Also, I think the ratings are accurate for me because I have clicked the "not interested" button on at least as many movies as I have rated, so Netflix is able to not only know the movies I like and don't like, but also the ones I have no interest in seeing either way. I think that helps their algorithm a lot.