Psychologist Beating Math Nerds in Race to Netflix Prize

s1d writes "An almost-anonymous British psychologist named Gavin Potter has suddenly risen to the top of the Netflix prize charts. With his very first attempt, he got a score which took the BellKor team seven months to reach. Currently at a score of 8.07, he has only five teams ahead of him now in the race for the ultimate Netflix algorithm. 'Potter says his anonymity is mostly accidental. He started that way and didn't come out into the open until after Wired found him. "I guess I didn't think it was worth putting up a link until I had got somewhere," he says, adding that he'd been seriously posting under the name of his venture capital and consulting firm, Mathematical Capital, for two months before launching "Just a guy." When he started competing, he posted to his blog: "Decided to take the Netflix Prize seriously. Looks kind of fun. Not sure where I will get to as I am not an academic or a mathematician. However, being an unemployed psychologist I do have a bit of time."'"

Average psychologist

[Anonymous Coward]

that had an undergraduate degree in psychology, a masters degree in operations research [wikipedia.org] that after being well employed for a number of years -- "In 2006, he left his job at IBM to explore the idea of starting a PhD in machine learning, a field in which he has no formal training. When he read about the Netflix Prize, he decided to give it a shot -- what better way to find out just how serious about the topic he really was?"

Misleading summary

[Zen Programmer]

The summary makes 2 references to Gavin Potter being a psychologist, but it ignores the part of the article that notes he has a master's degree in operations research. This is very much an OR problem. Still, it is impressive that he has been able to do as well as he has considering his competition. Good luck to him!

Umm....

[palegray.net]

He might be a psychologist, but his venture firm is named Mathematical Captital [mathematicalcapital.com], after all. His partners appear to have advanced degrees involving mathematics.

Re:how many GB is the dataset?

[Paeva]

The dataset is about 660MB to download. It unpacks to 2GB of about 18,000 text files.

Re:how many GB is the dataset?

[Paeva]

There are 480,189 customers that rated 17,770 movies. The total number of ratings that you're given is 100,480,507. Each user/movie/rating is accompanied by the date of the rating, as well. You then have to submit predictions for the ratings of 2,817,131 additional user/movie combos (they tell you the user, the movie, and the date, and you need to predict the rating). You submit these predictions to Netflix, and they tell you the root-mean-square error between your predictions and the actual ratings that those users gave those movies.

Re:how many GB is the dataset?

[Paeva]

Each of the movies and each of the users represented in the test set have some corresponding known ratings. I think the minimum is around 10 known ratings for users and maybe 3 or 4 for movies. And there are examples on the opposite end of the scale... one user has rated nearly every movie in the training set, and most of those ratings were 1 star. If you have any more questions, feel free to check out my blog - my Slashdot profile links to it, and you can find my email there. I'm Dan Tillberg, currently in seventh place in the contest ;). I'd love to help answer any questions you or anybody else might have (or at least help point you to good resources), and I certainly encourage anybody that's interested to try their hand in the contest :).

Re:Free Idea

[justinlee37]

Contest participants retain ownership of the code they write, but the winning team must license it (non-exclusively) to Netflix.