Last week’s post on the Netflix Prize brought more readers here than anything else I’ve written.

Being vain I tracked its progress – it got few readers in the first couple of days, then a burst because of being picked up by the Economist Link Mafia of Brad DeLong and Marginal Revolution – thanks to both. But the big spike came when someone picked it up from there and posted it to reddit. Then it made its way to some other tech sites like ycombinator and geekpress, and that kept things going for a while. After a week, the server’s disks are having a chance to cool down.

There were lots of good comments as well on the various places it got listed (as well as here).

ZH suggested using a Java app together with HSQL to do analysis. I’m confident that SQL Anywhere is as fast as HSQL, but you are right that you could do more with the data in an application than with simple queries like I was doing. However, in the end you have to deal with sparse matrices, and databases don’t usually store or manipulate those well.

… Continue reading

If being drives you to despair,
If your dreams are turned to dust.
If darkness waits in every meeting.
If all you build begins to rust.

Then stop a moment and be thankful.

Chances are you haven’t had a pencil in your brain for 55 years.

Continue reading

Today the Netflix Prize Competition has been running for 300 days.

Online DVD rental outfit Netflix caused a real buzz last October when it announced the competition. If anyone can come up with a recommender system for predicting customer DVD preferences that beats its own algorithm (Cinematch) by a certain amount, Netflix will hand over $1million. The prize got a lot of attention because it exemplifies the idea of crowdsourcing. Not only does Netflix rely on crowdsourcing of DVD ratings (user ratings of DVD titles) but the competition itself is an attempt to use crowdsourcing to develop the algorithms to make the most of those ratings. Instead of doing the work itself, or hiring specialists, Netflix lets whoever anyone enter their competition and pays the winner. The competition is still in progress: Netflix says it will run until at least 2011. So now the initial buzz has died down, what can we learn from the Netflix Prize?

First, the competition details (see here (PDF) for a short paper by two Netflix employees). Netflix made public a database of customer DVD ratings (tweaked to ensure privacy) that included over 100 million individual ratings of … Continue reading

There’s nothing that isn’t obvious here. I’m just so mad I have to post something.

First the Americans  say that uranium enrichment is a valid part of a civilian nuclear program, but only for countries it likes.

If you were Iran, how would you respond to nuclear proliferation among American allies?

Now we hear about a huge arms deal with Saudi Arabia (BBC) :

The United States is reported to be preparing a major arms deal with Saudi Arabia worth $20bn (£9.8bn) over the next decade.

It is said to be part of a strategy for countering Iran’s growing strength.

If you were Iran, how would you respond to additional American arms in the middle east?

Of course, the Americans realise that someone in the region is likely to be pissed off when large amounts of arms start rolling off the ships in Saudi Arabia. But they have a plan to deal with that:

To counter objections from Israel, they said, the Jewish state would be offered significantly increased military aid.

If you were Iran, how would you respond to increased American military aid to Israel?

Of course, the Americans realise that other Arab states … Continue reading

Honestly.

I lived in Oxford for a couple of years at the end of the ’80s and when I left it was in good shape. Now my sister moves there and within a month it’s a complete mess. Water everywhere.

What will it be like when she’s lived there a year? I shudder to think.

Continue reading

This is a difficult post to write.

Those of you who read the comments section of the blog will have seen quite a few comments by Dave Meleney, a libertarian from Colorado. Sadly I found out – simply because I looked at a google search that came to this weblog and clicked a link – that Dave died last Thursday in an accident while trimming trees. The link I clicked led to a brief note here.

As is the way with Internet contacts I knew a few things about Dave, but almost nothing about his real life. I knew that he was a Bonsai enthusiast who had grown huge numbers of bonsai trees, and yet I didn’t know how old he was. I knew that he had been to China, but not what he looked like.

The other thing I know is that Dave commented on weblogs a lot (both here at Whimsley and elsewhere, at Marginal Revolution for example). Most recently we had a brief exchange in which he posted a comment last Wednesday (link) that I replied to a few days later, not having a clue of what had happened in the meantime. We disagreed … Continue reading

My "Happy Shoes" series seems to have faded out one episode before I meant it to. Oh well, maybe I’ll get back to it soon. Meanwhile, here is a something a little different, which owes a lot to various posts by Chris Dillow at Stumbling and Mumbling.

The names of Vasili Arkhipov and Stanislav Petrov do not appear in most lists of 20th century heroes, but they should. After all, who else could claim to have literally saved the world?

Arkhipov’s moment came during the Cuban Missile Crisis on October 27, 1962, when he was an officer on a Soviet nuclear-armed submarine. When the submarine was bombarded by an American ship an intelligence officer on board thought "that’s it – the end" and the captain gave the order to prepare to fire a nuclear missile. Had the missile launched, nuclear war would have begun, but firing a nuclear warhead required the approval of three officers and Arkhipov prevailed on his fellow officers to wait — and things calmed down. When the story became public in 2002 Thomas Blanton, director of the US National Security Archive, said simply: "A guy called Vasili Arkhipov saved the … Continue reading