I just came back from Strata 2013 BigData conference. A pretty interesting event, considering that Hadoop wars are apparently over. It doesn’t mean that the battlefield is calm. On the contrary!
But this year’s war banner is different. Now it seems to be about Hadoop stack distributions. If I only had an artistic talent, the famous
would be saying something like “Check out how big is my Hadoop distro!”
But judge for yourself: WANdisco announced their WDD about 4 weeks ago, followed yesterday by Intel and Greenplum press releases. WDD has some uniquely cool stuff in it like non-stop namenode, which is the only ‘active-active’ technology for Namenode metadata replication on the market based on full implementation of Paxos algorithm,
And I was having fun during the conference too: we were playing the game ‘whack-a-namenode’. The setup includes a rack of blade Supermicro servers, running WDD cluster with three active namenodes.
While running stock TeraSort load, one of the namenode is killed dead with SIGKILL. Amazingly, TeraSort can’t care less and just keep going without a wince. We played about a 100 rounds of this “game” over the course of two days using live product, with people dropping by all the time to watch.
Looks like it isn’t easy to whack an HDFS cluster anymore.