Can SEI really teach you how to be Hadoop contributor?

Or of anything else for that matter?

I am kidding you not… I just got this email from SEI. In the interest of full disclosure – here it is:

To the attention of:

The Software Engineering Institute (SEI) has been asked to conduct a sample survey of committers to the Hadoop Distributed File System. The results will be used to supplement existing documentation that can be used in providing guidance to HDFS contributors as well as support committers in preparing their own HDFS contributions.

You are part of a carefully chosen sample of HDFS committers for the survey. So your participation is necessary for the results to be accurate and useful. Answering all of the questions should take about 15 or 20 minutes. Any information that could identify you or your organization will be held in strict confidence by the SEI under promise of non disclosure.

You will find your personalized form on the World Wide Web at Please be sure to complete it at your earliest convenience — right now if you can make the time. You may save your work at any time, and you may return to complete your form over more than one session if necessary for any reason. Everything will be encrypted for secure transfer and storage.

Now, let’s follow the link and dig out some pearls which, I am sure, has to be in the work of such a venerable organization. What are they covering exactly?

  • Reducing unnecessary dependencies and propagation, e.g., identifying cyclic dependencies between classes in the source code 
  • Difficulty in managing data
  • Difficulty in managing namespaces
  • Identifying location of bugs
  • difficulty finding test suites
  • Communication between application
  • Reducing unnecessary dependencies and propagation
  • yada-yada-yada

Ah, I think I got the picture…. boring… 1534th research in a row on how to write effective code. Something, I like in particular:

  • “You are part of a carefully chosen sample of HDFS committers” – no shit, there’s a plenty to select from, of course.
  • Are you familiar with the (HDFS) Architectural Documentation at” – what? Are you kidding me? How the architectural docs for an ASF project ended up there? Has the design came from Hawaii? Or you could not found it where the project belongs – on Apache site?

Here’s the news, my dear doctors from SEI: just try to sit and write the code, learn from others; grok the best gems written by bright practitioners. That’s pretty much what it takes – one doesn’t need nothing like CMMI in order to create great software. I will let myself to make even a stronger assertion: one needs processes in place to make a bunch of ineffective and inexperienced folks to produce something useless that can be later sold to an idiot customer with a lifetime of support fees attached.

Meanwhile, the reality is that today you see the ratio of three software “managers” graduated by US universities for every decent developer who doesn’t need help in the day one to find his own butt with both hands, a GPS navigator, and a flashlight.

The main reason an open source software is thriving today and constantly kicking ass of companies with established processes is because people aren’t afraid to fail nor to experiment on their own dime and time. In other words, they don’t give a shit about CMU teaching them how to write great code – they just learn it in the field and then do what it takes by learning from others. You don’t a formal training for that, clearly. Perhaps, Khan Academy is what really need.

You know that old saying “If you can’t do a job – go to management; if you can’t manage then teach”. I would amend it by “…; if you can’t teach – go to research of software processes”.

Although, I won’t be totally surprised to see some fat-ass book on how to contribute to Hadoop coming out from CMU very soon. And it might even become a best seller on Amazon or something. But I know for sure that by the time OSS community will be far away onto making the next great thing!

And the other day I shall tell the story of that grad student from Berkley who was all set to write the greatest benchmarking “solution” for Hadoop – that deserves a separate post, because the guy was learning from CMMI, I guess.

Am I too acidic today? Must be this damn sunny California weather or something.


Author: DrCos

Dao-Clinicist, Groovy mon, Sprechstallmeister / Concerns separator / 道可道 非常道 / Disclaimer: all posts are my personal opinion and aren't of my affiliations

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s