July 26, 2011, 10:59 PM —
Doug Cutting has changed the way that IT does Big Data. Hadoop, the Open Source project he started, has made it so that any company with access to a rack of commodity PCs and a reasonable amount of programming skill can do the type of large scale data analysis work that was previously done only on supercomputers. Enterprises such as Amazon, eBay, Facebook and IBM, all the way to the Federal Reserve Board of Governors are taking advantage of tremendous value offered by this Open Source hit. Hadoop is a game changer.
A lot has been written about Hadoop's technology. We wanted to go one step further, to learn about the man who's made large scale data analytics an everyday part of the IT experience.
ITworld: How did a guy with a linguistics degree from Stanford create an Open Source hit?
Cutting: The linguistics degree is a little deceptive. There was no Computer Science undergraduate degree offered at Stanford when I was there. You could go into electrical engineering. Other options where math, philosophy or linguistics, all of which involved studying computation. So I ended up taking a lot of Computer Science courses as well as Linguistics courses. I think that my sub-major was something like, Computational Linguistics.
ITworld: How did you get to Open Source?
Cutting: After close to 15 years in the software business, I had a piece of software that I'd written on my own time, figuring that I would commercialize it. That was Lucene. I wasn't very interested in building a business. Negotiating license fees and paperwork around that was stuff that I didn't enjoy. What I really wanted was for people to use the software, which was a theme I found through my career.
I had been involved with Excite in the 90s. I'd gotten to the point where I spent many years writing software there and the software was gone from the Earth for all practical purposes. The company went bankrupt and all the software was swallowed into some intellectual property black hole.