Doug Cutting talks about Hadoop, and open source

By Bob Reselman, ITworld |  Software, Hadoop

Cutting: LISP became object-oriented in the years that I was using it. I did a lot of object-oriented programming in LISP in my days at PARC. I worked at Apple for a few years, and learned to develop in C++. I wrote a full text search engine there. I think that it was used in Spotlight, eventually.

ITworld: Did you get OOP right off the bat?

Cutting: There wasn't a lot of object-oriented programming that I ran into as an undergrad. We had an assignment in a LISP class to build a system that was object-oriented and I was totally baffled by it. I remember in retrospect being totally at sea in that assignment, and I find it humorous years later that I totally failed to get it.

Working on Interlisp there were object-oriented patterns. I became very comfortable with the patterns. By the time I ran into it in C++ and Java, it seemed very natural. There wasn't a big leap to be made. But, the first time I saw [OOP], it was definitely very strange to me, and I did not get it.

ITworld: I've come across SQL programmers that have trouble with the Java end of Hadoop, and report feeling a bit "not good enough" as a software developer. Do you find this self-deprecation to be a fallacy?

Cutting: It's definitely a fallacy. The way I've always thought about the object-oriented [paradigm] is that it's a pattern, that even if you don't have support for it in the language, it's a way of structuring software. Object-oriented languages make it easier.

ITworld: When you were doing Hadoop, did you have any idea that it would become a hit?

Cutting: No. I saw this marvelous technology, but you couldn't take advantage of it unless you worked at Google, which is a pretty small portion of the software development community. The papers were hits. But to a large degree they were irrelevant because nobody had [the framework to use it], except for these guys. So I thought, I know how to solve that. You write an Open Source implementation and everyone will have it.

I was interested in providing the world with great tools to make search engines. I was excited to be able to try to bring this technology to the world through Open Source. Google was able to come up with the ideas behind [Hadoop]. But, because of the way they were structured, they weren't able to give it to the world, except as ideas, which is great. They could have kept it as their secret sauce, but they choose to promulgate it.

ITworld: What were your boyhood ambitions?

Join us:






Answers - Powered by ITworld

ITworld Answers helps you solve problems and share expertise. Ask a question or take a crack at answering the new questions below.

Ask a Question