eHarmony uses Oracle to do much of the initial matching. But for its hardcore data-processing, eHarmony relies on a 50-node Hadoop cluster. Hadoop is speedy, says Essas. "What used to take hours now with Hadoop takes just 30 minutes," he said.
That's important, because Harmony is rescoring its relationship matches whenever new members sign up, or even when existing members update their profile.
Hadoop also forces eHarmony to keep its data in key-value store form, rather than in a structured SQL format.
"It's really hard to build reusable data structures, especially at scale, in SQL," he said. Using Hadoop also makes it easier to figure out the cause of slow queries compared to using a SQL database, he said. And it forces eHarmony's developers to be more disciplined about what data it stores permanently, preventing the database from getting "too bloated."
eHarmony also uses Hadoop to do real-time analysis of the effectiveness of its TV ads.
"We know exactly how much each commercial yields in new customers," he said.
eHarmony's core technical team includes 40 engineers in its Los Angeles headquarters, along with the same number of developers based at various Eastern European firms.
Essas, who is of Russian origin, says he likes working with Eastern European developers because "they will challenge you on what you ask them to do."
"They will tell me, 'Joseph, this is just stupid,' and I will say, 'What? Aren't I paying you?' But this enhances our creativity overall," he said.
A former engineer on Yahoo's search team, Essas said there are a lot of similarities between "matching people and matching search queries to people."
Asked if eHarmony would ever consider taking its love-match algorithms into other match-making endeavors, he said: "We would definitely consider it. We are experimenting and playing with different businesses."
The main prerequisite, Essas said, is that there potentially be a critical mass of users. "You need millions of users to create statistical meaning and make it very successful."