After posting the latest story about the Hudson/Jenkins split last Wednesday, I thought that the two projects would go on their separate ways with developers and users from both camps settling in to work on each project in relative peace.
Apparently, it seems, not.
I was contacted by a representative of the software company Sonatype later that day, who wanted to know if I'd like to hear "alternative view on the topic." The notion that the Jenkins community represented a broad movement to break Hudson code away from Oracle, I was told, did not comprise the entire story. Not wanting to just give one side of any story, I readily agreed to a meeting.
Which led to my phone interview with Sonatype founder and Chief Technical Officer Jason van Zyl on Thursday. van Zyl, in case you're not familiar with the Java community, is also the creator of Apache Maven, which can use Hudson as a continuous integration tool. So his interest in the goings-on with Hudson and Jenkins is rather high.
van Zyl has several concerns about the break of Jenkins from the Hudson community, not the least of which is how a 228-voter election could decide the fate of Hudson. The 214 for/14 against vote did not represent the over 23,000 users of Hudson, who van Zyl emphasized cared a lot of about the stability and performance of Hudson.
van Zyl wanted to paint this is as 214 users imposing their will on a 23,000+ Hudson community which, at less than one percent, would indeed be a bit absurd. But I'll admit, I pushed back on his number a bit. Sure, there's an estimated 23,150 or so installs of Hudson--the data is from Kohsuke Kawaguchi, creator of Hudson and one of the leading proponents of the Jenkins move, himself. But are those installs all representing active contributors to Hudson? Who was actually eligible in this vote?
van Zyl replied that there are about 1,300 members of the Hudson mailing list, whose members held the political franchise to vote on this topic. So, in that context, the 214 votes for Jenkins represented about 17 percent of the total Hudson community, still a small minority. Representing it as something larger, van Zyl said, "was a bit disingenuous."
The Sonatype founder also dismissed the importance of the one most important issue for the Jenkins team, which led to the Jenkins team giving up on continuing to try to talk to Oracle about any sort of reconciliation: Oracle's alleged refusal to give up the Hudson trademark to the Hudson community. van Zyl stressed that this contention was overblown by the Jenkins team, highlighting that his company was able to secure the rights to the Hudson trademark in two days.
Was there money exchanged for that trademark, I asked? No, van Zyl explained, Sonatype was able to get that trademark free of charge, and only had to agree to one important stipulation: to use the Hudson trademark, project members would have to use only the core binaries from the Hudson tree. And change to the core Hudson binaries, Oracle told Sonatype, would mean that the software would no longer be eligible to be called "Hudson."
For the Jenkins team to claim that they could not secure the rights to the Hudson trademark when they indeed could, may mean that there is something larger going on here, van Zyl explained. While he did not come right out and say it, he seemed to imply that the Jenkins community might not be able to live with that requirement of not touching the core binaries.
Sonatype, he added, was trying to get back to using standards-based offerings in its Java developer product line.
"CloudBees [a Jenkins supporter and Kawaguchi's current employer] may not want to go in that direction," van Zyl said.
This was the crux of van Zyl's concerns about Jenkins. Customers using the Hudson plug-ins might not be able to maintain full compatibility with Jenkins should they decide to move forward to the newer software project. This is why, van Zyl explained, he and his colleagues at Sonatype had decided to throw their lot in with Hudson.
Under Hudson, van Zyl emphasized, there would a release every six weeks and the Hudson team will be working on whittling down a "huge backlog of bugs." This sort of continuity is vital to the enterprise customers who use Hudson, he added.
Naturally, after this call, I wanted to clarify some things from the Jenkins side of the fence. What about the only 214 pro-Jenkins votes, I asked Kawaguchi? He replied via e-mail on Saturday:
"As for the voter turn out, it is normal in mailing lists to have a substantial number of inactive members--those who are subscribed to the lists and filter e-mails to a dedicated folder, so that they can still post a question whenever they need to. But of those who have voted, 214 was in favor to rename and 14 was against it," Kawaguchi wrote. "In addition, all the reactions [on] the web was very positive, and the activity in the mailing lists after the split has a startling difference, with Jenkins lists seeing 400+ e-mails while Hudson lists mostly seeing just 'unsubscribe' e-mails. So I don't see anything wrong in drawing the conclusion that the community supported the rename."
Kawaguchi's statement to me on the trademark issue was more detailed: "With respect to the trademark, our primary issue is the right as the developers of the project, not as the user of the project. As you probably know, in OSS projects like Hudson, the meritocracy model is expected--where people doing the actual work gets to influence the decisions more.
"But this doesn't work well when a minor contributor gets to decide what gets ultimately called as Hudson, as it effectively works as a veto power--imagine a situation where the developers have near unanimous consent to do X, and Oracle wants the opposite. We effectively cannot do X because they can refuse to declare the end result Hudson.
"The incident in November made us realize that this was not just a hypothetical possibility but it is happening--in that event, X was 'move code to GitHub.' So the community wanted to make sure that it'll not happen again, and our suggestion was to assign the trademark to a neutral foundation that won't weigh in on day-to-day technical matters. In our eyes, that would restore the meritocracy model and create a level playing field.
"With respect to the fundamental changes, we've always worked hard to ensure backward compatibility with the earlier versions. Our somewhat unusual version numbers like '1.396' is a sign that there was no major API breakage at any point in the project history. We've been making all the changes in an incremental way, and I personally don't see any reasons to change that fine record.
"If anything, I'd say it's the Oracle Hudson project that appears to be attempting more fundamental changes to the core binaries quite unilaterally," Kawaguchi concluded, referencing the same Sonatype blog statement from van Zyl linked earlier in this article. "[W]hereas the Jenkins project is working in a more transparent manner, as can be seen in our governance meeting that we just held in public today. This might be a naive view, but I can't help but think that there's a bit of cultural difference between the top-down approach Oracle operates in this project and the bottom up approach the community has long operated under. That might be a part of the reason we couldn't reconcile the trademark issue."
It was very interesting that Kawaguchi highlighted van Zyl and Sonatype also as parties about to make changes to the core Hudson binaries--I had not made him aware that it was Sonatype who had raised the exact same concerns about his own employer CloudBees.
For their part, Sonatype is adamant that backwards compatibility is a high priority.
"Sonatype intends to make all our changes backward compatible. We have established this precedent with Maven 3.x where we created a path forward for innovation while remaining backward compatible with Maven 2.x. We would never change the API for Hudson users. We've been very public about this strategy, as it is almost exactly the same strategy we executed with Maven 3.x," van Zyl told me.
One thing is for sure: the Hudson/Jenkins break is not going to be clean. Both sides harbor some clear professional resentment towards the other over how things at Hudson used to be handled. More importantly, it's very apparent to me that each camp wants to position itself as the "official" Hudson, with Sonatype and van Zyl loudly picking sides to go with Hudson. Each side is very wary of the other's motives, as well: with Jenkins and Hudson suspected of breaking compatibility soon by the other team.
It's a stark reminder that ending relationships can be a messy thing, even professional ones.