Hi I'm Bruce Taylor and this is Voices on ITworld. Our topic today is extreme programming (XP) and what it means as a loose subset of the agile software development in project management movement. Our special guests today are Kent Beck and Cynthia Andres, co-authors of "Extreme Programming Explained," now in its recently released 2nd edition from Addison-Wesley Professional. I'm also delighted to have joining us in this conversation Tom DeMarco. Tom is a well-known author, consultant and lecturer on software engineering and management practices. Welcome to you all and thank you for joining us.
All: Thank you.
Bruce Taylor: Kent, first to you. If you had to choose one key learning that has surprised you between the publication of the 1st edition and what you know today, what would that be?
Kent Beck: It's not all about programming. It's not all about programmers. Programmers aren't somehow special and to be protected and coddled. I used to say often that programmers were children. They liked not to be yelled at and to have more toys, and I think that was kind of my attitude for quite a while and I saw myself as one of those people. And that's not the way I view the world now. I think programmers are, or at least can be, adults and can and should, for the good of development and themselves, act that way.
Bruce: Tom DeMarco, you've read both books. What surprised you?
Tom DeMarco: I believe that the most surprising thing happened out of the context of the two books and the second book really reflects upon that. And that is that during the period 2000 to 2005, the whole notion of XP and agile approaches in general have come of age. When I first saw the submitted manuscript for XP, it was truly extreme. It was something that shook me because I looked at it and saw that it seemed to be a contribution from left field, a contribution from -- it's like the Citroen car seems to have been designed by people who had never ridden in any other car. They started everything from scratch and came up with wonderful concepts, but just not the same concepts as others. That's what XP looked like in 1999 to me when I saw the manuscript as it was submitted to Addison-Wesley. In the subsequent five years, XP has become mainstream, that while I wouldn't say that everybody does it, everybody is affected by it. Everybody's practices have changed because of the work that Kent did, and not just Kent, but other members of the XP community and other contributors to the agile movement -- Alistair Cockburn and Scrum, Jim Highsmith and so forth. So that's a big change.
Kent: Tom, I'm curious when you say mainstream, I think of I should be able to go anywhere and see people doing these things and so compilers are mainstream and Java is mainstream, but maybe that's my perspective versus yours, but it doesn't seem from that point of view that XP is at all mainstream�
Tom: Well, maybe not quite, Kent, but one of the things that I tell people when the topic comes up is that whether XP is right for you or not, you need to know about it. And what has become mainstream is people knowing about it. That everybody knows what pair-programming is. Everybody knows what test before code is about. That didn't used to be true. People know the term XP. People are aware that there are some social dynamics in XP that set it apart from the world of methodologies as we previously saw -- things like the 40-hour week and energized teams and sitting together and story boards. So the parlance I think has become mainstream. Whether the practice has become mainstream, I think it depends on where you go. And a lot of places I go, it is. But I don't doubt there's plenty of the world where that is not so.
Bruce: One of the things that struck me about the book, and Tom, you just said it in terms of social construct, but that it's a very human values approach to a discipline, to a technique, a set of techniques. I highlighted some of the language in the book and originally I actually did attribute this in my own head to Cynthia's work and she has demurred from that, but I'm going to ask her to comment on it. Some of the things that I highlighted were words like transparency, responsibility, vulnerability, open and honest communications, keeping commitments, trust, mentality of sufficiency, playing full out -- I mean the list goes on and on and I still think underneath that Cynthia, a lot of your thinking had to be inside of that. Do I have that completely wrong?
Cynthia Andres: No, I would say that that is true, that Kent and I talked about those ideas and issues and developed them together. I probably brought up some of the concepts and that we worked together on which terms would most clearly convey what we were trying to say. I think that the first book was in reaction to a culture for programmers that was basically camping in their cubicles in Silicon Valley where people were spending their entire lives living out of cubicles with food brought in and laundry taken out. And it was a reaction to that to set up what's good for programmers? How can we work in a way that's good for us? And I think the shift in the new book is what's good for software development and everybody who's involved in the process. And talking about things like transparency and trust and accountability those are all ways of improving the process for everybody involved.
Bruce: Tom, last evening you made a comment about Kent's belief in the 40-hour work cycle and you mentioned it again just a minute ago. I know that both you and Kent talk about slack inside this book, [in fact] you have written a book entitled Slack - [and] one of the things that I got was this notion that a longer work cycle than the 40-hour week or a longer work cycle than what is a normal workday, that not only is it quite likely that we are losing productivity, but that we're actually subtracting value from what's already been accomplished. And so this is to both you and Kent, do I have that right? Is that being suggested here?
Tom: Yes, I think you do have it right and I think it's true in general. I think companies are kidding themselves when they work a 50-hour week. That they just get less done than they would if they only worked 40 hours. And I don't mean less per hour, I mean less total. But I think it's particularly true in the context of XP. I had the experience of taking part in the intensive workshop on XP where I actually pair-programmed with another participant and went through all of the steps of XP. And I have to tell you, it was exhausting. That there's something so intensive about pair-programming, so intensive about this one. I don't mean just learning it, but I mean applying it. So though I think that that point is true of knowledge work in general, that overtime is a mistake, it's particularly true of XP because it is intellectually commanding. You just don't have more than 40 hours in you.
Kent: I would agree that there's a sense that if you do it well, if you're really focused and intense, then at the end of a good solid work day, the most value you can bring to your team is to get some rest. Exhausting sounds like it's just, it sucks the life out of you. And that's not how I feel. I feel a sense of satisfaction. I can point to concrete things that I've done that are different now than they were at the beginning of the day, a sense sometimes of exhilaration because I know that I've set myself up to do the same thing again tomorrow. And so it's not like running a marathon where you collapse at the finish line. It's more like okay, I'm done with this for today and I would love to do this again tomorrow. So exhaustion doesn't quite capture that.
Cynthia: Pair-programming is very socially intensive for people who are used to spending a lot of their time alone, that's a big shift. On the other hand, there are a lot of moms in the world who spend a lot of their days in that kind of an intensive social interaction and survive. So it's a matter of changing styles and getting used to a different style of work.
Tom: And of course, what I was doing as I was pair-programming was also learning, so there was a double taxation there. Still, I believe that it is more intense both intellectually and socially than anything that a non-XP organization, non-agile organization has gone through before in the name of programming and that therefore, the 40-hour week becomes more germane in the context of XP than it would if people were just proceeding in the 1960s ways.
Kent: Well, I think it's important to be aware of your energy levels, which is why the second edition gets away from a particular number of hours. That number of hours is actually -- there are people whose hackles go up when you say 40-hour work week. So in the 2nd edition it's called energized work, the same practice and it says, be aware of your own energy levels and absolutely, as a programmer's energy level drops below a certain level, you start subtracting value from your program. You don't do the design you should do. You don't write the tests you should write. You don't automate the tools you should automate. And you make more mistakes. There's lots of ways you can take value out when your energy level gets down. That could be at the end of eight really good solid hours of programming or six good solid hours of programming or you could be sick and need to go home or there's lots of reasons why that energy level can drop. Or you might just need a vacation.
Bruce: Well, there's a twist on responsibility you just said, that someone being truly responsible for their own energy level. Really a very interesting notion.
Kent: My early jobs didn't happen in an environment like that. Other people tried to be responsible for my energy level. They tried to say you ought to be here on the weekends. If you were committed, you would be here on the weekends. There was either subtle or sometimes really blatant pressure to do that and how could they know whether I'm ready to put in another good day or what I really need is rest?
Bruce: And that calls for a very different cast on project management and I can tell you that as a former manager of people, that's exactly the kind of thing that I used to do. I would say you are not committed if you're not willing to come in on Saturday and finish this work. It strikes me that line supervisors and team and project managers have a much different responsibility in XP.
Cynthia: I think XP is a change of context where you're not stuck in an organization, you're not stuck in the relationship with your manager, where you take responsibility for yourself and your work and your relationships as far as you can from your side and if that's true, then you choose to do things like write tests, communicate with the people around you that make that all run smoother.
Tom: I think that's also, Cindy, part of the backlash that some people have about XP is because it becomes self-coordinating. Responsibility is pushed down in the hierarchy and I think managers in particular are saying, well what the hell am I here for if not to tell a given person when he ought to be testing and when he ought to be coding and when it's time for a build.
There's a world of worry between those two extremes that it's hard for many managers to yield that much control of the coordination. I've become convinced over the last few years that managers say they are coordinating so strongly -- you're micromanaging in some sense -- coordinating so strongly because people left to their own devices won't coordinate. And I've now become convinced that quite the opposite is true, that they leap in and grab coordination because they're worried that people will self-coordinate if they don't do it. And that therefore they won't have that as part of their role, which feels like a managerial role. And I think XP is a very natural kind of self-coordination where that work, that coordinating work is pushed down the hierarchy.
Kent: Well, some of it is. I think of managers as people with more experience, a broader perspective, perhaps more wisdom about what's going on and good people skills. And looking at the connections between if a manager has nothing better to do than to come and tell me I haven't written enough tests that day -- to me, that's not a lot of value to be added there. That's not about me and my relationships to other people or the team and its relationships to its customers and suppliers. I mean to me that's where the value of having somebody with a step back with broader perspective is.
Tom: I think the key phrase here is, if the manager has nothing better to do. I think if you replace that with, if the manager has nothing easier to do, then that would lead you a slightly different direction. Because coordination is the relatively easy thing, what Cindy was talking about is managers could manage, that's hard stuff, that means involves motivation, it involves interpersonal relationships. So I think that is the distinction between real managers and imposters who find themselves pushed into that position and end up looking for the easy stuff to do. And the easy stuff is what people will do on their own if you don't do it.
Bruce: So Kent, let me ask you, when I read this book, I could envision an organization as a matter of strategic development, taking on XP and saying it's going to take us a year to get up to speed, that the training and development time, the teambuilding time, the getting it right time is going to take a year before we see the real first effects of it.
Kent: I think that's both too short and too long. I think you'll see potentially real effects in the first couple of months. I have clients who've seen dramatic drops in defect rates in the first two months of development. I've seen dramatic increases in productivity in the first two months of development. They're very proud of that. I also have clients that I've worked with for five, six, seven years and they're still improving dramatically. They'll say, okay, now we've got it. And I go back a year later and they say, okay, now we've really got it, we're going even faster than that. So I think there's a sense in which, if you're not seeing payoff from your efforts in XP in a matter of months, then it's time to take a step back and look at that. At the same time, where you get to in a year, you may think that you're rocking and rolling at that point, but there's a whole lot of potential improvement beyond that.
Cynthia: Extreme programming also suggests that you're delivering value every week all the way along the way, so it's not a year of setup time. You don't stop doing what you're doing to spend a year learning how to do XP.
Bruce: So if I go to other management practices, as an example, I know with some certainty that changing behaviors, giving people with self-awareness the opportunity to change a behavior, modify a behavior is a six-month deal, sort of at the inside, what happens inside XP that alters that?
Cynthia: It depends on intensity of motivation, on what the culture is in the first place, on the team that's working and how well they work together. If you have a team that's not working well together and all of a sudden they're communicating, the change could be dramatic.
Tom: I also think you need to take account of the natural work modes that people have, which in many ways, I mean what they would develop if left to their own devices, which many ways lie closer to XP than to a more formal methodological approach. They're focused -- they would be focused as XP focuses them on building useful stuff as opposed to constant or long periods of worrying about how we're going to build it before we ever do a thing. So I don't think it's an unnatural change. I think it's a very natural change.
Bruce: And the weekly cycle supports that, I would think. The faster cycle.
Kent: Yeah, because you have weekly accountability. You don't have time to -- you can't spend two months working on an architecture. Well you can, but it will just take you four months to do so because every other day you're working on functionality and then you work on the architecture some more and the functionality some more. Because you know every week you have commitments to your customers about what features they're going to see at the end of the week.
Bruce: So Kent, what are some other examples of how customers can be brought into the planning and to get a maximum amount of customer feedback within XP?
Kent: Sitting together with a customer, having a relationship with them that is an ordinary professional relationship with them just as, say I'm a programmer on the team and I have a relationship with this person who knows about the domain and this person who has really used the software for the last five years and this person who's a programmer. In that environment, it doesn't even feel like feedback, it just feels like you're working on something together. So to me that's the most powerful way that you can connect with your customers.
Bruce: In your view of a whole XP team, does a whole XP team include the customer?
Kent: Yes. They're part of the way you create value with software development and for decades we've held them at arms length through formality and documentation and the clinging desperately to a lack of social skills and there's an opportunity to create a lot more value.
Cynthia: It's easier to produce what the customer wants if you ask them rather than guess.
Tom: We have been examining ways in which XP has -- my phrase -- come of age between 2000 and 2005, reflected in some changes in the book. Two of them we haven't discussed yet that are supported by the variation between the two editions, but also that I know of from talking to you, Kent, is first of all, your sense of how big a project XP could be applied to, I think, has become freer, you seem much more optimistic about applying it to projects that are bigger than you would have considered okay five years ago. And the second is I felt you had a kind of a religious sense at the beginning about what you would have to do in order to be able to assert your doing XP, say if you don't do this, this, this and this and this, and you can call it what you want, but it's not XP. I think if I've got you right, I think you're now looking at the different practices of XP as being useful and justifiable in and of themselves and you don't seem to feel so religious about the whole and the uniformity of application of the ideas.
Kent: I think to the degree that religious or dogmatic sense of you have to be doing all these things or you don't get your gold star is really my insecurities coming through. What if you take half of my advice and then you fail? Well, what does that say about me? Well, it probably doesn't say very much about me at all, but I wasn't clear on that. And I think I am now. I think the practices in XP aren't nearly so central as the values and the principles. I think that if people really keep their minds on respecting everyone who is touched by software development, they keep their minds on how are we going to find mutually beneficial solutions to all of our problems that the practices will fall out. So to me, there's a whole chapter in the new edition that says the question, is this an extreme programming project or not, I think is a nonsensical question.
Tom: Which I think is a different attitude from what you had five years ago.
Cynthia: I think Kent's perspective has broadened from the binary dogmatic programmer of you're doing it right or you're doing it wrong, to your process is your process and how you improve it -- I mean there's lot of ways to improve. There's at the level of being clear about your values and your principles, which guide your behavior and then there's which individual practices do you choose to use. And the practices that you choose, hopefully, are the ones that are going to improve your process the most. The more practices you add together, the more combined effects you get.
Tom: I also think that people that leave the rest behind for a moment are going to find themselves drawn to it later. Because if the values are there, the practices, they should converge on at least some of the practices.
Cynthia: Yes and if you're doing the practices that work, you back to your principles, well, this is great. Based on my principles, what am I going to do next and you'll keep adding more and more of the practices.
Kent: Now regarding your question about scaling up XP, this is a topic that I've got to say I'm more of a follower than a leader on. I'm reporting what people have done with XP, not speculating about what might be possible. People built organizations of a hundred all working in an XP style and -- kind of to my surprise, and then I looked at what they did and it all makes good sense. It's not exactly the same as what you do if you have 10 people in one room, but the values are all applicable and the principles are all applicable. The practices at the scale of the individual teams inside that organization can be very, very similar, or you sometimes have cases where they're quite different. So one organization that I talked with a lot is Saber and they have C++ legacy code with no tests and they do a quite different process with that than they do with their Java code that's been developed test-first from the beginning, because the defect rates are so very different. So their practices are different and that makes perfect sense to me.
Bruce: Kent, I'm going to give you the last word. This has been fun, thank you all. Thank you Kent. Thank you Cynthia. Thank you Tom. And Kent, besides telling everybody to run out and buy the book, what would you most like software developers and project managers to know about extreme programming?
Kent: What I'd like them to know is that there is the possibility of improvement out there, that they're not stuck doing things the way they've been doing them. But that the flip side of that is if you're not stuck and you stay in your place, then you're responsible for that. That it's possible though to have a work life with integrity where you act in ways that are in accord with your values every day that software development gets better when you do that. That's not why you would do it, but one of the consequences of that, of being clear about your values and matching your behavior to that is that there's less stress in your life, that you're more creative in your work, that you're more able to pay attention to other people, that your learning accelerates, that your defect rates go down, that your ability to find insight into problems and to tackle them in satisfying ways all goes up as a consequence of aligning what you do with what you really believe.
Bruce: That's a reason to buy the book.
Kent: I think so.
Bruce: Thank you all.
RELATED RESOURCES
Buy
Extreme Programming Explained: Embrace Change (2nd Edition)
Listen to
a book excerpt recorded by Bruce Taylor
