December 15, 2009, 2:20 PM — The previous Smart Development aimed to make the point that comprehensions are practical, and chose Python as an example language. A commenter appeared to believe the focus was on Python, and correctly observed that comprehensions aren't new for Python.
He's right, of course, as a follow-up comment details. For more on what is new in Python, or was, see Barry Warsaw's wrap-up of features in 2.5, 2.6, and 3.0, and Richard Jone's perspective on 2.4, 2.5, and 2.6.
[ See also: Python and the power of 'first class' everything ]
Gabriel Genellina explains a subtlety in Python's coordination of list comprehensions and generator expressions: the two are almost, but not quite, isomorphs, which share their syntactic kernel, and nearly all of their values. To me, this nicely illustrates the difficulty of managing a big project: to include generator expressions in Python sure seems like a good idea--they're powerful, and neat, and even fit in well with other parts of the language--but it also introduces complexity. Complexity is a heavy weight. Complexity adds to the burdens of maintenance, documentation, testing, ...; perhaps worst, it consumes the resource in shortest supply in most projects: the attention of those qualified to make and explain design decisions.
Sage and Python
Python sure makes a lot of right decisions, though. Sage is an example of how good Python can be. "Sage is a free open-source mathematics software system ..."; here, "mathematics software" means that Sage can do not just arithmetic (a service to accountants, say), or give numerical answers to linear algebra problems which are useful to engineers or financiers, but actually solve calculus equations and far, far more abstract problems that are meaningful to working mathematicians. Sage competes, in several ways, with proprietary Mathematica, which is familiar to more readers.
Python performs at least a couple of functions for Sage, gluing together a wide, wide assortment of pieces coded in C, C++, and other languages. Sage founder William Stein issued a memoir a few days ago on, among other things, his path from closed- to open-source in his mathematical explorations.
One of the interesting aspects of Dr. Stein's automathography for me is its naïveté in regard to software issues. I don't mean at all to be insulting in that; I simply find it instructive, and rather frustrating, that someone of such monumental accomplishment (Sage is very large, and very important) still communicates through a blogspot filled with distracting <br />-s, and took so long to learn the essential of what he needed from Python. I haven't asked him yet whether he's aware that "... free open-source ..." is a battle-cry in some corners of computing.
All that is a reflection on software and its complexity, not on Dr. Stein. He began to use Python in 2003. As he recounts, it was mid-2004 before he settled on the Python-Pyrex combination that is fundamental for Sage, and 2005 before he deployed one copy of Sage to its first user! This represents a personal frustration to me, because, during the span on which his narrative focuses, 1999-2005, I helped multiple engineering groups through much of the analysis he describes he painstakingly repeated for Sage. I was even in touch with mathematicians only a couple of acquaintances distant from him, before he began Sage.
Sage isn't great because Python's a perfect language, or its developers have magical powers. Python's a good-enough language--but it also was a good-enough language fifteen years ago, and Lua or Tcl might have worked out even better for an alternate-history Sage. Python has been good-enough for Sage, though, Python has a healthy ecosystem around it that energizes enthusiastic youngsters, and Dr. Stein made a choice, enlisted a critical mass of colleagues, got to work, and kept working, until his success was apparent. Smart Development focuses on programming techniques, and especially languages; I do so, though, knowing full well that these make only incremental differences in meaningful projects--maybe 10%--where most of success has to do with focus, hard work, good teamwork, and clear thinking.
