July 08, 2014, 6:00 AM —
Lots of non-programmers seem to be learning how to program these days. However, one group of non-programmers who have been programming for a long time are economists. They often have to write code in order to run and evaluate their complex mathematical models and simulations. If you’re an economist, then, faced with having to do some coding, which programming language is best for your use?
Two economists, S. Boragan Aruoba of the University of Maryland and Jesus Fernandez-Villaverde of the University of Pennsylvania, recently set out to answer that question. The results are contained in a new paper published by the National Bureau of Economic Research titled A Comparison of Programming Languages in Economics (Fernandez-Villaverde has the made the full paper available for download). Their basic approach was to compare the time it took a handful of programming languages commonly used by economists to solve a well-known economic model.
Specifically, the model they chose to solve is the stochastic neoclassical growth model, considered a “workhorse” in the field, using an algorithm which is representative of the computing economists need to do. They coded the problem in three compiled languages (C++, Fortran, Java) using a variety of compilers and five scripting languages (Julia, Python, Matlab, Mathematica, R) and ran the programs on two platforms, Windows and Mac. The languages were then evaluated based on their code execution time.
Their paper lays out the full set of elapsed time results for all the language/compiler/platform combinations. Here are the top-level takeaways.
C++ is the fastest programming language for economists
While Fortran is a very popular choice for economists, it was the C++ that ran the fastest on both Mac and Windows. The elapsed time for C++ compiled with GCC was 0.73 seconds on the Mac, while it was 0.76 seconds for C++ compiled with Visual C++ on Windows. Fortran was a close second on both platforms; execution time on the Mac for Fortran compiled with GCC was 0.76 seconds, while on Windows, it was 0.81 seconds for Intel Fortran. The only other language to come close was Java, which had executions times of 1.95 seconds on Mac and 1.59 seconds on Windows, still twice as slow as C++ and Fortran.
Compiler choice matters
While C++ and Fortran dominated the other languages in performance time, the choice of compiler mattered significantly within those languages. Generally GCC compiled code ran significantly faster than Intel compiled code on the Mac, while the opposite was true on Windows, though the spread was greater on the latter. Intel-compiled code ran about twice as fast as GCC-compiled code on Windows for both C++ (0.90 to 1.73 seconds) and Fortran (0.81 to 1.73).
Scripting languages were considerably slower than compiled ones
Not surprisingly, scripting languages fared worse than compiled languages, but sometimes a whole lot worse. Julia did the best, coming in close to Java, about twice as slow as C++. Matlab was about 10-times slower than C++ on both Mac and Windows, while Python code ranged from being about 40-times slower than C++ on both Mac and Windows (for Pypy) to well over 200-times slower (CPython) on the Mac. R ranged from 500 to 700-times slower, while Mathematica did the worst, being 800-times slower than C++ on the Mac. The authors did find, however, that the performance of scripting languages could be greatly increased by compiling parts of the code when possible. Matlab speed goes to just twice as slow as C++ when compiling into Mex (C++) files, with a similar increase found for Python with Numba.
One of the important caveats of the study is that the researchers made no attempt to use special features of each language to optimize the code, instead opting for similar implementations across the languages as a way to control for programmer skill and because it would make comparison difficult.
What does this research mean for the general population of software developers? Probably not much, since professional programmers with deep knowledge of these languages could optimize the code and, presumably, come up with very different results. In fact, the authors have made their code available on GitHub, so if you’d like to see how the language perform when optimized, have at it.
How relevant are these findings, though, for economists? Even that’s questionable, since the choice of programming language will depend on things other than code execution time, like learning curve. Plus, different languages may be better for solving different kinds of problems. Finally, execution time may ultimately not matter much to economists anyways, so long as they can get a reliable solution using a language they know. But, if you’re a young economist trying to decide what language to invest some time in learning, these results suggest that C++ is the way to go.
In any case, I still find it interesting that a non-programming profession which does quite a bit of coding is making a serious effort to consider which language is best. The authors plan to consider the performance of functional programming languages in a forthcoming paper. I’ll report back on that once the paper is available.
Read more of Phil Johnson's #Tech blog and follow the latest IT news at ITworld. Follow Phil on Twitter at @itwphiljohnson. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.