'The Code Has Already Been Written' 253
theodp writes "John D. Cook points out there's a major divide between the way scientists and programmers view the software they write. Scientists see their software as a kind of exoskeleton, an extension of themselves. Programmers, on the other hand, see their software as something they will hand over to someone else, more like building a robot. To a scientist, the software soup's done when they get what they want out of it, while professional programmers give more thought to reproducibility, maintainability, and correctness. So what happens when the twain meet? 'The real tension,' says Cook, 'comes when a piece of research software is suddenly expected to be ready for production. The scientist will say 'the code has already been written' and can't imagine it would take much work, if any, to prepare the software for its new responsibilities. They don't understand how hard it is for an engineer to turn an exoskeleton into a self-sufficient robot.'"
I expected more (Score:4, Insightful)
The abstract in Slashdot is pretty much the whole text in the linked post. The other 3 paragraphs repeat the same idea.
Re:I expected more (Score:5, Insightful)
One-off vs. Product (Score:5, Insightful)
Productization of software is a completely different exercise. You have to make your software work for a larger crowd on a plethora of devices. You actually have to consider how your software fits into the larger product lifecycle. The key difference here is that you have customers that you need to keep happy.
This is so true (Score:5, Insightful)
You can often tell whether someone is "programming as a means to an end (of your own)" versus "programming to build a tool for someone else". For instance, I have experience in the financial industry. Quite a lot of traders see coding as a means to implement their cool new model. Looking at their code, you can often tell. It's as if everything was built to just exactly fulfil the requirement, with no thought to the fact that those requirements might change. But of course, they do change. So you get hacks and workarounds, and cut'n'paste cargo cult code. Kinda like what those Orks in Warhammer 40K might make. And of course the problem with spaghetti code is that if you write it, nobody can ever help you solve problems/improve it. It's the coding equivalent of painting yourself into a corner. There's loads of smart traders out there with an excel spreadsheet that actually is an extension of their personalities (In fact it's their Magnum Opus. Everywhere they go, they try to take this quirky little file with them). Every little hack is something only they can explain (comments, yeah right. Do your body parts have explanatory comments?) and only they can fix if wrong.
On the other hand, you sometimes hire a guy who is a programmer, but knows nothing about the domain. Very good with OO models and that kind, but you have to teach them everything about finance. What's a settlement date, what kinds of options exist, etc. You get what you ask for, because they know how to turn problems into object models, but you have to ask VERY carefully. And teach. Unfortunately, not everyone has time for that, and so you end up with something that still doesn't quite do what it's supposed to.
So you often end up gettings guys who understand the problems, but can't program, programming. And guys who can program, writing the wrong program.
Re:I expected more (Score:5, Insightful)
Re:I expected more (Score:5, Insightful)
The whole premise is stupid anyway. I've worked with plenty of scientists in national labs that turn out production grade, maintainable code; and programmers who didn't. The core issue is getting people who write code for reuse by others to follow guidelines, regardless of title or profession.
Because you can point to a few (very few) exceptions does not make the story untrue in the vast majority of cases.
Scientist code is usually a giant JUST-SO story, sufficient to derive the results they need for the task at hand.
They either don't have, or avoid putting in data that will crash the program so limit checking is not necessary.
Crashes are fine if they do nothing more than leave a trail of breadcrumbs sufficient to find the offending line of code.
Output need not be in final form, and any number of repetitive hand manipulations of either the input or the output are fine as long as the researcher does not need to spend more time writing any more elaborate code.
This is perfectly fine. The cabinet maker makes jigs. They are designed for their own shop and no one else has exactly the same saw and exactly the same gluing clamps. When the cabinet maker sells his shop, these jigs become useless. Nobody else knows how to use them.
The scientist who takes the time to do a full fledged, fully documented, maintainable, fail-soft package for analysis of data that is unique to their project and their apparatus is probably not doing very much science, and probably not doing their intended job. That budgets force them into this situation is not unusual.
It happens every day in industry, academics, and research. To hand waive it away by saying you know someone who delivers the full package merely calls into question your own understanding of the meaning of a complete, fully documented, maintainable, transferable, and robust software package.
This is a real problem (Score:5, Insightful)
The issues surrounding transitioning research S/W written by scientists into honest-to-goodness production systems are ones I'm very familiar with.
At my company, a lot of energy has been put into bridging the gap over the years with varying results. I believe that the root cause of the problem is that research S/W is not an end-product; typically for scientists the end-product is a research paper, white paper, proposal paper, etc., for which the S/W is only a tool for getting to the end-product. As soon as the experimental (or proof-of-concept) S/W returns the desired results, the software is considered "done".
In contrast, production S/W is often THE end-product for developers, so a lot more attention is given to robustness, re-usability, etc. All the standard thinking that you want to go into your production S/W.
One big issue for us is that the research S/W is almost always written in Matlab, while the production code is written in C++ and Java. The single largest source of bugs in our systems is porting S/W from Matlab to C++ or Java. (As an aside, please let's not talk about the Matlab 'compiler', nor Octave. -- we've already tried them both, and they're both performance hogs and also create SCM and CM nightmares).
We experimented with requiring that the research S/W be written in C++, but it was a disaster. The scientists couldn't get anything done, and the code was just awful. So, back to Matlab it was.
And, my experience is that people who I have a great deal of respect for, who I consider brilliant in their fields, holding PhD's, etc., have produced the crappiest Matlab code I've ever had the sorrow to read. My favorite instance was the use of these local variable names within a single function of research S/W that was considered "done" (true story):
i
ii
iii
iiii
iiiii
iiiiii
And, of course, little documentation as to the mechanics of the code. And believe me, it gets worse from there. Bear in mind that the code does indeed work for its particular purpose, and may well be ground-breaking in that particular research domain. But "done"? Ready for production? Not without a major porting effort (which is really a re-writing effort). The most mysterious thing to me, though, is that the scientists, for all their intellectual firepower, don't understand that it's a problem.
The solution we've converged on is to require our bizdev to be responsible for funding efforts to rewrite the research code and get it integrated into the product baseline. And, the bizdev types can't proclaim a particular capability "done" (eg., sell it to customers) until they've funded and executed those efforts. It took years of education to get to this point, but things are moving along much better then before.
Re:This is so true (Score:4, Insightful)
"It definitely is functional but hardly has any of the features consumers demand."
isn't this expected (or at least not surprising). if I were a NASA engineer, I'd see the program as a tool to help me accomplish the larger task. the more time I spend on tools, the less time I spend on progress to the larger objective. I'd write the program as quickly as I could.. would not care about UI, functionality, usability or anything else, I built it for me to use - as long as the output satisfies my needs, i'd consider the task done and move on.
Re:One-off vs. Product (Score:5, Insightful)
There's a nice concept devised by Ward Cunningham which captures this issue nicely: "Technical debt".
Failing to put in the effort that makes code maintainable during its construction incurs a notional "debt" which the software carries with it. Future developers working on the code "pay interest" on this debt in the form of time wasted on understanding and modifying the crappy, undocumented code, or on fixing bugs that wouldn't have been present if the code were better. Sometimes, those future developers may decide to spend time refactoring, building tests or documenting, and those cleanups pay down the "principal" on the "debt". After their cleanup work is done, future work has smaller interest payments (less effort for the same results).
Startups often deliberately decide to incur great amounts of technical debt on the theory that if the revenue starts flowing in they'll have the money to fund paying it down, but if they don't start getting some money the whole company will evaporate.
For scientific research, it's pretty clear that it also makes sense to incur lots of technical debt in most cases, because there's little expectation that the code will be used at all once the research is complete. Even when that's not the case, I think few scientists really know how to create maintainable software, because it normally is the case. I don't see a lot of scientists spending time reading about code craftsmanship, or test-driven design, or patterns and anti-patterns, or... lots of things that at least a sizable minority of full-time software engineers care a lot about.
I guess the bottom line, to me, is that this article is blindingly obvious, and exactly what I'd expect to see, based on rational analyses of the degree of technical debt it makes sense for different organizations to incur.
Re:I concur (Score:2, Insightful)
leverage debt (Score:5, Insightful)
That's a great post, of the kind that saves me a lot of typing. You covered the first-order considerations brilliantly.
What you missed was technical debt blindness, which has been around since forever. Books I read around the time of the Mythical Man Month talked a lot about maintenance syndrome: that the original development team would be regarded as brilliant for producing working functionality at tremendous speed (undocumented, with no error handling for edge cases), then the first maintenance team would all be fired as underachievers for adding hardly any new functionality in the first year or two.
Turns out it's hard to erect a machine shop over top of adobe mud brick construction without adding some reinforcement to the structure, which usually takes a lot longer than the entire original edifice.
You can instead take a wrecking ball to the first iteration, but this rarely works out as well as hoped. You end up with far more ambitious adobe mud construction built with a whole new generation of unproven tools. At some point you have to bite the bullet and ferment what you began with.
People hide debt blindness behind widely divergent construals of simplicity, where "simple" usually turns out to be a euphemism for any decision that sidesteps paying down debt in the short term.
For professional software engineers, there is one true simplicity to rule them all: generativity and compositionality. Can you build the next layer on top with any hope of having it work and able to support an ongoing stack? For us, it's a long term game of pass the baton. For everyone else (management, scientists) the endgame is to cash out, and take credit elsewhere (e.g. publication biography).
Unfortunately, a citation is not a formal linkage that the compiler either accepts or rejects. By the standards of compositionality, citation is payment in dubious coin. Citation is not falsifiable. Scientists still count their citations even when they come from papers that are full of crap, peer review notwithstanding. For a professional software engineer, when you start instantiating objects from one library inside an abstract expression template library, you come face to face with compositionality in a way that few scientists can even imagine, having weened at the outrage of being improperly cited.
Technical debt blindness on the part of management quickly turns a software engineering shop into a highly non-linear fiasco. We've all seen this.
Somehow this game works out better (for the participants) when played by bankers with leverage debt. But now it's my turn to pass the baton, since that deserves a whole lot more typing and I've done my bit.
"It works for me" syndrome (Score:3, Insightful)
I work for a group at NASA. One of our group's tasks is to take scientist-written code and wrap it for distribution to hundreds of remote sites around the world. We try our damnedest to run the code as-is, but fairly often have to modify it to remove stuff like:
* Hard-coded input and output file and directory names
* Small and arbitrary length limitations on file pathnames - I've run into buffers that were declared as 53 characters in length, probably because that was what they needed on their system
* Large arrays being allocated on the stack - Linux distros have different default stack sizes
Most of the problems stem from the picky crap that C makes users go through for simple stuff like string manipulation. Both the scientists and the downstream developers like me would be MUCH happier if the scientists worked in something more forgiving (Matlab/IDL/Mathematica, or a flexible scripting language with a decent interface to heavy duty math libraries).