Software Carpentry Project's First-Round Winners 52
GenericBoy writes: "The four first round winners for each category of the
Software Carpentry competition have been
announced. It's really exciting to see all these potential replacements for tools that are in such wide-spread use. If you haven't heard of Software Carpentry yet, they're hosting a competition (for BIG $$$) for developing Free tools in the Configure, Build, Bug-tracking and Testing categories." As source code becomes less and less an obscure concept, it's good to see that some people are working to make its construction less obscure as well.
Re:Someone explain to me... (Score:2)
How about some decent documentation for autoconf/make/libtool etc. Everything I've found has either been reference, assumes you already know how the tools work, or is only for setting up a specific type of program.
When I've been able to get these tools to work, they're excellent, but trying to work out how to fix the couple of problems that turn up in each case is extremely frustrating.
Re:Someone explain to me... (Score:2)
However, a lot of the proposed programs (maybe half?) build on prior systems without any real new ideas. Some of the other ones look somewhat vague and use a lot of different terms. I liked the XML description of feature sets, but it looked too complex/verbose.
So I guess I'd rather see all this energy go into coming up with good examples for how to use the existing (robust, working, tried-and-true) tools. As usual, the best reference outside of good documentation is a real-live, working ugly program/script.
(For example, I learned a lot about Makefiles looking at both generated and custom-made ones, so now I have my own hybrid "favorite" Makefile-style...)
---
pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
Because (Score:2)
Re:RAISIN RACES (Must read!) (Score:1)
That, and I dislike the idea that a single langauge is ideal for all situations. Personally, I think a consistant User Interface to these utilities is much more important than what langauge they are implemented in. If they all work and act similar, then they'll be easy to use. And if people aren't forced to use Python, you might end up with better results....at the least, you'll most likely end up with more results, simply because you aren't locking people out.
When fixing the problem, fix it smartly. I agree with some of the goals of this project, I just really don't like how they're going about it.
By the way, MODERATORS SUCK !!!
It's about elegance (Score:1)
So what's elegance? Most people say it's the combo of simplicity and power. Not only does the python language itself possess these qualities, but by requiring python scriptability in project proposals, they pass on these qualities to the entire development process . Simplicity, because only one very simple language is required to enhance all tools in the environment. Power, because python is a modern object-oriented scripting language whose focus on usability, consistence and extensibility is extreme.
I mean, I love unix and the whole toolset philosophy as much as the next guy, but all those little languages are ad-hoc, limited and a pain to learn. By focusing on one very good language the software carpentry project can bring a significantly more elegant development environment into widespread use.
course, I think they picked the wrong language :-) Check out Olin Shivers' paper [mit.edu] for the knockout blow.
Re:Bleh... (Score:1)
Nathaniel P. Wilkerson
NPS Internet Solutions, LLC
www.npsis.com [npsis.com]
Re:Wow, looks good to me... (Score:2)
Documentation and DOL are extremely important... reminds me of a development tool that I have used that would compile psuedo code... haha. Funny stuff.
Have to agree. As it happens, I've been looking around lately for some higher-level design tools, and unfortunately, there's not a lot out there in the free-software realm. freshmeat lists Libero [imatix.com] and TCM [utwente.nl], which look good, but probably aren't the ultimate in software design.
Speaking of doc, what do you guys feel is the best tool for managing it. I mean something akin to doc++ or Doxygen. Are there any other better solutions?
I haven't really used any of the doc-generating programs out there-- but let me mention gtkdoc, which for C API's is probably the best of them all. (See here [gnome.org] for an example-- ain't it beautiful?) It works through Docbook SGML and the Jade processor to produce HTML, TeX, man, and anything else you have a stylesheet for. Unfortunately, the whole thing is a b**** to get working. It's a huge mess of Perl scripts and weird Makefile rules, I tried it one day and barely made it out alive
(But man, results like that GLib documentation would be worth it!)
Of course, however, all those tools assume that your approach to documentation is placing specially formatted comments (Javadoc-style or otherwise) above each of your module/type/function definitions. I don't know of a better way to go about it-- tying the implementation and documentation together makes it hard to neglect keeping the docs up to date-- but it surely isn't the only approach.
Re:Am I the only one who find this a bit offensive (Score:1)
What does money have to do with free software? I doubt very much if any of the top contestants would have bothered with the contest if they didn't enjoy the project itself. Money is just an added incentive. Besides.. A coder's gotta pay for his caffeine and cigarettes somehow, right?
If they don't take pride in their work, and produce crappy code, then they don't deserve to win anyways.
Re:Wow, looks good to me... (Score:1)
Knuth's Literate Programming [stanford.edu] is the complementary approach -- putting the code inside the documentation. There was some clunkiness in the original tools, but that has probably improved by now.
Both approaches are great for encouraging programmers to keep code-level documentation in sync with the code (though, of course, they can't guarantee it). Unfortunately IMHO this undervalues documenting your analysis and design -- you CAN put such documentation in your code, but I'd guess that most people don't.
Re:Python? (Score:1)
Mandated language may be great for 'them', its just not so good for 'us'. (I know its already the case with make, e-macs etc).
Anyway, if we use one language shouldn't it be e-lisp? I mean, everyone uses emacs don't they?
Tom
For portability (Score:1)
And the choice of Python is because of its portability.
Now that portability question if very important as far as you want to let everybody produce its own software or free software, even if you didn't live in a Unix world...
Have a look at Computer Programming 4 Everybody [python.org] for a little more info on that subject.
Re:Someone explain to me... (Score:1)
Lots of us who write tools do so to solve a specific set of problems we face every day. We know what the issues are that face us, and focus on fixing them. Few of us start by saying "how is the average Joe Programmer going to use this stuff?" Maybe we (the contestants) should have focused more on user-centred design approaches, like O-O use cases or XP stories -- or even Fred Brooks' suggestion of writing the manual first; you get some reasonably friendly descriptions of "how to do X with this tool" up front.
Maybe the SC folks can push user-centred design as a requirement for the deferred category (the one where they realized they needed to rewrite the requirements and run a second contest).
Vesta (Score:1)
[At the risk of letting the cat out of the bag too early...]
Anybody who finds this interesting might want to take a look at Vesta [compaq.com]. It's an advanced revision control/build system developed at Compaq's Systems Research Center [compaq.com] (the people who brought you the Personal Jukebox and co-developed the Itsy [compaq.com]). I've been using it for over a year, and going back to make/CVS style development now feels downright primitive. Among some of Vesta's cooler features:
It's not available today, but there's a lot of talk about releasing it.
Re:Complier? (Score:2)
but maybe we'll see an open sourced win9x compiler
There already is an open source compiler for windows: look here [cygnus.com].
--
Reply from Software Carpentry Project Coordinator (Score:4)
My thanks to everyone who commentd on the first-round results in the Software Carpentry design competition. I've replied to a few points directly below. Please note that many of these issues are addressed in the project's FAQ [software-carpentry.com] as well.
Allowing multiple languages moves the burden of learning new syntax and execution models from developers to users. (Essentially, it allows a minority to be lazy, while creating work for the majority.) Feedback from the 140 students who took the software engineering courses I co-taught at Los Alamos National Laboratory [lanl.gov] in 1998-2000 was unequivocal: as long as we keep making the tradeoff in favor of developers, people will continue to be frustrated and confused by the tools we build.
As an aside, one entrant from Europe said that he'd be a lot more impressed with programmers' sincerity if they put as much effort into creating documentation in multiple (human) languages as they do into arguing over what (machine) language should be used for coding...
We did ask people to keep their designs as language-neutral as possible, and I for one would be very excited to see implementations of (for example) the winning bug-tracking entry in Perl, Java, and other languages as well as Python. (If nothing else, it would permit a real "compare and contrast" study of the languages.) However, since we had chosen an implementation language, we felt it would be dishonest not to say so.
One posting to this thread said that the discussion list had "wandered off" onto the topic of representing programming languages in XML. This actually isn't off-topic: XML means that we finally have a standard language-neutral way of representing hierarchical data, such as the static dependencies in a makefile. However, we still don't have an equivalent way to represent procedural information (loops, conditionals, etc.). As a result, any non-trivial makefile (or configuration file for any other tool) must be bound to a particular language: most makefiles, for example, are tied to the syntax of the Bourne shell. If anyone's looking for a way to change the world, they could start by fixing this...
No: it's "free software" as in "free speech", not as in "free beer". As Richard Stallman said [software-carpentry.com]:
We are awarding prizes because we think people should be rewarded for hard work. The only regret is that we can't award more...
Amen. As the FAQ [software-carpentry.com] says:
Less formally, COM is proprietary, CORBA is bloated, and EJB is immature and lacks stable bindings for other languages. Creating a lightweight, usable, cross-platform, multi-lingual component system would be a good way to address the issues that Rob Pike raises in: http://cm.bell-labs.com/who/rob/utah200 0.ps [bell-labs.com]
Amen again. Everyone who has suggested categories for next year's competition has mentioned version control, and no-one seems happy with the tools we have today. However, we had to start somewhere...
Greg Wilson [mailto]
Software Carpentry [software-carpentry.com] Project Coordinator
Re:Am I the only one who find this a bit offensive (Score:1)
For instance, if you buy a car, you are free to change the colour, buy new wheels and tires, modify the engine, add a new stereo system etc. etc. Some of which may void the manufacturers warranty I suppose, but, the point is you are free to do it and you must then accept the consequences. While you own the car, you can drive it where you want. In the end you are free to sell the car to someone else. i.e. It's yours.
OT: SPAM-proofing (Score:1)
Remove the letters S, P, A, M, then rot13 to email.
My God man!! Why not PGP it, and provide the public key in the
When this type of obfuscation is necessary, SPAM needs to be killed [mindworkshop.com]!
It's Alive (Score:1)
ZDNet : give it up, others have.
Re:Scripting (Score:1)
unusually Well With Others. Making an API that
allows these tools to be scripted by other
languages probably won't be too hard.
Alex.
Re:Wow, looks good to me... (Score:1)
The only standard way of writing descriptive text together with source code is ASCII. Javadoc does too little, as do may other `solutions' for this. A better approach would be to include documentation production into the systems that Source Carpentry are after (so that design/analysis/in-code-documentation are expected parts of a program)
John
Am I the only one who find this a bit offensive? (Score:2)
I'll probably be accused of being a "zealot", but I really think that big-money competitions just aren't in the spirit of free software. There's nothing wrong with receiving some financial compensation for your work, but coding solely to earn money was exactly the concept that free software was supposed to eliminate.
Yu Suzuki
Re:Am I the only one who find this a bit offensive (Score:3)
So basically you're saying that free software is supposed to eliminate all commercial software (including free software written by a commercial interest). That is not true: the free software movement was started because (RMS felt that) all software should be free for people to use (and modify and copy and redistribute). And RMS is OK with the sale of free software; see http://www.gnu.org/philosophy/selling.html for more information about this.
Who cares if code is written for no other reason that to make money? As long as the source is available and freely modifiable, it's just as "good" (in the moral sense) as a piece of software written for love alone. It is "free".
Note that personally I think BSD/MIT code is more "free" in the sense of allowing freedoms to the end user. Just to keep people from responding to this with "GPL is not free", etc, etc.
reason for python (Score:1)
You're probably among the few ... (Score:1)
To me I do not concern myself with how much money a certain entity is making of his/her software, as long as it useful. I could not care less if whomever wrote it made millions or is bankrupt.
I'm on the mailing list (Score:3)
although I didn't send an entry in. I was mainly interested in the "Config" section.
It's been interesting. As soon as the entries were posted, some people who hadn't seemed to have read the spec immeaditly began posting stuff justifying the way they did things. There's nothing wrong with that in itself, but the fact that there was a lot of money hanging over the judges decisions made it worse than most lists - the list has driffed further and further off-topic in the last few days.
As soon as the results were announced, it went pretty quite again, apart from some interesting (but mostly offtopic) discussion of languages which are written entirly in XML.
I'm hoping that all those who didn't win don't abandon the project, but find a way to continue to contribute.
As for those who are saying that they don't see a need for these tools: Neither did I, until I heard some horror stories about people who couldn't build their softwear because of bugs in the build tools on some platforms, and the makefile/autoconfig scripts were so full of work arounds for other (Operating Sytem) problems that no one could figure out how to fix them. These are nuclear physicists we're talking about, too.
Bleh... (Score:2)
That, and I dislike the idea that a single langauge is ideal for all situations. Personally, I think a consistant User Interface to these utilities is much more important than what langauge they are implemented in. If they all work and act similar, then they'll be easy to use. And if people aren't forced to use Python, you might end up with better results....at the least, you'll most likely end up with more results, simply because you aren't locking people out.
When fixing the problem, fix it smartly. I agree with some of the goals of this project, I just really don't like how they're going about it.
Re:Bleh... (autoconf problems...) (Score:1)
- How may times has configure failed, giving some error message but no further explanation and no information dumped to config.log?
- Who decides what is and isn't a valid architecture description (i.e. i386-pc-linux-gnu works with some configures, not with others, i586-pc-linux-gnu works with even less)
- Running autoconf... if it drops out with an error saying `fatal bug, contact the authors', how does this help me compile the program??
- Running autoconf, autoheader, automake, libtoolize, auotgen.sh, etc. Why so many programs? Why are they in different languages, why do they seem to end up second guessing each other?
Things need to be changed to the effect that you don't get a huge bourne script with the tarball that may or may not work, and for which configure.in can't be fixed or used to construct a new configure program since yourJohn
Re:Bleh... (Score:1)
Nothing prevents me from doing so. However, one of the primary goals for this project, which I do agree with, is to create a set of tools all of which have a similar user interface, to make learning them and using them easier on inexperienced developers.
Redesigning multiple important tools, including autoconf/automake, GNATS/Bugzilla, and a regression testing suite is *not* a small project. It requires a number of people, and a lot of work. This project is providing that framework, they're just limiting it unecessarily, in my oppinion.
Wow, looks good to me... (Score:2)
I think the open source community needs to spend more time in the software design phase. Where I work, we don't code until we have TONS of diagrams/description of logic done. Overall, I think it leads to a much better product.
Documentation and DOL are extremely important... reminds me of a development tool that I have used that would compile psuedo code... haha. Funny stuff.
Speaking of doc, what do you guys feel is the best tool for managing it. I mean something akin to doc++ or Doxygen. Are there any other better solutions?
Catcha later,
Ryan
"Don't nargin your MEX files!"
Re:another misusage of moderation points (Score:1)
Intersting paper.
The only problem I have with it is it is so UN*X centric that it ignores the vast number of Very Very high level development environments that are available on more "traditional" platforms.
A few worth mentioning from the 70s are:-
MarkIV - a very propriettery report generator.
FOCUS -- One of the very first proctival 4GLs, and still very widely used.
The current state of the art in VHLLs is probably represented by Forte which was recently bought by Sun, and, still IEF (used to be from Texas Instraments -- bough by someone else recently).
What all these languages have in common is that they try to model an entire problem space, Report Writing in the case of MArkIV, to, Business Systems in the case of Forte.
For low level problems there is nothing better than a low level language. Extremely sophisticated high level languages like C++ plus STL will always disappoint becuase they are centered around coding tecniques rather than problem solving.
Incidently if you are interested in a programing language where you could insert a diagram try XEROXes "MESA". It alowed you to use the normal VP wordprocessor as your editor. Text in "courier new" was considered compileable code, everything else was comments -- a very neat trick which I have never seen emulated in any other language!
Re:Python? (Score:1)
Whilst they could have left the detail of the implementation language till later (allowing for other languages such as PERL), it would make things more difficult at a later stage to decide on one language and mandate that for the implementations of all the winning designs.
So far as the XUnit suggestion, who would want to write an implementation of all the tools in every language??
John
Python? (Score:3)
They never said Python was an ideal language for all situations, just that it did enough well to fit their design. You're right, of course, that allowing other languages creates a bigger participant pool, but adding other languages introduces other support problems; language compatibility across platforms, versions, OSes, patches, etc. It's tough enough with ONE language, let alone many/all if they didn't require Python.
I mean, part of the initial design document was ease of use, maintenance, and management. Single language seems a reasonable way to get this. Single OS is not, but notice they support NT and Linux. Hopefully Mac and other Unix support will fall out of the fact that it's Python code(Python runs on other systems, of course), and that no Win or Linux dependent design is implemented.
-AS
Re:Bleh... (Score:1)
The actuall language used doesn't really matter the ideas are the power of programming, ... the creativity. Porting it over to another language or platform is just simple mechanics.
Are you serious? Would you write a heat exchange simulator in Perl, or grep in FORTRAN?
Bjarne Stroustrup wrote something like "a computer language is not only the code in which we express what we want the machine to do, but also the framework in which we model our understanding of the problem". It's been a couple of months since I read this, but the general idea is along that line. Some problems are *so* obviously better modeled and solved in certain languages.
OTOH, maybe you mean that the better (clearer) your code is, the easier it will be to port it to another language. I agree with that. Brian Kernighan says that porting a program to another language is the "ultimate portability test", altough that doesn't mean he encourages porting per se.
Someone explain to me... (Score:3)
Not only are the standard tools good, but there are already alternatives! (cook, autoconf, configure, blah blah blah...)
---
pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
Scripting (Score:3)
Also there is the question of migration: suppose I want to use one of these super-wizzy tools on my existing project: can I just feed my existing makefiles into a converter to set them up? If not, then they will have a hard time gaining penetration, since old projects will never move over, and many "new" projects start out as mutations of old projects.
Another thing that my (cursory) scan of the site didn't turn up was interfacing to source code management: I'd like to see a good integration into CVS, so that I can easily build different trees from my source base.
Lastly, while they are requiring some degree of cross-platform support (IMHO if you can get the tools working under NT as well as *nix, you have a high probability of getting them working under any OS) do they have the ability to support building the same project on different hosts without modifying the project? For example, I develop embedded systems running VxWorks: I can theoretically build under NT or Linux, but keeping my make files so that they will work under both is a nightmare (Thank you Microsoft for not using / as a directory seperator! Thank you for not having a decent shell! May I have another?)
I'm just glad somebody is looking at this part of the development process and saying "That's great, but is there a better way?"
Python... but no code? (Score:5)
Complier? (Score:1)
2 things: (Score:1)
Chris Armstrong
Re:Bleh... (Score:1)
The fact that they mandate the use of Python is one of my primary objections.
What, is something preventing you from writing a tool in $OTHER_LANGUAGE and announcing it on freshmeat? So it can't be entered in a Python competition. Oh well.
Re:Bleh... (Score:2)
On the other hand. I think they picked some really good tools to work on. I don't have much experience with the other two tools, But make and autoconf I have to say are *seriously* in need of some updating. Make has a bad tendancy to break on stuff that I worked on years ago, requiring a whole lot more work then I really want to put into it trying to fix it. And autoconf, geez, call me lazy, but its just too complex to be worth the trouble, at least in my experience. Putting some money on the table is certainly one way to make things happen. Kudos to ACL for making this possible.
Spyky
Why do we need these tools? (Score:3)
blah blah blah is a very weak argument compared to the statements found, for example in the paper for BuildConf. Also you only list two programs, cook and autoconf. Configure, last I checked, is an autoconf generated script.
Of these programs I've only used autoconf, and I must agree with the author of BuildConf, that it's poorly documented and confusing. It forces a syntax on you (m4) which is rarely used in other situations. It does many things right, but this, combined with a lack of good documentation on it, is a fatal flaw.
And more importantly, why does it matter if we get other, better tools? You may prefer better documentation to autoconf, I may not. Does that make me wrong, or you wrong? not at all. There's room for all of us, and software darwinism will help choose the best (most adaptable, most powerful, most appropriate) tool.
----------------------------
Re:This... (Score:1)
Re:Python? (Score:2)
The great thing about XUnit (for example) is that there is an implementation for each language (even VB)
And everytime you want to upgrade the code or do a bugfix, you have to write, debug and synchronize x sets of code. It's a nice idea but personally, I'd rather have that army of coders do something else ...
Besides, learning a language isn't that hard, especially one like python. Most techies know more than one language anyway and learning more has never hurt one's resume :)
Re:Python... but no code? (Score:2)
Chris Armstrong
Re:Bleh... (Score:1)
Then I came to the conclusion, so what? If any of these utilities is actually useful it will be sufficiently generic that it can be reimplemented in other languages. Someone will produce a C version, which will subsequently be be ported to every platform in existence, and maybe, just maybe, we'll be one step closer to replacing 'make' with something better. So in the end, I see this contest as a good thing, though the python lobby might not like my take on it.
Re:Bleh... (Score:2)
Yes there is... Machine Code.
(-1, pretentious)
Re:Bleh... (Score:1)
On top of that they pick Python, which is probably the only language they could pick that I have absolutely no knowledge of :-)
So take an afternoon and learn it. www.python.org [python.org]
Seriously, if you've programmed in any other language you'll pick up Python in no time. Plus, you'll be glad you did.
Re:Bleh... (Score:1)
Re:another misusage of moderation points (Score:1)
But yes, note to moderators, take the time to read that sig of mine before you moderate. Call me "Troll, troll, TROLL" all you like in an Anonymous reply an ye must, but please back it up with some reasoning...
---
pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
Re:another misusage of moderation points (Score:2)
As well as newer, research-oriented languages and platforms which are to Unix and Perl as those are to, say, CTSS and assembly.
For low level problems there is nothing better than a low level language. Extremely sophisticated high level languages like C++ plus STL will always disappoint becuase they are centered around coding tecniques rather than problem solving.
I'm sorry, buty I don't consider C++ to be much sophisticated, or even high-level. It's just as low-level as it was in the days when it was merely the preprocessor Cfront. It's still little more than a portable assembly language with some more-or-less-OO features tackled on. (See my recent post on compared "OO" languages for more on that.) As far as I'm concerned, C++ is mid-level at best; traditional-ish and imperative but relatively abstracted languages such as Perl and Python are high-level, and only those which are radically different from the way in which the computer actually operates (pure functional languages such as Haskell, logic languages such as Prolog, rewriting languages such as Maude, pure OO languages such as Self) can truly be called very high-level. (Of course, some languages span much of the spectrum, such as Common Lisp, which allows you to program at any necessary level of abstraction, from C-like programming with unboxed values, hard pointers and - gasp! - even a GOTO form, to something close to VHL - it fully supports functional programming, and I myself once wrote a three-page "PRolog In Common Lisp" embedder.)
Incidently if you are interested in a programing language where you could insert a diagram try XEROXes "MESA". It alowed you to use the normal VP wordprocessor as your editor. Text in "courier new" was considered compileable code, everything else was comments -- a very neat trick which I have never seen emulated in any other language!
Hrrrm. I might note that this isn't really the best way do do things - it's a better idea to use a structured writing tool such as LyX, with output indicative of content semantics, not of content layout, to write literate programs (as described by Knuth), and have in conjunction with a program (maybe even a plug-in for said editor) to "weave" these programs into compileable code and formatable text, as needed.
In blatant self-contradiction, I now present a hastily written Perl filter which, as requested, turns an HTML file into Perl code.
#!/your/path/to/perl -w
undef $/;
$file = <>;
print '#!/usr/bin/perl -w';
print "# $1\n" if m!<title>(.*)</title>!i;
print "# Preprocessed from $ARGV[0] on ", localtime(time), "\n";
print "# PART $c\n$1\n\n"
while m!<font
&& $c++;
exit;
FAQs and Rules (Score:1)
Chris Armstrong
Offtopic? (Score:1)
Of course I could be wrong maybe the moderator read the post and thought it should be moderatated down. If this was the case troll, flaimbait, or overrated would all be better choices. Also if the moderator thought this guy was a flaming idiot and couldn't be more wrong a post detailing exactly how he was wrong would be a better option than any moderation.