Make Out with SCons 77
Nice2Cats writes "I stumbled across this and can't find a mention of it here yet: SCons, a stable alternative for the vilified make. Built-in support, they claim, for C, C++, and Java, built-in support for use with SCCS, CVS, and BitKeeper, supports .NET, and works with Linux, *BSD systems, Windows NT, and Mac OS X. I'd love something to would finally get rid of the awkward (no pun intended) make, and having a system based on Python would make it double good. Maybe one of the serious C/C++ programers here could give a verdict on if this is the dragon slayer we've been waiting for."
Be careful... (Score:1, Funny)
SCOns? (Score:1)
Re:SCOns? (Score:1, Redundant)
Re:SCOns? (Score:1)
Questions (Score:5, Insightful)
Is SCons viewed as a replacement for Autoconf and Automake, or just a replacement for Make?
Built in support for...
What does this mean? Will SCons somehow only work with languages that SCons has built in support for?
How does SCons compare to Apache's Ant?
Re:Questions (Score:3, Informative)
Well, I haven't actually looked at scons, but in Make, there is only built-in support for a limited amount of languages. For instance, (maybe this has changed recently) with ObjC you have to define how to take care of
How does SCons compare to Apache's Ant?
Well, the fact that it isn't based on Java or XML gives it
Re:Questions (Score:2)
The good thing about doing ant conf files in XML, si if they change the format, you can use xsl to transform ALL of your xml to a new format.
As for the java thing, run's most everywhere.
Re:Questions (Score:1)
Re:Questions (Score:5, Insightful)
Layers upon layers of complexity. XML, XSL, Java, Ant Schemas...all for compiling a program. My sig is appropriate, here, too. A build system for software should be easy, intuitive, transparent, etc. etc. There has yet to be a replacement for make that has these attributes (makes tab syntax really isn't that big of a deal, either, even though that's what most people latch onto when complaining about it).
Small shell scripts and makefiles. Is more really necessary to call cc and ld or javac or whatever for each file in a directory tree? The differences among platforms can't be so dire as to require gigantic build automation tools, which introduce more problems than they solve!
Re:Questions (Score:4, Insightful)
Whoahoahoa.. no. Blatantly wrong. If ant 1.0 used one particular XML format, and you upgraded to ant 2.0, to go from an ant 1.0 file to 2.0 file, you'd apply an xsl style sheet ONCE to upgrade it all. Any future XML you write would be in the 2.0 format. Simple as that.
Bash? csh? sh? tcsh? bash 2.0? Which shell are you suggesting is usable across them all?
ant is just a program, just like make is. You run it against a config file of a certain format, just like Makefiles have their own format. Ant is no more compicated than make. hell.. you can use the core stuff from ant to run any program across many files, just like make. The targeted tasks are just to make those easier.
Re:Questions (Score:2)
What do you do when XSL can't adequately map the files? What do you do when the transformation gets you only 90% of the way there requiring time-consuming debugging and finish work? Murphy's law applied to nifty ideas like XML means that there will absolutely be unanticipated shortcomings that aren't well understood due to the complexity and immaturity of the tools being used.
Bash? csh? sh? tcsh? bash 2.0? Which shell are you suggesting is usable across them all?
A
Re:Questions (Score:4, Interesting)
No. A build system for software must reflect the complexities what it is attempting to build. No build system for C++ programs can ever be fully "easy, intuitive, transparent, etc. etc." because there are C++ programs that aren't any of the above. Same for C.
Ever tried to compile XFree86 by hand from source? It must be possible, because Gentoo can do it, but I've never managed myself. While it can probably be made easier, the simple fact is that it's a complicated set of programs with a complicated build order and no matter how hard you try, it would take a full-fledged AI to make it "easy".
On the other hand, a pure Python project, even a large one, needs no special build system at all, because Python automatically compiles with few or no hassles (and what hassles exist can not be solved by a build system, they tend to be user error). Of course large projects are rarely all Python (building C extensions, compiling human-readable data representations into faster data or code, etc.), but since Python is more simple, the build system can be equally simple (or in this case, non-existant).
Make can be improved on, it's a decades old design with multiple layers of hacks jostling each other. A new, clean design can at least simplify away the parts of the build process that involve "fighting with make". But a build system for C can only become finitely simple, or you'll start to lose capabilities that C has because your build system can't handle them.
Small shell scripts and makefiles. Is more really necessary to call cc and ld or javac or whatever for each file in a directory tree? The differences among platforms can't be so dire as to require gigantic build automation tools, which introduce more problems than they solve!
Proof by counterexample: autoconf exists, therefore it is clearly necessary, because nobody would trouble to build it if it didn't address a problem. (Whether it solves it is another question, but given its vast popularity in the open source meritocracy I'd say the onus is on you to show that it truly causes more problems then it solves; every project that uses it is a vote against that statement.) You've clearly never written massively cross-platform code; grab a decent sized package, look at the output autoconf generates, and look at what the program does with it. Odds are every single line you see during configure, with the possible exception of some really standard ones that seem to be a standard part of the package ("CC works?"), is used somewhere in the program, especially lines like "Is your pthreads implementation broken?
Re:Questions (Score:1)
Re:Questions (Score:2)
look at the output autoconf generates
Worse, as you begin to try to standardize coding of your configure.in and Makefile.in files, it begins to dawn on you just how much more boilerplate could be automatically generated.
Ergo, automake [redhat.com].
However, by the time I started learning the syntax for automake and tried to figure out just exactly how libtool can magically make shared library construction and maintenance a cross-platform reality, I get tired of the whole damn business and think: there has got to be a
Re:Questions (Score:2)
Compiling a program and linking it into executables, libraries, jar files, etc. simply isn't that hard to do. A programmer that doesn't understand how to do it has some learning to do. Tools that claim to make the learning unnecessary are misleading--what does the programmer do those tools break and leave them stranded?
Ever tried to compile XFree86 by hand from source?
Re:Questions (Score:2)
Nothing is perfect. Autoconf was built, essentially, to abstract imperfection from the developer.
Read up on its history, it's in the info pages. Back when Linux was a nothing, people wanted free software to compile on pretty much any machine they could. They all varied in obscure ways, some had features others did not and so on. Before autoconf, pe
Re:Questions (Score:2)
I'm not exactly sure what the original problem was that motivated autoconf,
It seems the programmers write what they please and, then, invent these kludges of tools to deal with those programs.
Between these two statements, combined with the fact you didn't deny it despite obviously going over my message with a fine-tooth comb, you're only reinforcing my original guess that you've never writt
Re:Questions (Score:1)
I'm glad you are such a good judge of character.
I actually don't disagree with what you wrote about your experiences as an uber-elite programmer, because that's the reality of working with a large corporate bureaucratic mess of software that was created over years of short-sighted decision-making. My original argument was simply against the kludges being presented in the open-source world as suitable build tool
Re:Questions (Score:1)
Just write everything in Perl, and you don't have these problems.
Re:Questions (Score:2)
Re:Questions (Score:1)
>
> than Perl.
Only in theory.
In practice, some systems use csh as the default sh. In practice,
getting any sh at all on some systems means installing an entire
POSIX layer that doesn't come standard, but Perl can be installed
without that. In practice, some POSIX systems don't implement the
full POSIX standard, so you end up tracking down niggling bugs
where one platfor
Re:Questions (Score:1)
I've looked at many many different supposedly better replacements for make. None of them do the job as simply as make.
Re:Questions (Score:2, Insightful)
simple script to do the transformation. XML helps enforce a certain
amount of regularity, but it's by no means the only regular format.
Personally, I like XML pretty well in general, and I really like
what OO.o has done with it in particular.
I'm less enamored with Java, but that's another thread for another day.
Re:Questions (Score:2)
True, but writing parsers for things like, perl, is a pain in the butt. At least with XML, writing XSL to transform is realtively easy. Even for ant.
Thank god make isnt' written to use perl as its language. *shudder*
Re:Questions (Score:1)
If the software were written in pure Perl, you wouldn't need make
to install it.
Re:Questions (Score:3, Informative)
Adding implicit rules in make is not difficult. It's in the man page (even Solaris' crappy man page for make describes it well enough, and the system default rules file has lots of examples).
Well, the fact that it isn't based on Java or XML gives it a good head start
Agreed.
Re:Questions (Score:2)
Sorry, didn't mean to imply that it was difficult. My point was that having "built-in" support for a subset of languages in scons is no different than make. And I don't doubt that adding ObjC support to scons would be much harder...
Re:Questions (Score:1)
No Screenshots? (Score:4, Funny)
Re:No Screenshots? (Score:1)
Already there... (Score:3, Informative)
It's called Ant [apache.org].
Daniel
SCons? (Score:2)
and thought "No thanks"
SB
stable alternative to make? (Score:3, Interesting)
One thing I never liked about make was the dependence on whitespace in the makefile. Hopefully this Python-based solution will fi...uh never mind.
I predict Make will be around for a while still. I've used Ant but Make is fine for 90% of what I do.
Re:stable alternative to make? (Score:2)
Yay, let's build a replacement for make, using a language that repeats the biggest design flaw of make.
Re:stable alternative to make? (Score:1)
Then I found Python. I didn't like it in the beginning but I found that Python code is very easy to read because it enforces a certain format.
Of course, Python doesn't repeat the biggest mistake of Make: It doesn't bother if you use spaces or tabs as long as your tabsize is 8.
And th
Re:stable alternative to make? (Score:2)
Re:stable alternative to make? (Score:2)
Does not solve the problem, but makes it even wors (Score:2)
Re:Does not solve the problem, but makes it even w (Score:5, Informative)
It can, among other things, generate Makefiles or Visual Studio project files, or run the build itself. It does have "autoconf" style checks, which you can embed in the sconstruct/makefile. It's built for parallel make from the start. It does not suffer from network clock skew.
And it has much better multiplatform support than any tool I've seen so far (automake/autoconf, make, qmake, tmake, cmake).
It will take time to mature, and it doesn't solve the problem, but it does makes things much better.
YAMR (Yet another make replacement) (Score:2)
So now my C, C++, or Java programs can be dependent on yet another language runtime environment? Awesome!
Yes, there are limitations to Make, but why hasn't a genuinely better replacement materialized in the last twenty years and become truly widely adopted (in a manner like Perl, for example)?
Truth is, if you avoid fancy GNU extensions and stick to POSIX, make
the make replacement is (drumroll please): sh (Score:2)
make's syntax is so incredibly baroque. Its basically only to do one thing, but over the years its gotten so much crap stapled onto it that its a time trap. Add to that automake, which has to be layered on top just to keep make doing its job. Stop the madness!
[rant-off]
Nowadays I just write my build scripts in sh. Its on every platform. It takes less time in the long run because its a real programming language.
For example, in make when you want to do arcane-thing-X, you have to dig through
Re:the make replacement is (drumroll please): sh (Score:2)
Not that I'm a big fan of either, but the combination of two really makes a powerful system.
Re:the make replacement is (drumroll please): sh (Score:2)
(well, OK, time for complete honesty. I do use really bash, and I do occassionally have a bug on windows from windows' non-unixy filenames. There, that's off my chest.
Re:YAMR (Yet another make replacement) (Score:2)
The Java community uses Ant almost exclusively, and Ant kicks the living crap out of make as far as, well, everything.
True, the C++ community has had trouble coming up with anything better than make, but come on -- we're talking about people that consider C++ to be a reasonable 21st-century programming language, which already leaves me with serious doubts ab
SCONS is awesome (Score:5, Interesting)
Since then I've shifted several of my projects to using SCONS as the build system. I'm sure it's theoretically possible to achieve what I was trying using make, but I wasted many hours wading through man pages and howtos without success, and SCONS did it with almost zero effort.
I haven't tried using it on Windows yet, but my understanding is that you can use the same script under both platforms.
So, just one data point, but my experience with it has been 100% positive.
-alyosha
Re:SCONS is awesome (Score:2)
I work on a project of about 600+ cpp files, we build on linux (gcc2 and gcc3) and windows (gcc2 and gcc3 on cygwin and mingw, msvc). Scons worked almost unchanged with the cygwin and mingw versions of gcc. VC++ required some minor changes, mainly because of the specific command line switches, and because the guy who 'ported' it was very anal about integrating it perfectly with his miscr
I never thought I'd see the day (Score:2)
Re:I never thought I'd see the day (Score:2)
My SCons experience (Score:5, Informative)
We use SCons at work to build a huge, highly componentized project with upwards of 5000 source files. It's hands down the best experience I've ever had for building large projects. The other two big projects I've been on used GNU make and Microsoft nmake, and neither was satisfactory.
The nmake-based system required running "nmake depends" before a build, and it simply wasn't reliable. Paranoia was high; when we hit a bug while developing, the first thing we did was a clean build, just to make sure we weren't looking at a build problem. (It was justified paranoia -- half of the time, it WAS a build problem
SCons was loads faster than our (admittedly bad) GNU make recursive build system, though noticably slower than our nmake build. With that cost, though, you get a correct build every time, something we never got in our "make depends" system. The paranoia is gone. SCons caches dependencies, so the cost for this accuracy isn't as bad as you might think. There were hiccups in the past, but the C++ dependency generation has been rock solid for at least nine months now.
I have a little toy project at home with maybe 25 source files which also uses SCons. I use MinGW's gcc at home, and SCons worked well with it "out of the box". Right now all I do is maintain a list of source files to build just like I would with any other system, but the setup was painless, and it's awesome to know that I can write Python to solve any problems I might hit later.
My big complaint with SCons is that a no-op build (running SCons when no source has changed) takes way too long on our big project, making iterative development painful. There's a command (-U) to build only components in a subtree of your file system, but that doesn't really speed things up much. Hopefully they can improve on that. That's about my only gripe, though.
Re:My SCons experience (Score:2, Informative)
(SCons developer here)
Yes, no-op build time is one thing we continually optimize for. Believe it or not, it's about 50x faster than it was at our initial release :)
Also, there are several things you can do to sacrifice potential build correctness in order to speed up no-op build time, which are documented in the SCons Wiki. [scons.org] Unfortunately the Wiki appears to be on the fritz right now.
Our philosophy is that, by default, we provide an absolutely correct build, which means recalculating/rescanning a lot
Re:My SCons experience (Score:2)
Why, of course it was. Remember, Recursive Make Considered Harmful [tip.net.au]. By just applying that insight to a medium library after first reading the paper, I saw accuracy go way up and scan time go down by an order of magnitude. Oh, and no-op builds went to maybe 10 secon
A-A-P (Score:3, Informative)
Something similar/related is A-A-P [a-a-p.org] from Bram Moolenaar of vim. A 1.0 version was just released last week. It is a make+Python kind of thing. There was short article [osdir.com] about it at osdir a few months ago.
Larry
Re:A-A-P (Score:2)
Yeah: I'd have thought that The A-A-P Project or JAM
Perforce.com, Jam [perforce.com]
would be the ticket...
Has anybody compared 'em?
Re:A-A-P (Score:2)
Re:A-A-P (Score:2)
Thanks for the info
: )
So now anyone compare The A-A-P Project with SCons?
Slashdotted? (Score:1)
And I just checked out - Gentoo has an ebuild for it too. Go Gentoo!!
Nandz.
jam (Score:3, Informative)
The jamfiles are easy to set up, and much faster to calculate dependencies -- I built freetype via gnu make and via jam. Jam was done building before make had even started compiling!
Also, props to perforce. It's proprietary (just like bitkeeper), kicks cvs's ass (just like bitkeeper), and is free for open source projects (just like bitkeeper). MySQL uses p4, IIRC. Unlike bitkeeper, you can get a single-seat version free (I use it for my internal projects, though CVS would probably suit my needs).
Re:jam (Score:2)
Make vs. make (Score:3, Interesting)
One thing I see is a lot of people referring to how slow and unreliable recursive makes are, and they're completely correct. However, you can make your life a lot easier by using make but getting rid of the recursion, using a make with an 'include' feature. See Recursive Make Considered Harmful [tip.net.au] for nice write up on why recursive makes don't work, and how to do it correctly.
I converted our build system, and no-op builds went from a few minutes to about 5 seconds. Really. If I change one file, all the necessary stuff gets rebuilt, and none of the unnecessary. Every time. With a little gmake if-then-else (all in one place, not scattered around in a bunch of files), the same Makefile and segments work on Linux, AIX, and Solaris. And while the organization of the files is a little unusual for those who are used to the recursive style, they are just makefiles, no new syntax.
The downside? It pretty much requires gmake, but since this particular product is proprietary, we control the build machines. I don't think you can use automake...but I consider that a positive :-). You could use autoconf -- just put all the config stuff in one makefile fragment, and include it into the master.
Re:Make vs. make (Score:2)
1) Get rid of cyclical dependencies!
2) Separate global vs local interfaces.
3) Have a centralised make system that works 90% of the time by filling in a list of source files.
4) Differentiate between libraries and executables.
How do you do this? We use a simple build order.
1) Copy out the changed global header files.
2) Build your libraries.
3) Build your shared libraries.
4) Build your executables.
Easy enough to do. We have a subsyste
Re:Make vs. make (Score:2)
In other words, you've worked around the flaws in the recursive make system by forcing an artificial structure on yourself. Which is not to say that some of your structure isn't a good idea by itself, but the first time you forget to build something in the right order, you have a broken build. Or, you spend a lot of time building stuff that isn't immediately needed.
I can't over-emphasize how nice it is to be able to be able to type 'make foo' from anywhere in the source tree and be confident that what I'l
Re:Make vs. make (Score:1)
As with any large scale software engineering, you will need to put some sort of structure in or you will fail. Not might, not possibly, but will.
You will need some things like code style guides, documentation guides, testing gui
Re:Make vs. make (Score:2)
SCons: so far so good (Score:2, Interesting)
We are using it on Windows (MSVC and Intel compilers), Linux, IRIX, and Mac OS X (gcc and two versions of CodeWarrior). Handles all of those with ease. It can do things like properly handle dependencies on auto-generated source and header files, which
Blah (Score:2)
Everyone tries to replace make and freinds, and everybody fails. (Well, at least fails to be a true replacement that's actually better in every way). A few small tweaks aside, you can't really make serious design improvements to the standard unix make and related tools without taking some large tradeoffs that make it unacceptable for at least some uses. It's a very flexible tool as it stands, and it's hard to make it better without losing that flexibility.
SCon vs ant for java? (Score:2)
For a large primarily Java project ant is the hands down winner. It has two advantages over make. Out of the box, ant "knows" how to build java and java technologies. The same way that the make "knows" how to build C/C++ and UNIX style stuff. Sure you can do this in make but not without a lot of additional rules and setup... and it never feels as seamless as in ant. The second thing ant does is only use one java JVM to do th
parallel builds on Win32? (Score:1)
Re:parallel builds on Win32? (Score:1)
BB Scons config file (Score:2, Funny)
Plan 9's solution (Score:1)
Plan 9's new take on `make' is `mk'. Unfortunately, I wasn't able to find a good on-line source for the original paper on mk:
I seem to have no trouble finding things that reference this paper. The following might also be helpful:
Yet Another Build Tool (Score:2)
The nice thing about SCons is that you code the "makefile" in Python instead of some build-tool specific language. Of course, that's also its main disadvantage. Build tool languages are typically nicer for this specific problem domain. Not to mention you need to install the Python run-time environment first. Well, not everybody wants to install it just for the sake of a build script.
As for its other claims to fame, there's nothing new about them
Re:Yet Another Build Tool (Score:2)
a) It is 100% free software, unlike, say, Jam
b) It can work with C and C++, which are the most common languages (on Linux at any rate) today, whereas for instance Ant is tied to Java.
c) Python is a widely known and easy to learn language.
d) They have great documentation.
I've already encountered several projects pondering moving up the "next level" in tools, ie Subversion with some new build system (usually scons). At some point, one of the new system
Re:Yet Another Build Tool (Score:2)
By the way, do you use Cook and Aegis in Cygwin? I've tried to build them in Cygwin once - both seem like still not ported completely yet.
Awesome (Score:2)