Has Software Development Improved? 848
earnest_deyoung asks: "Twenty-five years ago Frederick Brooks laid out a vision of the future of software engineering in "No Silver Bullet." At the time he thought improvements in the process of software creation were most likely to come from object-oriented programming, of-the-shelf components, rapid prototyping, and cultivation of truly great designers. I've found postings on /. where people tout all sorts of design tools, from languages like Java, Perl, Python, Ruby, and Smalltalk to design aids and processes like UML and eXtreme Programming. I'm in a Computer Science degree program, and I keep wondering what "improvements" over the last quarter century have actually brought progress to the key issue: more quickly and more inexpensively developing software that's more reliable?"
Thinking about thinking.. (Score:5, Informative)
Absolutely! (Score:4, Informative)
If Perl had an IDE that was as easy to use it would dominate the world. (more than it already does)
Interesting article posted by Joel about this... (Score:1, Informative)
--Shashank
Re:Functional languages (Score:2, Informative)
But the learning curve for most people seems too much. OO on the other hand looks similar to more common programming paradigms. Unfortunately I have always found it does not scale well.
nope (Score:2, Informative)
Re:Here's a thought... (Score:3, Informative)
I've been spending a lot of time programming in PHP lately, and although my code might be more readable, (especially to someone with just a C programming background) I spend a lot more time working around quirks in PHP (either version issues, library functions, or design flaws... sometimes all three!) than I ever would in Perl.
However, the repositories do make all the difference. Although there isn't quite as much out there for PHP as there is for Perl, there's still a substantial amount, and that helps make it all bearable.
Re:I haven't programmed much lately... (Score:1, Informative)
Well, nobody's perfect
Nowadays we've got great tools like Flash, scripting languages like VB Script, and markup languages like HTML.
Not sure what that has to do with your first statement about java, unless you're just spouting random thoughts. Or perhaps you still think java is for making silly applets for web pages. How 1997 of you.
Not every programmer these days is a old COBOL nerd, ASM coder, or C junkie.
Right, some people see are slightly smarter in that they just use the right tool for the job and feed their families.
I yearn for the days when Borland was great. The Turbo C++ and Turbo Pascal products probably got half of the programmers in the 80's, late 90's started.
Got to agree with you here. Borland's old products were great back then. I still use their c++ source code color scheme in the preferences of most IDEs I use. Gotta love dark blue background with the yellow and white text.
Nowadays I tend to like things like RAD tools and code-completion.
Best tools, present and past (Score:5, Informative)
Here is my take on the best environments available today (in order of best first) - I only include stuff that I use:
- Cincom VisualWorks Smalltalk - a really fast coding environment with support for SOAP/WSDL/UDDI/all possible database connectivity, built in web development tools, huge class library so often new applications are very little new code
- Microsoft VisualStudio.Net - I almost hate to include this one, but I am trying to be honest here
:-). Last year when I was writing a J2EE (Java Enterprise) book, I took a good look at the competition and liked C# and .Net.
- Java with light weight web services - I am a HUGE fan of Java on the server side, done light. By light I mean that I like to just use servlets and JSPs, etc. and only use heavy weight J2EE stuff when I really need transaction support, etc. that EJBs provide
- Common Lisp - lots of great commercial tools from Xanalys and Franz, and lots of great free tools like CLisp, CMULisp, OpenMCL, etc.
- Python - Love it for small projects, munging text, etc.
- C++ - only use it when I must - great runtime perfromance, but a pain to code in
It seems like so much work today requires deploying web services so I appreciate tools that make it easy to develop basic application logic separately from any user interface, get it working, then write web and/or standalone GUI interfaces.-Mark
Warnng: I am blogging now: http://radio.weblogs.com/0115954/
2nd Edition of "The Mythical Man Month" (Score:2, Informative)
Frederick Brooks lays out a summay of various rebuttals to "No Silver Bullet" (there have been, 'a couple' of papers on the subject)...
Its worth a read.
Re:Is it engineering, or not? (Score:5, Informative)
I disagree. That might be true in many cases, but some of the truly great engineering feats were produced quickly by a small team of very, very good people. A classic example is the Lockheed Skunk Works under Kelly Johnson. The SR-71 was conceived in the late 1950s, yet has performance numbers that are unequalled even today. Even more amazing, it was designed and built by just a handful of people. Compare that with the thousands of people it takes to build a new military aircraft these days. Other examples of the Skunk Works' prowess include the P-80 (USA's first operational jet fighter), the U-2 (high-altitude spy plane), F-104 (first mach 2+ jet fighter), etc.
The moral of the story is you can perform very high quality engineering in a very short time (the P-80 took something like 140 days to design and prototype!!) if you have an incredible team with no red tape in the way. Unfortunately, this is rare.
Re:Here's a thought... (Score:2, Informative)
Take a look at Python [python.org]. It is as powerful as Perl, if not more so, and excels where Perl is lacking: syntactical clarity. I often wish C++ was as clear as Python.
Recently I was faced with a little programming job: Recursively parse a directory tree, examine all the files in said tree and do a complex search and replace operation that required some smarts that couldn't be implemented easily with regular expressions. I dreaded to hack that up in C. Even though I had programmed some perl a while back, the syntax was so convoluted that I couldn't remember any of it. Fortunately, I had recently bought a couple books on Python (but hadn't looked at them yet). After one hour of reading and coding, I was done with the little project! Granted, the code did not utilize a lot of powerful features, but it worked like a charm and got the job done. Since then, I've been using Python with ever increasing pleasure.
The only issue with Python is the "grouping of statements by indentation" which may take you about 10 minutes to get used to unless you hate it so much that you can't get over it. But before you condemm it, read Guido van Rossum's comments [python.org] on the matter.
An interesting comparison of the major scripting languages with C++ and Java can be found here [ipd.uka.de].
IMHO, one of the major factors of increased programming productivity is the spreading use of powerful, interpreted languages like perl, python and Ruby.
Design, design, design (Score:3, Informative)
The most important thing in developing a product is having the "right" design.
What he was getting at is that, if you rush into devleopment (coding in this case) without having fully thought the design though, you could end up shooting yourself in the foot later on. Redesigning and recoding something later on in its development cycle can be hideously expensive and time-consuming. Also, depending on the industry, it can be fatal (think lost contracts from being late). It's absolutely vital to think all the issues through regarding your product, not just in the short term, but what you might anticipate for the long (and in many cases, the very long) term.
eXtreme Programming == Try it first (Score:4, Informative)
This is where so many people get it wrong. Making software is not analogous to making buildings. Making software is analogous to designing buildings. (You'll notice that the Design Patterns movement is based on a technique for architects, not builders.)
(And, by the way, if you think real-world construction projects follow a simple waterfall model like that, you should read about the Panama Canal [powells.com].)
What makes you think that if you design the hell out of it up front and build strictly to that design you won't find, six months or a year later when the project's finally finished, that you'll have built it wrong anyway? Or worse, what happens when halfway through you realize that your design was wrong, or your requirements were inaccurate or inadequate -- and you're locked into a process that requires a ream of up-front paperwork before you can change what you're building?
Again, coding is a design task. Everything else is just requirements gathering.
I think you've missed the point of XP's approach to unit testing. The unit tests aren't written to "check the code" -- I agree, it's pretty pointless for someone to write a test that proves that his code does exactly what he coded it to do. The unit tests are written to describe what the code is supposed to do -- they're like a design document that can automatically validate the code that implements the design.
Also, pair programming -- even when it's not between "two people of equal ability", so long as they both have enough ability and they're communicating well -- goes a long way toward alleviating the problem of having the watchmen watch themselves.
Don't knock XP if you haven't tried it.
Eclipse. (Score:2, Informative)
Seamless integration with CVS, automatic refactoring tools, incremental compilation, full integration of unit testing, plug-ins, etc, etc... you owe it to yourself to try this fantastic open-source tool. /t
It's all about the process (Score:3, Informative)
Re:still no silver bullets (Score:4, Informative)
It is not so much as pouring out millions of lines of code, but how useful those millions of lines of code are. Best example is the Air Traffic Control system which took lot of time and money. But then again wasit useful? The methodologies goal is not the amount of code written but the quality of software produced. But even though organizations (claim to) follow those methodologies strictly, in 99% of cases they have not.
But then one might think, so if we follow as preached do we get "finished" product? That is where the concept of "No Silver Bullet" comes in. In my research I have found that people are so much consumed in methodologies and their hype, they forget something important - "Common sense" . There are tons of examples of actual projects where developers opted to follow what is preached over their better judgement. Why? One simple reason you do not have to answer to anybody if you follow those "Best Practices". Hey after all that practice was best.
Then what is the reason that they fail time and again. When I spoke to a few colleagues about this, they immediately say "Hey those guys were incompetent". Maybe but not all. The problem is with the terminology itself. Consider the term "Best Practice". The biggest flaw with this term is that we forget that it worked in a certain situation with totally different requirements. It might work the next time in a similar situation but it is not guaranteed, because the situation is different, the environment is different and requirements are different. We should understand that the term 'best practices', does not represent the only 'right way'. They should not be blindly followed, instead adapted to each situation. If we rephrase the term as "Best practices for me, your mileage may vary" we can see them in a whole different light.
As far as I think, doesn't matter if you are CMM level 5 certified. If you lack common sense, you can never produce good software.
Re:Java and GPL - slower software, less incentives (Score:1, Informative)
Re:Cheap, Good, Fast - Pick 2 (Score:3, Informative)
How often have you seen job ads or RfR's stating "C++ programmers, 3 years experience"? It's bollocks. Any C++ programmer, no matter how bad he is at his job, will one day be a "C++ programmer with 3 years of experience". Yet, companies, project managers and HR people often fail to discriminate between the good and the bad programmers.... while there is no other field of endeavour where the difference in productivity between good and bad workers is greater. Metrics suggest a factor 10 to 100. Incidently: this is bad news for the good programmers amongst us. If no one discriminates between the good and the bad, you'll get the same pay as your lesser brethren.
Finding good programmers instead of hiring more to do a rush job, is the way to go, since the one programmer will not have the communication overhead that the "equivalent" team of ten average programmers has. We have done this a few times and found that Good and Fast go together well.
Re:Absolutely! (Score:2, Informative)
It does all that plus a bag of chips!
Seriously, it works well for an IDE. Give it a look. The only thing I don't like is the GTK version doesn't include printing.
Re:Functional languages (Score:5, Informative)
ML is a functionnal, statically typed language. Lisp is dynamically typed, which means type is attached to values instead of to variables. The improvments in both cases over C might be that the language offer more support to the programmer. Of course it always depends on the task at hand: you would not program an OS in Lisp or ML, but a lot of other interesting things could reasonably be done in Lisp.
Take for example Web applications: with more powerful languages that allows you to manipulate continuations you can do really impressive stuff. Sometimes I'm laughing when I see how the 'industry' falls behind in term of cutting-edge technology. I know that what I use will be discovered in a few years (probably under some other name), and becomes the Big Thing.
What is sad in computer science is that every now and then something old reappears and is considered the new Big Thing: think Java (which is a dumb OO language), or think XML (which is just a painful syntax for S-expr).
Also, forget about proving correctness for practical software programs. They will always get too big to allow you to prove correctness. But if you really want to be able to prove 'partial' correctness, then yes: Functional Languages will be a *much* better choice than C, since you can abstract state from your tests. But the thing is that sometimes it is very convenient to use side effects, for such mundane tasks as outputing text for example
Purely functional languages might be scorned by some as toys, but they have their use in research. Hybrid languages like Common Lisp are somewhat more practical.
As for hugeness of source code, I see 2 reasons Lisp would help overcome the problem. First, you can write less code that do more (reducing the source volume). Secondly, Lisp allows for interactive development. You don't have to recompile anything if you change your application. Just reload the function you just changed, and there: everything is in place. Some bug? Correct it, resume execution. Want to try something? Try it, look at how your app is responding. The ability to change a program while it is running is really empowering for the programmer.
But anyway, I'm not saying anything new in here, it's just that people wont get it, or that they'll forget. But hey, should I care if my competitors still use half-assed languages and keep hiring 'Java-style' programmers?
Re:Functional languages (Score:2, Informative)
In my (limited) experience with ML, it makes it very easy to code. The code is dense, but readable. A small number of lines can get a lot of work done. And what's better - you'll never get a segmentation fault unlike C. If you can get your program past the type checker, then it's pretty much going to run. And if you've handled all your exceptions and made sure that there's no unmatched patterns, your program will basically always run. It's really a pleasure to code in - far more so than I've experienced with C.
Objective-C + OpenStep APIs (Score:3, Informative)
Couple that with the open-source API SPECs (oppose "implementation"), and you have an amazing combinaison.
On the Mac, this is now known as Cocoa. Cocoa is an object framework that's now mostly accessible via the Java programming language.
For Linux (x86, but soon PPC as well), thetre's AfterStep [afterstep.org], an open-source implementation of everything that had made the NeXT a NeXT, including the dev environment.
If you have access to a Mac, get yourself some tutorial and explore Cocoa programming. If you only have access to a Linux box, get yourself a complete install and explore what you can do with this.
No amount of description actually gives any justice to how amazing this dev environment actually is.
If you're tempted to explore further, o'Reiley has a couple of Cocoa books, but the very best Cocoa books out there is written by Skott Anguish and al, and is called Cocoa Programming [cocoaprogramming.net].
Looks upon the steel book gathering dust and weeps (Score:3, Informative)
The problem is that the big players priced the Compiler based Common lisp development frameworks right out of the reach of most business developers. They also failed to take full advantage in the increasing capability of the PC, turning their noses up at in favour of past glories of the Symbolics machines. A good Lisp compiler could factor out the "cons", producing code sometimes surpassing the performance of C++.
The void was filled mostly by C++ and now Java, lesser beasts dispite their current quality standard libraries, overly verbose and fragmented incomparison to Common Lisp elegance.
Today there are open implementation of the lisp compilers [cons.org], but they still lack a comparable development enviroment to the commercial varients. Sadly, since I started professionally in 1988, I have not had one job or contract were they would consider the adoption of Common lisp, and I have never programed professionally in it. David Betz's XLisp [mv.com], and later XScheme was the closest I came to using it at work for scripting, and at home for some early AI-planning system hacking that has yet to see the light of day.
In my opinion, IBM's Eclipse IDE [eclipse.org] has finally comming close to surpassing the old benchmark of the commercial Lisp IDEs, it even has the ability to plug in refactoring tools. But then I can remember when, thanks to the "cons"-ed Lisp, manupulating the source of the program was as easy as mainpulating the data,
Re:The skunk works was a hack shop (Score:1, Informative)
And no, it's not "sure to blow up on you". Before they were decommissioned, I got a chance to be as close to an SR-71 as a civilian ever gets. It was sitting in the hangar, leaking fuel. One of the maintenance guys thought it would be fun to get a rise out of us and took his lighter and held it down against the fuel on the ground. No big kaboom
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:1, Informative)
Maintenance costs were one of Kelly Johnson's big pet peeves; he had some good ideas about maintenance, and he was right. For example, why do military personnell service aircraft? They're rotated every couple of years, so there's a constant retraining cycle (read: expensive). Instead, why not contract with the OEM or a third party to do maintenance? Much lower turnover, more experience, and so on.
Re:Best improvements (Score:1, Informative)
I'm unconvinced of your arguments on codegen. The really sad reality is that with gcc-3.2.1, gcc has parity with most expensive, proprietary, we-know-this-architecture-best-becasuse-we-made-i
Icc 6.0? MSVC++? It amazes me that they can't pull anything better out of the bag. Yo, gcc is FREE. These jokers should be able to pull 50% improvements out of codegen, yet the most I've seen is about 5-10% on benchmark codes. I've yet to see codegen differences in "Real World Code."
Sparc tools? Hello, are you for real? Spend a grand a seat for an ancient C++ compiler, a more ancient C++ runtime, and bullshit support? I don't think so. I've not seen codegen improvements over 10% on sparc and sparc64 for C/C++. Put up or shut up: what are you measuring (fortran)?
Have you ever tried to use MSVC++ embedded tools?
HP tools (aCC), IBM tools (xlC), etc? ICK!!!!
If you had, you'd realize why people use gcc, even when they get a slight performance hit. The larger codebase makes the compiler way, way more solid.
To tell you the truth, the user community for gcc is pretty amazing, and far more helpful that I've seen from compiler vendors who are not free-software based.
I'm not that old! (Score:2, Informative)
Re:Java and GPL - slower software, less incentives (Score:2, Informative)
Lets not confuse Java the language and its performance with Java GUI framework. Just like you would not compare C++ the language with the Microsoft windows framework.
The Java framework sacrifices some speed for generality. So see what you save writing it once and delivering it to multiple platforms vs re-writting the GUI portion for another platform. There is method and cost to this madness.
Re:Looks upon the steel book gathering dust and we (Score:1, Informative)
There are plenty of Common Lisp tools (IDEs, web servers, utilities, etc.) available at reasonable prices or free: Allegro Common Lisp, Corman Lisp, CLISP is free...
Go read the Lisp Tools [alu.org]
Everything you need is available. Get a Lisp today!
Eiffel (Score:1, Informative)