Has Software Development Improved? 848
earnest_deyoung asks: "Twenty-five years ago Frederick Brooks laid out a vision of the future of software engineering in "No Silver Bullet." At the time he thought improvements in the process of software creation were most likely to come from object-oriented programming, of-the-shelf components, rapid prototyping, and cultivation of truly great designers. I've found postings on /. where people tout all sorts of design tools, from languages like Java, Perl, Python, Ruby, and Smalltalk to design aids and processes like UML and eXtreme Programming. I'm in a Computer Science degree program, and I keep wondering what "improvements" over the last quarter century have actually brought progress to the key issue: more quickly and more inexpensively developing software that's more reliable?"
Cheap, Good, Fast - Pick 2 (Score:3, Insightful)
Logic requires careful thought, and careful thought requires time.
Re:Cheap, Good, Fast - Pick 2 (Score:5, Insightful)
The answer is finish that degree and hope your institution teaches you enough about these principles: effective design, KISS, machine states, and proper error handling.
After a few years in the field I've found that these, paired with knowledge of a language and it's libraries is as close as you are going to get (although I'm still working on perfecting the second one)
Cheap, Good, Fast - Take all 3, if you're good (Score:5, Insightful)
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:4, Interesting)
On the other hand, it's _probably_ as cheap as good and fast is gonna get.
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:4, Insightful)
It is actually cheap compared to the usual practices especially in big companies (hiring 100 cretinoids to midlessly click and drag). The problem is that such teams are not a commodity readily available on the market. You cannot just go out and buy one. And they are hard to manage so the average PHB prefers the monkeys
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:5, Insightful)
It is actually cheap compared to the usual practices especially in big companies (hiring 100 cretinoids to midlessly click and drag). The problem is that such teams are not a commodity readily available on the market. You cannot just go out and buy one. And they are hard to manage so the average PHB prefers the monkeys
This hits on a very critical point - talented people who get the job done are the solution, but they aren't always easy to find and they aren't always what PHB's expect.
Finding talented people requires one be able to recognize the talent, be willing to pay for it, and be willing to use it properly. That's a tall order for many people. You can do it, but you have to really understand what you're doing.
Secondly, very talented people require proper management - that's sometimes at odds with common managerial philosophy. If you have someone who is good at what they do, micromanagement, not giving them proper resources, etc. can minimize the impact of the talent. Not understanding personality quirks of certain talented populations can be disastrous.
So, people go with what they know, even if 100 codemonkeys are hired as opposed to 50 talented people, even though the 50 talented people may save you 25% of your budget.
I'm fortunate. Where I'm consulting now is a place where my manager is an IT guy, knows how people work, and lets us to our job as long as we file progress reports. When he sees a talent/skill, he maximizes it. He talks to people as people.
I got lucky.
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:4, Insightful)
***
Actually, often 3 great developers can do the work of 30 codemonkeys. It really is that much of a distinction.
The problem with the advances in computing tools is that people have become dependent on the tools, and have no understanding of the underlying technology. Then, when they get stuck, they can spend weeks or months trying to figure it out, while it only took the good programmer about 30 minutes on Google, because he knew what to look for.
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:4, Insightful)
Which is definitely part of the problem.
Software Companies do NOT neccessarily exist to produce good tools and products. They exist to make money and please the shareholders.
This does NOT mean they are out to produce the best, they're out to sell. Hopefully it is the best, but . . . well, no more needs to be said.
That clouds the initial question on the possibility of Silver Bullets. Even if they're out there, we've got to wade through tons of irrelevant stuff to find our particular Silver Bullet.
And a lot of our vendors are NOT helping. That's one reason I like open source. People may participate for different reasons, many far from noble, but the product and usefulness are a major focus.
Re:Cheap, Good, Fast - Take all 3, if you're good (Score:3, Funny)
Yeah, but an advanced tactical fighter aircraft is utterly trivial compared to the complexity of a software system.
not a money pit... (Score:4, Interesting)
Johnson and Rich were also proponents of having Lockheed engineers maintain their aircraft instead of having military personnel do it. Using Lockheed engineers would save a ton of money because they aren't rotated and therefore don't need to be retrained. Sure, Lockheed would get some money out of the deal, but not as much as the gov't would save.
The Skunk Works wasn't perfect, but overall they were pretty good with money under Kelly and Ben.
Re:Cheap, Good, Fast - Pick 2 (Score:3, Informative)
How often have you seen job ads or RfR's stating "C++ programmers, 3 years experience"? It's bollocks. Any C++ programmer, no matter how bad he is at his job, will one day be a "C++ programmer with 3 years of experience". Yet, companies, project managers and HR people often fail to discriminate between the good and the bad programmers.... while there is no other field of endeavour where the difference in productivity between good and bad workers is greater. Metrics suggest a factor 10 to 100. Incidently: this is bad news for the good programmers amongst us. If no one discriminates between the good and the bad, you'll get the same pay as your lesser brethren.
Finding good programmers instead of hiring more to do a rush job, is the way to go, since the one programmer will not have the communication overhead that the "equivalent" team of ten average programmers has. We have done this a few times and found that Good and Fast go together well.
Re:Cheap, Good, Fast - Pick 2 (Score:5, Insightful)
I often see authors in the ACM and IEEE journals tearing their hair out worrying if software engineering is finally going to come to a magical "maturity" and if professional software engineers are really putting theory into practice, etc, etc. It is my feeling that, in general, professional software engineers are applying good engineering practice when they can and try to at least be conciencious when they cannot. So what is it that is stunting the progress toward software engineering's maturity?
Engineers putting theory into practice are only part of the equation. Engineers must work under management and it is my belief that the management of software engineering is in a less mature state than software engineering itself, at least in practice. As such, I believe managers of software projects often severly underestimate how long software projects should take, which leads to time budgets that are unrealistic given the requirements for a project.
There is often a comparison drawn between house building and software. The question: "Would you live in a house built with the same reliability as software?" is often bandied about. I think such a question misses the point. I think the more accurate question is: "Would you live in a house where the architect and structural engineer had two days to make the plans and the builders had 2 weeks to build it?" Of course you wouldn't. You would expect the architect, engineer, and builders to have enough time to build the house safely. That doesn't mean giving them infinite time but it means giving them enough time to do the job properly.
The sad thing is that many of the managerial problems that inhibit software engineering from progressing have been known for decades. Unfortunately, this information seems to have failed to penetrate into management circles or the information has fallen on deaf ears.
We know what the problems are, but some of the solutions are not in our control as engineers.
Changes in software management must take place in order for discoveries in software engineering to bear fruit.
Re:Cheap, Good, Fast - Pick 2 (Score:4, Funny)
Re:Cheap, Good, Fast - Pick 2 (Score:4, Insightful)
Management is part of the discipline of Software Engineering. You will find extensive discussions of process and management in any book on the subject. There is a significant amount of theory, literature and expiertise on the subjects of time and complexity estimation, resource requirements, managing change, measurement, quality testing and reliability.
What is not covered is the management of the business pressure, which requires that you use less people, cheaper tools, and deliver sooner. Because everyone knows that its easier to build software than to build a bridge (or building). Or more specifically because fewer people die from faulty software and its easier to slap on a disclaimer because, given the track record, people don't expect software not to collapse.
The answer is that Software Engineering needs to become a recognised professional qualification, and accredited engineers must adhere to the ethics and codes of conduct which govern other engineering professions. When your boss tells you to do it on the cheap, you have an obligation to say "No", and a body of professionals to back you up.
Aside: if you want to refer to yourself as a Software Engineer, please have some familiarity with the differences between that role and a developer. The SWEBOK [swebok.org] is a good place to start.
Re:Cheap, Good, Fast - Pick 2 (Score:4, Insightful)
I totally agree with this. If I've learned nothing else in the last 15 years, it's that the most significant amount of time in any project should be spent before any development begins. Establishing functional and technical specifications, and choosing the appropriate platform (both hardware and software) are all more important than the development itself. The only way to improve efficiency here is to make sure that the "users" who are requesting the development have a firm understanding of how to build their requirements while keeping in mind potential limitations of various potential platforms. Although the numbers may vary depending on who you talk to, I generally feel that 60% of any project should be spent on capturing requirements and making the basic design, 10% should be spent on implemenation, 20% should be spent on testing, and a final 10% should be spent on documentation. Sadly, the final 30% spent on testing and documentation, while probably having the most potential to save the company money in the long run, is usually left unfinished in order to keep the project within time and budget constraints.
Having said all of that, I do believe that the advent of rapid prototyping tools has actually helped for the requirements phase. If a picture is worth a thousand words, a demonstration is probably worth 10,000. Providing a "mock-up" of potential functionality truly helps users to make decisions on what they *really* want, instead of what they think they want.
Re:Programmers *are* the problem (Score:3, Interesting)
Re:Java and GPL - slower software, less incentives (Score:3, Insightful)
Open source and collaboration (Score:3, Interesting)
Re:Open source and collaboration (Score:3, Insightful)
I must say (regrettably) that open source does not mean more reliable and better code automatically. The major problem is the lack of leadership, for example a plan of features that is followed and proper design of the GUI. As features are added from multiple sources both the GUI and the code can easily get bloated.
As it is hard to control what features are added, and to get good support I doubt that open source software will take over most large consumer base markets anytime soon.
I'm not saying that there isn't possible (look at Linux) but I doubt it happening quickly.
Functional languages (Score:5, Interesting)
Re:Functional languages (Score:5, Informative)
ML is a functionnal, statically typed language. Lisp is dynamically typed, which means type is attached to values instead of to variables. The improvments in both cases over C might be that the language offer more support to the programmer. Of course it always depends on the task at hand: you would not program an OS in Lisp or ML, but a lot of other interesting things could reasonably be done in Lisp.
Take for example Web applications: with more powerful languages that allows you to manipulate continuations you can do really impressive stuff. Sometimes I'm laughing when I see how the 'industry' falls behind in term of cutting-edge technology. I know that what I use will be discovered in a few years (probably under some other name), and becomes the Big Thing.
What is sad in computer science is that every now and then something old reappears and is considered the new Big Thing: think Java (which is a dumb OO language), or think XML (which is just a painful syntax for S-expr).
Also, forget about proving correctness for practical software programs. They will always get too big to allow you to prove correctness. But if you really want to be able to prove 'partial' correctness, then yes: Functional Languages will be a *much* better choice than C, since you can abstract state from your tests. But the thing is that sometimes it is very convenient to use side effects, for such mundane tasks as outputing text for example
Purely functional languages might be scorned by some as toys, but they have their use in research. Hybrid languages like Common Lisp are somewhat more practical.
As for hugeness of source code, I see 2 reasons Lisp would help overcome the problem. First, you can write less code that do more (reducing the source volume). Secondly, Lisp allows for interactive development. You don't have to recompile anything if you change your application. Just reload the function you just changed, and there: everything is in place. Some bug? Correct it, resume execution. Want to try something? Try it, look at how your app is responding. The ability to change a program while it is running is really empowering for the programmer.
But anyway, I'm not saying anything new in here, it's just that people wont get it, or that they'll forget. But hey, should I care if my competitors still use half-assed languages and keep hiring 'Java-style' programmers?
Thinking about thinking.. (Score:5, Informative)
CVS / RCS (Score:5, Insightful)
I seriously doubt that a program like Linux could flourish without programs like CVS.
furthermore - many of the programs that do this sort of thing can be used for any programming language... you could even use it for simple documents.
Re:CVS / RCS - the next step? (Score:5, Insightful)
You don't need a new version control tool, you need a refactoring tool.
Progress Balanced By More Demands on Developers (Score:5, Interesting)
However, I think they're equally balanced out by huger demands on programmers. Once it's realized that a programmer can do 2, 3, or 10 times as much work by using more efficient methods management is quick to pile on 2, 3, or 10 times as much work!
This isn't really unique to programming either. I think it's universally applicable to any area where technology permits greater productivity.
For example, look at all those ads from the 50's. Things like the microwave, the vaccuum, and the dishwasher were supposed to usher in a new era of leisure. Do we have more leisure? No, we have less, as those luxuries become necessities and we cram in more activities in out new-found time in order to stay competitive.
Re:Progress Balanced By More Demands on Developers (Score:5, Interesting)
Today we have JDEdwards OneWorld. Written in C++ and other state-of-the-art tools. The distribution runs about 10 GB, with a working set for development purposes of 1.5 GB. Its function? To provide accounting and manufacturing management support for manufacturing companies. How well does it do? Is it any better than ManMan? I will leave that for you to decide. But hey - you can cut-and-paste directly from OneWorld to Excel. That's a gain I guess.
sPh
Re:Progress Balanced By More Demands on Developers (Score:3, Insightful)
While I no doubt respect ManMan (especially with a kick-ass moniker like that), I could easily see ManMan looking on enviously at OneWorld's application and system interop features. Try RPC on ManMan..
Re:Progress Balanced By More Demands on Developers (Score:3, Insightful)
What are the training costs for the two? Is OneWorld significantly easier to learn and use? Does it interoperate with more 3rd party programs? Is it more friendly with respect to data input and output?
Odds are good that ManMan is actually more efficient for a trained operator - but the cost of getting that trained operator is relatively high. On the other hand, you can plop down half a dozen monkeys in front of OneWorld and get results.
True or not? If not, then I definitely have to wonder what advantages OneWorld actually presents over ManMan, other than support and maintainability (which, without a doubt, are huge requirements in the business world).
a few thoughts.... (Score:3, Insightful)
But seriously, a lot of people develop beneath the "enterprise level" and some of the buzzword concepts just don't scale that well for smaller project.
In my opinion, the two things that have really made a difference are databases (as opposed to manually creating file formats) and the object-oriented paradigm.
My best advice is to use everything with a grain of salt because there is always something "better" on the horizon.
more quickly and more inexpensively? (Score:5, Insightful)
Based on the last 20 years either working in or supporting government efforts, I'd say yes. However (there is always a however), it depends on the sophistication of the developing organization. The cowboy shops still suck. Those who have embraced more formal processes have become more reliable. It is a 2-way street though and requires the customer to be more sophisticated as well. It doesn't do a damn thing to have a development shop at CMM-5 if the client doesn't understand the need for process and doesn't understand the software development challenges.
Re:more quickly and more inexpensively? (Score:5, Insightful)
You want to know what we've really discovered about software development in 25 years? The same thing we know about booze and marijuana: spend your money on high quality, b/c the increase in quality outpaces the increase in expense.
CMM-5 and crap like that are amusing diversions for middle management types, but keep in mind that middle management is the art of creating an organizational structure that is more important than the people who populate it. It is a recipe for reliable mediocrity, like bread from a factory.
Eventually, it will be realized that top technical personnel are like good lawyers (and not a thing like wonderbread): essential to an organization over a certain size, not readily "manageable" in the sense that the typical weasel would like, and not readily identifiable on the basis of objective criteria.
Or just keep hoping for a magic bullet that will allow a caveman to captain a starship, your choice.
Best programming tools ever... (Score:5, Insightful)
Don't know about improvements.... (Score:5, Funny)
Re:Don't know about improvements.... (Score:3, Insightful)
I'd say: Score 5: tragic.
But damn I agree. Perl zealot needs not apply: I've been there, I learned, I understood, I realized nothing good is coming from there.
CGI.pm --> the way to generate HTML documents? wtf, the function names are even more verbose than typing in tags directly. That's why people will always embed html in print statements in Perl. My eyes still hurts just remembering how f***ing painful Perl is sometimes.
Components and Containers (Score:3, Insightful)
still no silver bullets (Score:5, Insightful)
These technologies and methodologies have allowed us to keep pace with Moore's law. The amount of software we develop today would simply be impossible using the state of the art in 1970. We routinely poor out millions of lines of code in mammoth projects that take sever hundreds or thousands of man-years to complete. The only reason we are able to do so is because of improvements in development methodologies and technology.
The (huge) research field that studies these technologies and approaches is called software engineering.
Re:still no silver bullets (Score:4, Informative)
It is not so much as pouring out millions of lines of code, but how useful those millions of lines of code are. Best example is the Air Traffic Control system which took lot of time and money. But then again wasit useful? The methodologies goal is not the amount of code written but the quality of software produced. But even though organizations (claim to) follow those methodologies strictly, in 99% of cases they have not.
But then one might think, so if we follow as preached do we get "finished" product? That is where the concept of "No Silver Bullet" comes in. In my research I have found that people are so much consumed in methodologies and their hype, they forget something important - "Common sense" . There are tons of examples of actual projects where developers opted to follow what is preached over their better judgement. Why? One simple reason you do not have to answer to anybody if you follow those "Best Practices". Hey after all that practice was best.
Then what is the reason that they fail time and again. When I spoke to a few colleagues about this, they immediately say "Hey those guys were incompetent". Maybe but not all. The problem is with the terminology itself. Consider the term "Best Practice". The biggest flaw with this term is that we forget that it worked in a certain situation with totally different requirements. It might work the next time in a similar situation but it is not guaranteed, because the situation is different, the environment is different and requirements are different. We should understand that the term 'best practices', does not represent the only 'right way'. They should not be blindly followed, instead adapted to each situation. If we rephrase the term as "Best practices for me, your mileage may vary" we can see them in a whole different light.
As far as I think, doesn't matter if you are CMM level 5 certified. If you lack common sense, you can never produce good software.
Best improvements (Score:5, Insightful)
Now you wouldn't think of developing on UNIX with anything but GCC and the associated build tools.
In 1990 you were stuck with whatever C compiler your vendor shipped, and there were more than a few dodgy compilers out there. Modern compilers with useful error messages have done more than anything else to make debugging and development faster and easier for me.
Re:Best improvements (Score:4, Interesting)
Actually a lot of people would. The quality of code generated by the SUNpro and MIPSpro compilers on SPARC and MIPS processors respectively leaves GCC in the dust. GCC really only comes into its own on x86, because Linux (or *BSD) on x86 is the platform that it gets used most on. GCC is portable, yes, but it isn't built for compiling high-performance code. So you need to ask yourself whether getting binaries that execute 2x as fast is worth using a slightly less well known compiler for.
I can say this.... (Score:5, Insightful)
The focus has definitely shifted away from algorithms and toward abstraction. This was supposed to make things easier, by letting the software do what it does best and keep track of bookkeeping, while we concentrate on building models and governing interations between them.
Some of it actually makes sense - the object oriented paridigm, component models, virtual machines. (VM's, by the way, go back at least 20 years in the literature -- I studied them in college in the late 80's. However, like Pascal, they were originally considered as an instructional tool, and nobody at the time thought that anyone would be damn fool enough to actually try and *implement* such a thing!)
But just like letting grade-school students use calculators for their arithmetic, I'm not certain these things are best for students. Sure, you get usable code out quickly, but without an understanding of the underlying algorithms and logic. I doubt many modern 'c0derz' could properly knock out a simple quick-sort, let alone a fully ACID SQL DBMS.
Away from algorithms? (Score:3, Interesting)
Re:Away from algorithms? (Score:4, Interesting)
From one point of view, only a tiny minority of today's programmers ever need to create "new algorithms"- so much documented study has gone on in that area, you'd only be reinventing the same wheels.
That can be a sign of maturity, that the field has evolved to the point where specialists can go their thing, and not force everyone else to understand the trickier aspects.
Looking at your examples- data converters particualrly- I don't see much room for new algorithms. From a restrictive, Computer Science standpoint, nearly everything you do will be isomorphic with a known existing algorithm (modulo "trivial", "cosmetic" variations, of course)
Naturally, people who don't understand algorithms will have difficulty selecting the ones they should re-use, and are at risk of using them wrongly. But that's a consequence of newer, user-friendly development tools- with a lower barrier to entry, less competent persons can enter the profession, and still muddle by.
Re:I can say this.... (Score:3)
Pretty much every programming concept/innovation today has roots that go back. OO is at least 20yo. VM were implemented, it was just that mainframes were the only things beefy enough to implement them effectively, not that they were overtly complex. The Java vm is not too technically different that the old pcode interpreters, again, nothing new here.
I doubt many modern 'c0derz' could properly knock out a simple quick-sort, let alone a fully ACID SQL DBMS.
Right but the problem is that for some strange reason, 20 years later, a coder still does have to know how to knock out a quick sort (or some such). We won't see true progress until we get to the point where your "average" coder absolutely will never need to know this, unlike the hodgepodge of high level/low level stuff we have now.
eXtreme Programming == NO (Score:4, Interesting)
1) Write your test cases up front... this is ages old. XP isn't as rigourous as others who say "and make sure other people write them".
2) Pair Programming, works for two people of equal ability. The two headed surgical team from the Mythical man month is a much more effective way of using two heads.
Basically things like XP sum up how long computing has to go to become an engineering discipline. In every other engineering subject there are critical elements:
Requirements
Design
Testing and approval of design
implementation
testing of implementation (throughout implementation)
Delivery.
Maintainance
For a construction project all of these elements are mapped out well in advance, which is why the construction industry can work on lower margins.
To become better requires not a "Silver Bullet" as Brookes says, the technology won't make the improvement. Its actually about people applying the rules _rather_ than looking for the Silver Bullet. Some projects succeed, others fail, there are reasons for the failures and the successes. But rarely do we learn from either.
XP is the embodyment of the non-engineering approach to computing that pervades this marketplace. The idea that you can build it wrong and change, don't design "code and check", have a unit test written by a bad coder to check his own bad code.
Brookes is right. At the end of the day computing success is down to a realisation of the soft-skills allied to technical talent.
If you have 10 brilliant people leading 100 average people... fire the 100 and support the 10 to do the delivery effectively. Make sure they follow a process, and make sure that the requirements are defined and change as little as possible. Make sure designs are verified, make sure code is reviewed.
Sure its less exciting that "just coding" but in the end it takes less time, costs less to maintain and delivers what the customer wants.
Engineering is a discipline, XP is just glorified hacking, only by becomming disciplined will software improve.
Re:eXtreme Programming == NO (Score:4, Insightful)
Just to nitpick on this particular myth. In construction there is the idea of "fast-path" construction, where building starts before the design is done.
The idea that all requirements and design are done before construction starts is a myth.
People hack buildings too. There were some famous cases of building "bugs" (eg. CityCorp skyscrapper wasn't stiff enough) or famous failures where design or implementation errors caused a building collapse.
Read "To Engineer is Human" and "Design Paradigms" by Henry Petroski for a start.
eXtreme Programming == Try it first (Score:4, Informative)
This is where so many people get it wrong. Making software is not analogous to making buildings. Making software is analogous to designing buildings. (You'll notice that the Design Patterns movement is based on a technique for architects, not builders.)
(And, by the way, if you think real-world construction projects follow a simple waterfall model like that, you should read about the Panama Canal [powells.com].)
What makes you think that if you design the hell out of it up front and build strictly to that design you won't find, six months or a year later when the project's finally finished, that you'll have built it wrong anyway? Or worse, what happens when halfway through you realize that your design was wrong, or your requirements were inaccurate or inadequate -- and you're locked into a process that requires a ream of up-front paperwork before you can change what you're building?
Again, coding is a design task. Everything else is just requirements gathering.
I think you've missed the point of XP's approach to unit testing. The unit tests aren't written to "check the code" -- I agree, it's pretty pointless for someone to write a test that proves that his code does exactly what he coded it to do. The unit tests are written to describe what the code is supposed to do -- they're like a design document that can automatically validate the code that implements the design.
Also, pair programming -- even when it's not between "two people of equal ability", so long as they both have enough ability and they're communicating well -- goes a long way toward alleviating the problem of having the watchmen watch themselves.
Don't knock XP if you haven't tried it.
faster and less reliable... (Score:3, Insightful)
They code crap. My personal experience with not to big software company's is they like to sell crap if they can and most of the time the other company also sells crap and after wards you dont even own the software you buy. Software bussness sucks atm.
Until the world wakes up and start demanding that software needs to work just like material stuff or else they wont buy it. It wont get better in the near future aslong we except crap from even big company's like Microsoft, which is becoming more reliable and stable but still sells a awful design if you ask me. oke maybe start a flame here but most software is like throw a way dishes instead of a rolce roys. And i think company's like microsoft showed the smaller software houses that its oke to sell crap as long it sells and it bugs me cause software has a bad name, but i know good software exists, bla bla bla, you got me pissed again slashdot....i hate humans
Quazion.
Software improvement in the last 25 years? (Score:4, Interesting)
Of course it is now easier to create software than before.
First of all, source management software wasnt available 25 years ago. Try creating a huge piece of software without any way to rollback changes, share the same source tree with other developpers, etc... (cvs/sourcesafe/starteam/etc)
Second, profiling tools. Hey, you want to know where that memory leak is? Where that CPU bottleneck is? Pretty hard to do when you were coding in cobol many years ago... Doing the same is way easier now with OptimizeIt and stuff like that.
I could go on and on but I must leave for work =)
One day... (Score:5, Interesting)
I've always believed that one day some bright college kid is going to come up with a completely different style of computer language and interface, and when that happens we will all slap our heads and go "D'oh! So that's how we should be doing it! Obvious!"
Like the web and P2P, the most influential ideas are often quite simple, and "obvious".
Absolutely! (Score:4, Informative)
If Perl had an IDE that was as easy to use it would dominate the world. (more than it already does)
Re:Absolutely! (Score:3, Interesting)
There are many. Others have mentioned Komodo [activestate.com] and the Visual Perl plugin [activestate.com] for Visual Studio.NET.
There's also Perl Builder [solutionsoft.com], which people rave about but I have not tried. They claim to be the most popular Perl IDE.
I would also like to add that an Open Source one, Open Perl IDE [sourceforge.net] is decent. I use it at home.
I use Komodo at work (because it can act as an IDE for other languages like XSLT) and really like the Perl and Regular Expression debuggers.
-bp
So, they've caught up to... (Score:5, Funny)
Yes, this is flamebait. Yes, I'm bitter and curmudgeonly. Perceptive of you to notice...
It improved sneakily (Score:5, Insightful)
I have to say (and I know I am preaching to the choir here) that for me the biggest improvement is really Open-Source and how it allows me to re-use existing libraries.
I do most of my work in Perl and the fact that I can call any of the thousands of modules of CPAN [cpan.org] makes all the difference in the World. When I worked in C in the late-80s we could either buy libraries, and have to go through the vendor to maintain them, or build our own, expensive and buggy toolkit. Nowadays I can build really powerful code in a few hours, using the combined efforts of dozens of authors and thousands of constibutors in the form of a few Perl modules.
For me that's the main improvement: instead of being part of a team of 5 people writing mostly memory management code I can now simply focus on the task and do it myself. Quicker, without weekly meeting, and having fun in the process.
Has Software Developement REALLY improved? (Score:5, Insightful)
NO.
The project management keeps moving the target. The customer still says "I don't know what I really want, but keep coding and I'll let you know when you get it." Analysts can't analyze, designers can't design, coders can't code, testers don't test (enough), ad nauseam.
Methodologies and philosophies of software development can only make up for so much. Sometimes, they are counterproductive. They "lower the bar", leading management to believe that the methodology can fill in for cheaping-out on hiring good developers. But we /.ers know the truth, don't we: Quality software comes only from quality developers--people, not methods or schools of thought or books by really smart people. Since quality developers are rare (like Leonardo DaVinci rare), quality software is correspondingly rare.
Here's a thought... (Score:3, Interesting)
perl is ugly to code in, and perl OOP is obviously a hack. I had a graphics/OOP professor who said, "nobody likes to program in perl, but it gets the job done." Obviously he lives in the land of language theory, where perl doesn't, but it gets the idea across...
perl gets the job done because of its massive collection of components.
I think I'd go further to say that the big improvement there is in repositories where you can get massive collections of components, as there have been languages like perl (in terms of having lots of stuff - PL/1 comes to mind) in the past.
Places like CPAN, Sourceforge and Freshmeat really make the difference. So ultimately, the internet is the means through which software development has sped up (at least when you're not talking about RAD-GUI development, which is another thing entirely).
Re:Here's a thought... (Score:3, Informative)
I've been spending a lot of time programming in PHP lately, and although my code might be more readable, (especially to someone with just a C programming background) I spend a lot more time working around quirks in PHP (either version issues, library functions, or design flaws... sometimes all three!) than I ever would in Perl.
However, the repositories do make all the difference. Although there isn't quite as much out there for PHP as there is for Perl, there's still a substantial amount, and that helps make it all bearable.
the proof is in the pudding (Score:4, Insightful)
Since the early days of computing in the late 70's, we've seen systems grow more larger and more complex. Whereas entire OSes used to be written by a single person (granted, a gifted person like the Woz or Bill Gates), these days we have companies like Sun and Microsoft with literally hundreds of developers and testers for a word processor, let alone the thousands of folks around the world who contribute to Linux, Apache, or KDE.
Given this incredible change in how software is developed, we'd expect to see systems collapse into instability and mayhem, but save for a few exceptions (Windows 9x, anyone?) this has largely not been the case. Windows XP, Mac OS X, and Linux 8.0 have proven, if anything, more stable and reliable than their predecessors. For an even more dramatic example, look at NASA's software or popular video games. It's clear that not only has software development expanded in scope exponentially, but it has become objectively better in the process. Development has never been better, and I see no reason why this trend shouldn't continue.
Do you like putting lipstick on a hippo? (Score:5, Insightful)
The big improvement in software productivity will come when we are able to successfully clone human beings. Corporations will create a stable of uber-programmers who they will then clone to be able to mass produce software. In the future, the open source war will be about peoples' genes, the software will be just a downstream product.
Plenty of Improvements - In a sea of dreck (Score:5, Insightful)
There are definite improvements to programming. Tools, concepts, etc. have evolved. There are no true silver bullets, but we've got a good selection of choices out there.
The problem, however, is threefold:
One of the real challenges for IT professionals today is to find the good tools and ideas out there, the ones that really have improved programming, and then actually get them into use. A good IT person is a good researcher and good at justifying their ideas to people.
No progress .... (Score:3, Insightful)
I've only been programming for 19 years (not 25) but I can say that I've seen absolutely no progress in software development givin your contraints of defining "progress" as being able to achieve the three goals (speed, cost, robustness). HOWEVER, I don't really blame the tools, rather the nature of the end result. The software we have to write has become significantly more complex, and at a rate that has surpassed our abilities to effectively create tools to deal with their complexity. "Back in the day" when a "good" business level word processor was 100KLines of code and written by a small group of dedicated and bright people, you could get very good results. Now, something like Word is significantly larger than 1MLines of code being worked on by programmers with 15+ years of experience to those who just got out of school to those who you wouldn't trust to add up your grocery bill let alone write a line of code.
It's like we still have hammers and nails, but the "market" wants us to build steel skyscrappers. So we come up with even better hammers and whiz bang processes to "better" utilize those hammers and nail together steel girders, but the fact is that those tools are woefully inadequate to truely make a dent in improving that which we build.
Net Access. (Score:5, Insightful)
Object Technology.. (Score:3, Interesting)
Design patterns also play an important role in allowing for a given design to be re-used, consumed, whatever the case may be. OOP related technologies such as UML, Corba, and now many XML based solutions are beginning to mature the field.
I am not as experienced as I would like to be with OOP, but I can say that I have been in the procedural world long enough to realize that there seems to be a divine power in OOP. It makes you *think* entirely different about problems, and by breaking a very large, very complex problem into approachable components. Allowing not only a single developer to build more complex systems, but systems that are well suited for re-use in future or existing systems.
Re:Object Technology.. (Score:3, Insightful)
I have over 30 years experience of embedded systems, and I would say that without exception the C ones were leaner, meaner and fitter than the C++ ones.
As for reusability, documentation is the key to reusability. It is conceivable that the syntax of some languages may help with this, but it seems there is more benefit in theory than in practice. All of us use the C standard library all the time (ie anyone using a browser is executing it) - now THAT is reusable. The Fortran library is widely reused, and no one ever accused Fortran of being object oriented. (I have heard Fortran users called "disoriented" :-)
The Silver Bullet (Score:3, Funny)
Available Tools and Libraries are a Lifesaver (Score:5, Insightful)
That's not the only one: The depth of CPAN, the Python library, even MFC/ATL are worlds ahead of hand-coding the guts of an app.
When I started programming, the Mac SDK had to be handled raw, and GDI was brand spankin' new. APIs such as DEC's FMS were considered heaven-sent(talk about dead ends!). Shortly after, class libraries were available, but expensive for an amateur. With Java and Perl, it's part of the (free) package.
I'm sure there's garbage code in those libraries -- there's always going to be bugs. But they're going to be relatively smooth to use, and they're already written, letting me focus on the business logic, instead of the presentation.
Metamodel creation and code generation (Score:3, Insightful)
Most documentation is horrid if it even exists (learn a human language first and use it to actually write meaningful comments, specifications, test scripts, internal and user documentation.)
Most of this industry doesn't know dick about SLAs or optimization for time (first) or space (last.)
Most of this industry doesn't know dick about configuration management, capacity planning or correctness.
The difference between duffers (most of this industry,) and the pros is that the pros don't "paint little master-pieces" in a a guild-like cottage industry. They generate "art by the yard" in industrial settings for mass dispersal, distribution and "invisible" use.
Good luck and remember, computing is nothing but a problem in N-Dimensional topology. If anybody tell you different, they are part of the problem, not part of the solution.
ALL objects have states and state transitions, code for that first and the rest will follow. Start from a thorough, correct and complete meta-model and you won't get into trouble later.
As for languages, CASE tools, GUIs, IDEs and the rest. Learn to do it the long and hard way first so you'll:
a) know what's being optimized and abstracted out,
b) appreciate it,
c) know what happens when it fails.
Re:Metamodel creation and code generation (Score:3, Insightful)
remember, computing is nothing but a problem in N-Dimensional topology. If anybody tell you different, they are part of the problem, not part of the solution
No. That's the problem solving part. The more important (and hard) part is defining the problem.
Observations (Score:4, Interesting)
What I have observed in the course of software development, in various companies, is that the management dictates that "thou shalt follow said process" and there will be the obligatory design reviews, spiffy diagrams, an all the huffing and puffing. But when the smoke clears, it still comes down to 9 months of work compressed into 3 months of actual work time, and everyone shifts into a hack 'n slash mode. The processes in place fail because of a lack or adequate time and inflexibility of deadlines.
Is it engineering, or not? (Score:5, Insightful)
Now I am not saying that you still don't need to have a good understanding of the language, or use good design, implementation, testing, etc. But I have worked at SEI CMM level 3, level 2, and am currently going through the process of evaluation where I work now, for level 2. But being at that level doesn't guarantee that your software will be good. It seems almost like we are attempting to fit the creation of software into the old engineering mold, because it kind of fits.
So to answer the question - I don't think that there have been any great improvements to obtain the goals you stated. Software relys on hardware, which seems to contstantly be changing. A bridge is a bridge, but the very idea of what software can do has been changing constantly over the last 25 years.
If you want reliability, look at what NASA has produced. Those are some engineers. Ahh, but you said 'quickly', didn't you? :-) If you want something rock solid, you can't have it tomorrow, if you want it to do anything remotely complex. I think one of the big things we as an industry has to realize is that our product (software) can be simple and fast to market, or more complex and take more time. And all of the variations in between. I haven't been convinced that what we do is engineering, but I haven't been convinced that it isn't. After all, it is a pretty young profession, compared to others. Imagine trying to explain to someone from 100 years ago what a software engineer does.
All that being said, I think that the obvious Slashdot answer is that the GPL and free software have been a huge force, but only in recent years. I think the two biggest forces in software development were the PC and the internet.
Re:Is it engineering, or not? (Score:5, Informative)
I disagree. That might be true in many cases, but some of the truly great engineering feats were produced quickly by a small team of very, very good people. A classic example is the Lockheed Skunk Works under Kelly Johnson. The SR-71 was conceived in the late 1950s, yet has performance numbers that are unequalled even today. Even more amazing, it was designed and built by just a handful of people. Compare that with the thousands of people it takes to build a new military aircraft these days. Other examples of the Skunk Works' prowess include the P-80 (USA's first operational jet fighter), the U-2 (high-altitude spy plane), F-104 (first mach 2+ jet fighter), etc.
The moral of the story is you can perform very high quality engineering in a very short time (the P-80 took something like 140 days to design and prototype!!) if you have an incredible team with no red tape in the way. Unfortunately, this is rare.
Standard libraries (Score:5, Insightful)
IMHO, one of the keys to writing good software is to not write very much software. Class libraries like the one in Java are almost invariably more feature-complete, more reliable, and better tested than roll-your-own libraries. Using existing libraries means you don't have to worry about off-by-one errors in your linked list implementation, or figuring out a good hash key for your string. While I think that all coders should know how to solve these problems, they shouldn't have to as part of their daily life. It's busywork, and giant class libraries do a wonderful job of making life easier. They let you concentrate on the important stuff - the algorithms, and marketing's feature of the week.
Yes, libraries have been around for more than a decade. The STL and Java both showed up in the mid 90's, though, and I'd argue they've made a huge difference since then. Yes, they have bugs. Yes, there have been other advances. Yes, there are special cases where standard libraries aren't appropriate. There's no silver bullet, but standard libraries are a huge advance.
Not Engineers, it's management (Score:5, Insightful)
I can't count the number of times last minute feature requests are required to be in a build. As software developers, we just deal with it. But quality suffers because of it. And the engineers get the bad wrap.
Do you think Intel management requires last minute features to the Pentium core, and tries to push them out the door with no testing? Do you think people building cars for Toyota decide to swap engines at the last minute, becuase GM just put a turbo charger on it's standard Prizm?
It's ludicrous the stuff that gets requested of software engineers.
My brother is an contract electrical engineer. He was complaining one time of a 'last minute feature request'. His project still had a year of development time left! I laughed so hard I almost puked.
Granted, given all that, the current model for software works. When software is required to be bug-free, it can be. Lots of software is bug free. You just don't hear about it, because it works. Look at NASA, the AIN telephone network, or Windows.
Judging From the Posts here, I say it has (Score:3, Insightful)
People are complaining about how using stronger employees to lead less-skilled employees doesn't work. People are complaining that the person testing the "bad coder's" code is usually the "bad coder" itself. These complaints begin to define the actual problem of "Why software development isn't improving." Each of these complaints, taken together and addressed, is the first step towards improving software design methodologies.
Best tools, present and past (Score:5, Informative)
Here is my take on the best environments available today (in order of best first) - I only include stuff that I use:
- Cincom VisualWorks Smalltalk - a really fast coding environment with support for SOAP/WSDL/UDDI/all possible database connectivity, built in web development tools, huge class library so often new applications are very little new code
- Microsoft VisualStudio.Net - I almost hate to include this one, but I am trying to be honest here
:-). Last year when I was writing a J2EE (Java Enterprise) book, I took a good look at the competition and liked C# and .Net.
- Java with light weight web services - I am a HUGE fan of Java on the server side, done light. By light I mean that I like to just use servlets and JSPs, etc. and only use heavy weight J2EE stuff when I really need transaction support, etc. that EJBs provide
- Common Lisp - lots of great commercial tools from Xanalys and Franz, and lots of great free tools like CLisp, CMULisp, OpenMCL, etc.
- Python - Love it for small projects, munging text, etc.
- C++ - only use it when I must - great runtime perfromance, but a pain to code in
It seems like so much work today requires deploying web services so I appreciate tools that make it easy to develop basic application logic separately from any user interface, get it working, then write web and/or standalone GUI interfaces.-Mark
Warnng: I am blogging now: http://radio.weblogs.com/0115954/
Communications have been the real revolution (Score:3, Insightful)
And in this domain, there has been a revolution, namely the Internet, and the arrival of cheap connectivity between developers anywhere in the world.
Prior to this, the only way for developers to collaborate was to be hired by the same company, work on the same network. Inefficient as hell.
Today any group anywhere in the world can create a project and work on this as efficiently as a small group in the past.
The irony is that the revolution does not care a shit about the technology used, and works as well for COBOL programmers as for companies cracking the human genome. It's about solving a purely human problem: communications.
Software development has *changed* (Score:3, Insightful)
Progress in software engineering (Score:5, Interesting)
Brooks, in the "No Silver Bullet" essay referenced above, stated that there is both essential and accidential complexity in software development, and because of that there never would be a "silver bullet" to slay the software "monster". However, there are fundamental practices that increase the likelihood of success and fundamental pitfalls that every project faces. And, in the end, the root causes of most failed IT projects are human factors; in fact, you could just cite the "seven deadly sins"--pride, envy, gluttony, lust, anger, greed, sloth--and probably hit the nail on the head.
In conjuction with that, far, far too many practitioners in the IT field lack one or more of the following:
To quote George Santayana (who is often misquoted):
Software engineering is hard enough--with all the human issues--without further handicapping ourselves with ignorance of all that has been already discovered and documented. Yet that is exactly what most IT workers do. Until we find a way to solve _that_ problem, the failure rate for IT projects will remain high indeed. ..bruce..
So, show me the goods (Score:3, Troll)
In my experience many in the IT field fall into one of two categories: Those who do the work and those who make a living telling people there are better ways to do the work (this group usually correlates with people who couldn't actually do the work themselves). Reading your resume [bfwa.com] it is readily aparent what category you fall into...
Opinion: Yes. (Score:3, Interesting)
I think the ingredients you mention have made production of software a better process over the past 25 years. Software applications can do more and can be built more quickly as result of those improved tools available to developers.
However, you still see a great deal of unreliability, bloated, and inefficient code because developers are trying to do much more than they did 25 years ago.
If all we needed to do was re-create the applications of 25 years ago, then the benefits of new techniques would be more evident. But people demand more and programmers want to create works up to their full personal potential and exceed what is currently possible.
Ragged-edge software is manifest evidence that we still are constantly crossing the barrier of human potential, that place where what is barely possible becomes what doesn't work. It's a good sign of innovation. And, it provides added impetus to keep trying to find more ways of improving the software development process.
It is obvious who the leader is (Score:4, Funny)
Better Abstractions *Have* Made Things Easier (Score:5, Insightful)
The most a high-level language can do is to furnish all the constructs that the programmer imagines in the abstract program.
A-men. But if it does that well, then it makes the job a lot easier. That's why going from C to Java or Perl was sheer relief. Actual strings? Real associative arrays? Whoohoo! And less memory management grief. Not to mention the component libraries available for things I hadn't even thought of yet. CPAN...
To be sure, the level of our thinking about data structures, data types, and operations is steadily rising, but at an ever decreasing rate. And language development approaches closer and closer to the sophistication of users.
True... but the user sophistication is increasing to. It seems highly apparent to me that with more experience and more shoulders to stand on, language and component developers are able to concieve of more and more useful abstractions. And because of the internet, they're more easily available for sharing, commentary, and change.
To sum up, I am much, much happier with the readily available toolsets I have access to now than the ones I had 15 years ago, or even eight years ago. They make developing much easier and much more fun.
It isn't engineering yet, and won't be for a while (Score:5, Interesting)
If requirements were as poorly understood and changed as fast in the physical world as they do in the software world construction would cost a fortune and most big buildings would never get finished (or would never be fit for purpose).
People who say things like "Make sure the requirements don't change" are living in a fantasy world where they want to blame their inability to deliver on someone else.
The rules haven't changed, get a high quality small team, get good access to a user who knows what they want and grow a system from small beginnings, checking at each stage that it all works and that quality is high.
Its all there in Brooks.
Its no surprise that the guys pushing the agile methodologies were all very succesful designers and developers anyway.
Software development improvements (Score:5, Insightful)
I notice the original post mentions several things that could influence the development time of a software project. I will address a few of these below:
1) Object Oriented Programming :)
This is one of the bigger Silver Bullets to be unleased upon the programming world. I don't think it entirely lived up to the hype. Most OOP is just for local project design, and heaven help you if you have to reuse code somewhere else. It isn't just a case of bad design. Problems like software design are actually ambiguous. The design process is not algorithmic; rather, it's heuristic. You use "templates" and "patterns" to represent your ideas. Trying to shoehorn real-world complexities into these cookie-cutter styles is difficult at best. Trying to further take those styles and integrate them with each other in a very large scale product is a hair-tearing nightmare. I think Tablizer would agree with me on this...
2) Reusable components ;)
The most visible place reusable components come into play is GUI programming. It's very, VERY simple to use a visual-based system (like Visual Basic, C++ Builder, Delphi, etc.) to create a GUI simply by dragging the desired components onto the blank form window. If anything has been sped up significantly in the past several years, it has been the GUI development.
Components are, of course, used in a variety of other places, particularly in run-time libraries of various programming languages. However, learning to use these components effectively takes more time and dedication than one might suspect as the syntax tends to be rather cryptic looking.
3) Java
Don't get me started. I am currently employed as a Java developer. I don't really like it a lot. The file scoping rules bug me. (Similarly, I don't like Python because of the way it enforces indentation.) Also, the Java IDE sucks. Whoever thought the entire GUI needed to actually be written in Java needs to be taken out and beaten with a stick. A large stick.
4) The Internet (and OSS) ;)
One thing I noticed that you hadn't mentioned is the Internet. I have never been exposed to so many programming concepts and new languages. There is an astounding variety of tools, and thanks to Open Source and researchers at various universities, you can try your hand at as many of them as you have disk space for. The 'Net can be a wonderful place, after all.
My advice to any new programmer would be to get online and start reading. Download and try out new languages, especially ones in different paradigms, like functional programming. The tools you need (such as compilers, editors, databases, GUI component libraries, etc.) are ALL there, free for the taking. The only real "silver bullet" is to make yourself the best programmer you can be.
Design, design, design (Score:3, Informative)
The most important thing in developing a product is having the "right" design.
What he was getting at is that, if you rush into devleopment (coding in this case) without having fully thought the design though, you could end up shooting yourself in the foot later on. Redesigning and recoding something later on in its development cycle can be hideously expensive and time-consuming. Also, depending on the industry, it can be fatal (think lost contracts from being late). It's absolutely vital to think all the issues through regarding your product, not just in the short term, but what you might anticipate for the long (and in many cases, the very long) term.
It's all about the process (Score:3, Informative)
Not enough! (Score:5, Insightful)
I've often thought over the last few years that we've made too little progress in making programmers more productive. I largely blame that on Microsoft, simply because it drives more software development with it's tools than any other entity. One language I've categorically made a decision to avoid is Visual Basic. I have always felt it was basically (sorry) a waste of brain cells. It has certainly done nothing to advance the state of the art.
In my opinion, one of the best things to come along in a long time is Java. The gentle reader may recall earlier posts along those lines. I enjoy C, and have spent the majority of my career doing C and C++. However, I have also spent _way_ too much time tracking down memory-related bugs. Often, they were in third party code. That is no way to run a railroad.
Java addresses almost all of the glaring deficiencies of C++, both in language design and in runtime safety. In my opinion, the best programming tools will be those that enable single programmers to tackle larger and larger projects.
Compared with C++, Java enables me to tackle much more ambitious projects with confidence. A team approach can never attain the efficiency of a single programmer approach. The "sweet spot" of software engineering efficiency is the largest project one person can tackle. Extreme programming is a useful hybrid that attempts to turn two programmers into one programmer. ;-) (Also teams can be nearly as efficient as single programmers if the system is properly decomposed into subsystems separated by simple interfaces. This rarely happens smoothly, in my experience. It takes a top notch group of people.)
One last note on Java - performance is now almost completely on par with C++. On my most recent round of benchmarks, Java (JDK 1.4.1_01) on both Linux and Windows outperformed C++ (gcc 3 and VC 6) on most tests. Dynamic compilation and aggressive inlining are that effective. The VM also soundly spanked the gcj ahead of time compiler in gcc 3. It thoroughly rocks to have truly cross-platform code that runs faster than native! Think how many religous wars would be avoided if 99%+ of software was available on all OS platforms...and think how much it would help Linux! :-)
If you want to see what's out there for Java, download either the NetBeans IDE project [netbeans.org], or the Eclipse IDE [eclipse.org]. Both are free and each has its strong points. NetBeans is a Swing app and includes a Swing GUI designer. Eclipse uses a new open source "native widget wrapper" library from IBM called SWT which has it's interesting points. You'll also need a Java VM [sun.com] (there are also others available from IBM etc.).
One last thought - wouldn't it be cool if web browsers had support for something like Java? I mean, you could deploy apps just by putting them on a web page! It wouldn't matter what the target platform was! What a great idea! (This paragraph was sarcasm in case you were wondering.)
More to it than code and tools (Score:5, Insightful)
In my experience, the actual process of coding has greatly improved over time but the process of developing software hasn't improved as much. As other posters have pointed out, object-oriented tools, technologies and techniques (among other factors) have greatly facilitated the generation of code but the management of the process; deciding what gets coded, when, by whom, etc. is little better now in actual practice than it was in the late 70's or early 80's. In fact, in my opinion the situation is in some respects worse.
Management of software development today makes the same mistakes and operates under many of the same misguided assumptions as it did back when I spent my day in front of a dumb terminal. Adding outsiders unfamiliar with the details of a project makes the project later, not earlier, etc.: all the platitudes we know and love are still with us, still the butt of Dilbert jokes.
Technology may change; people aren't quite so amenable to upgrades, I think.
OF COURSE IT HAS IMPROVED - Look at M$! (Score:3, Funny)
Security is at the heart of the Windows' design - It is so bloated, unstable and slow that no hacker ever gets a chance. Now you know why it got C2 certification.
The unfulfilled dreams of infinite modularity (Score:5, Insightful)
The idea of course being that good modular design, and good use of classes can increase reuseability ad infinitum, so that eventually there will be no task and no problem that cannot be assembled lego-like from blocks of premade reuseable code. The marvelous technology in Douglas Coupland's Microserfs ( called Goop wasn't it ? ) was really the epitome of this concept, a totally reuseable code system that was so generalized, so modularized that anyone with no more experience in programming than a child could assemble working programs for just about any purpose in a drag-and-drop virtual lego-building reality.
Any student of the history of science and science forecasting should begin to smirk at this point. Is it any surprise that these visions have not materialized? The hard truth, IMHO, is that logic is inherently not conducive to such high degrees of abstraction and modularity. For given tasks which are fixed and well defined with completely known and understood parameters and requirements than yes, abstraction and modularization can be a great boon in optimizing design and improving the development cycle. We can see the great advantages that this has yielded in some areas where the requirements are fixed and the applications are mostly derivative of each other. GUI design is a great example, and there are a multitude of GUI building toolkits & development environments that take great advantage of this.
However the whole thing breaks down when you move from the known to the unknown. When you try to extend this principle from fixed tasks to hypothetical ones in which the basic requirements are not envisioned. I would argue that there is a basic lack of understanding among some of the proponents of these techniques that at a fundamental level logic cannot be generalized out across all possible tasks, and all possible configurations. This was similar to the revelations of Godel's theorem in mathematics already in the 1930's - that any axiomatic system could be consistent but never complete. In reality adapting one set of code to a purpose other than that for which it was designed, or with parameters other than those originally envisioned usually is more trouble than it is worth, and often you would be better served by building new objects from scratch. You will always need intelligent educated people to design these things; there is no such thing as logical lego.
Unfortunately it seems to me that many have not gotten this message yet. Sun and Microsoft are still actively humping that dream of infinite modularity and drag-and-drop programming design. In my experience with both Java and .Net, I have found that I always run into blocks where the established object model is too constraining and has too many built-in assumptions about what you're going to be using the classes for, and so I have ended by coding new versions from scratch. Of course it may simply be the nature of the applications I'm working on, and your mileage may vary. Ultimately I think that for derivative applications this kind of abstraction and generalization is definitely an improvement, but when you come to applications that move beyond those fixed tasks it actually becomes an impediment not an advantage.
Literate Programming (Score:3, Interesting)
DEK has since written an entire book on the concept (_Literate Programming_ a CLSI series book) a decade ago, but one seldom sees source so provided.
There are some really cool example programs which're quite interesting (and educational) to read, for example:
Will Crowther's game Adventure - available here: http://sunburn.stanford.edu/~knuth/programs/adven
Or a CWEB version of the RPN calculator for K&R's C Book: http://www.literateprogramming.com/krcwsamp.pdf
Probably what really needs to happen is a way to post a program as a web page, then to click on a link on it, to automagically compile and run it....
William
Development Improvements (Score:3, Insightful)
Such things as Structured Design, and OOP have made coded reuse better.
What hasn't improved:
1) Programmers STILL refuse to use tools that could help them in productivity. (i.e. source debuggers, instead of writing printfs around everything and printing out your variables)
Tools tools TOOLS people. Use a source debugger and save yourself a great deal of time.
If you can't use a different language or infrastructure to write the code.
Sadly, many programmers still do not use source debuggers, citing a waste of time. But they will sit there and hack over and over again trying to understand the code they write with printfs!
Tsk tsk.
2) Cost Time Estimation. Wow, talk about almost zero improvement there. Almost zero, but not quite zero. After all, most people are now adopting an open source strategy so that even if your estimates are off, the cost penalties are reduced. Furthermore, most people are beginning to realize that you have to complete a full requirements document, and do some fact finding before you attempt to quote work.
3) Finally, the hardware we use to write software is vastly more powerful, and as a result we can run much nicer environments on our machines when we write code, such as API references, etc. I have far fewer references now days to things like Java for example than I use to have to keep on my desk. Primarily because with the rise of IDE's the development environment can answer alot of questions I might have about the language I am using to write the software with.
---
I would also like to point out things have got a little worse. If you believe like I do that 80% of the work in writing software is debugging it and maintaining it over its lifetime, then you like me have problems with our IDE's.
Primarily when our IDE's produce automated code for drag and drop environments. They produce horrible code, at the expense of saving time now, and end up costing a great deal of time later.
(Anyone use the latest
I primarily write only Java code, but even my SunONE environment produces some pretty cruddy stuff if I am writing a desktop app.
I think automatic code generation is a step backwards in many ways, and ends up costing more money to fix or maintain it.
I still think a code "repository" built by humans, and nicely documented, like a cvs tree for example, is the better way. Time spent on the CVS code repository for building customized pieces is time much better spent IMHO.
-Hack
Absolute progress, but not relative progress (Score:3, Insightful)
Bill Joy wrote vi pretty much by himself. Bram Moolenaar write Vim pretty much by himself; it's a huge superset of Joy's editor.
The original portable C compiler (PCC) was about 5,000 lines of C. No one even blinks if a graduate student writes a far more sophisticated language processor, e.g., a C++ compiler, a complete Java development environment (including a compiler and debugger).
The original SimCity was awesome. No one thinks twice of re-implementing it for a PDA or a Java-enabled web browser.
What's changed? Programmers don't have to worry so much about CPU, disk, or memory limitations. The tools (compilers, libraries, source control) are much improved. Some of the new languages are far more productive. There are also new practices, and the results of lessons learned, on how to do development; some programmers take advantage of these (not enough!)
But our abilities haven't kept up with our aspirations. Compare SimCity to the masively multi-player Sims Online. How do vi or PCC stack up against Eclipse [eclipse.org]? Look at the text formatters of twenty five years ago, and then look at OpenOffice (or Microsoft's stuff); at Unix v6 vs. Max OS X.
Software hasn't kept up with Moore's Law. We're running a Red Queen's race, going as fast as we can just to stay in one place; and we're losing.
Software not a scientific discipline (Score:5, Insightful)
Software is at best a cottage industry of craftmen that have widely different abilities. At worst, it is a cottery of alchemists that promote their own secret snake oil recipes and only succeed by sheer luck.
And I am a developer that calls himself a software engineer, so this is not a flame.
Why craftmen? Because we develop with tricks and recipes, often called "processes". But neither of these are scientific processes. They cannot predict the outcome of a software project within a definite set of constraints. They cannot be disproved. And they cannot explain failures. So they aren't science, they are rules of thumbs. The guy who makes a living by applying rules of thumbs and learned tricks is a craftmen.
Why alchemists? Because scientists publish their methods and their results. To the contrary, the software industry hides its customer references and does not publish its source code (with the notable exception of Open Source, with remains the exception in large-scale software projects). This is how alchemists and "savants" worked in the Renaissance. They hid their trade secrets, they had confidential relationships with rich patrons, and they feared full disclosure.
On top of that, each subbranch of computing has its own lingo and redefines as many words as possible. Mathematicians who specialize in field theory topology may not understand number theory, but at least, they use distinct, well-defined jargons. In computing, terms like "record", "field", "server", "link" are so overloaded that they are just noise.
About 60% of all software projects are cancelled or written-off as failures. I don't think civil engineering or, say, aeronautics have such a abysmal track record.
I hope that some day, we'll practice and teach Computer Science as, well, a science, not as a craft.
Answer: YES (Score:4, Insightful)
However, take a look at Parnas' classic paper on information hiding. [acm.org] In it, he makes the following statement (emphasis mine):
Is there anyone in the crowd that doesn't think he could write a shell script/perl script/etc. to accomplish this in 30 minutes or less? I'd be willing to bet I could do it in Python in under 15 minutes.This paper was written 30 years ago. It only pre-dated Brooks by 5 years. In that time, if my estimate is accurate, the development time for simple programs may have fallen by a factor of a hundred or more.
Software doesn't appear to have improved in that time simply because we keep trying to do more complex things.
Re:"I'm in a Computer Science degree program"????? (Score:3, Insightful)
Yeah, as a PhD student you get paid practically nothing (at least when compared to corporations), you have to work silly hours and possibly teach as well.
Yet, if you pick your topic carefully, you'll be working those hours on interesting and intellectually challenging bleeding edge stuff and not on some YAMC (yet another moron client) project that was handed over to you. In a nice group you can also choose whether you want to come in to work early in the morning or at around lunch time. Furthermore, teaching experience can never hurt a nerd. I used to hate teaching and I still don't like it. However, positive teaching experiences and the very act of confronting a group of smart, young people for four years improved my self-confidence, presentation and public speaking skills tremendously.
I dunno what world YOU'RE living in (Score:4, Insightful)
you're kidding yourself if you think Java keeps you from having memory leaks, and I have enterprise code to prove it
Re:I dunno what world YOU'RE living in (Score:4, Insightful)
Um. That was his whole point - bad engineers will always be bad engineers, no matter what the language. Java prevents an entire class of memory leaks by garbage-collecting unreferenced objects, but there's nothing in the language that stops you from doing something stupid like writing an unbounded cache.
Re:gpl dude (Score:3, Insightful)
Brooks seems to be telling us that instead of making our own tools we should be using tools made by others.
Open & commercially free software (software that you are free to make money off of) is the ultimate realization of that phillosophy, where anyone can see all the code and use all the code to make your product better.
Re:You're not going to like this (Score:3, Insightful)
Coding is not an engineering discipline. Coding is typing. Coding can be done from a Rose
I'm going to get flamebait for this but I don't care. I'm a programmer and I know my line of work is a crock of shit, but I'll take all the money my employer will give me.
Re:You're not going to like this (Score:3, Insightful)
On the other hand, if a sidwalk curb isn't designed right, it wastes some concrete. People die if the software in their pacemaker doesn't work.
It's the project, not the discipline.