Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×

Why Software is Hard 409

GoCanes writes "Salon's Scott Rosenberg explains why even small-scale programming projects can take years to complete, one programmer is often better than two, and the meaning of 'Rosenberg's Law.' After almost 50 years, the state of the art is still pretty darn bad. His point is that as long as you're trying to do something that has already been done, then you have an adequate frame of reference to estimate how long it will take/cost. But if software is at all interesting, it's because no one else has done it before."
This discussion has been archived. No new comments can be posted.

Why Software is Hard

Comments Filter:
  • by Mateo_LeFou ( 859634 ) on Saturday February 03, 2007 @05:56PM (#17876502) Homepage
    ...can anyone explain Vista's schedule in light of this discovery?
    • by edwardpickman ( 965122 ) on Saturday February 03, 2007 @06:02PM (#17876544)
      ...can anyone explain Vista's schedule in light of this discovery?

      Another law explains it, Entropy.

    • by __aaclcg7560 ( 824291 ) on Saturday February 03, 2007 @06:38PM (#17876838)
      All the programmers got better jobs at Google?
    • Re: (Score:2, Funny)

      by dreamlax ( 981973 )
      I heard that all of the programmers suffered head injuries from having chairs thrown at them.
    • by MindStalker ( 22827 ) <mindstalker&gmail,com> on Saturday February 03, 2007 @08:24PM (#17877534) Journal
      Its esentially the problem with any large company. Any project coming out of a large company spends 90% of its time in committee and debates, and generally gets watered down to the few things those committees can agree on. Personally I disagree on the idea that two coders arn't as good as one. One coder rarely has the motivation as well as the insight to see clearly his way through a project (getting stuck on a problem and having another viewpoint is ALWAYS helpful). But there is definatly a deminishing return as project teams get larger.
      • by Almost-Retired ( 637760 ) on Sunday February 04, 2007 @04:03AM (#17879576) Homepage
        I disagree on the idea that two coders arn't as good as one

        This is a point I've come to also after 72 years, the last 35 or so of it coding this and that although not much recently as I seem to be fadeing into the dim sunset of SS mentally.

        When working in assembly, then it may be that one person is the optimum number of coders as I've done some never before been done stuff in assembly several times in the early years, sometimes with hand assembly (on an 1802 board) where you look up the nemonic and enter the hex equ in a hex monitor. It took me about 6 months to fine tune about 3k of code, but it was still running 12 years later when I last checked in at that station. And still saving the station 2-3 man hours a day and giving them a better air product at the same time.

        But for a higher level language, I think 2 can be more productive, particularly when one knows what he wants to do, and the other knows how to do it once its properly outlined. Many times the coder himself is simply too close to the code to see the job it has to do, but the partner in turn has a good idea of what its got to do. The genesis of at least 2 fairly well known amiga programs were from the mind of a younger man in another dept at the tv station, and he would hack up what he thought might work but didn't, but once I knew the requirements, the final code more than likely came from my keyboard. He had the imagination that I lacked, possibly due to my advanceing age, and was in turn concentrating on his job's duties which I wasn't always aware (I had other responsibilities too) were being done at less than optimal methods. We sure made a good combo crew though.

        I have NDI how many man-years in in vista right now, but I dare say it is a substantial investment in both time and programmer salaries. I'd also wager that at least 75% of any one programmers day was spent conferring with other programmers as to the best way to do it, and get it done within the generally immutable confines of the .h header files. This is NOT to me, best use of the programmers time, so the what does it do, and how it does that, really ought to be seperated in any large project.

        As for re-using known good code ideas, or a 150 line snippet here and there, it is to be encouraged at every staff meeting, re-inventing the wheel is not good use of his time and as others have said, only serve to intro new bugs that then have to be run down and fixed. Programmers really should get over the attitude that I can write it quicker, and spend more time reviewing older code to see if it can be recycled. There is much knowledge in 10 year old code thats still in use everyday.

        Will my little treatise make any difference at the end of the week? Donbesilly, This is after all, /. :-)

        --
        Cheers, Gene
    • by donaldm ( 919619 )
      It is really quite difficult to program a ham sandwich into MS Vista and trying to prevent it from getting rather smelly and stale. Those refresh routines must be very complex. http://news.com.com/5208-1032_3-0.html?forumID=1&t hreadID=16504&messageID=142375&start=-1 [com.com].
  • by alshithead ( 981606 ) * on Saturday February 03, 2007 @06:00PM (#17876530)
    "But if software is at all interesting, it's because no one else has done it before."

    "Interesting" to me means something new and/or unknown...mostly. There are exceptions. Treading new ground always requires greater effort. If I cut a my way through virgin jungle then those who follow have a path.
    • by SQLGuru ( 980662 ) on Saturday February 03, 2007 @06:38PM (#17876844) Homepage Journal
      I take game programming classes. One of the instructors made some very good points related to innovation. His context was game wise, but since my background is business application programming, I can easily see how it applies here.

      When you innovate in a game, only make one....maybe two innovations. Otherwise, you skew so far away that you usually end up a complete failure. Applying it here: sure, keep things interesting by doing some piece new, but keep it manageable by keeping the rest of it "boring". You gain predictability while retaining "fun".

      Layne
      • Hah, gaming programing classes. You won't find one person thats important in the industry that's taken those (grunt-work is ok for allot now I guess). As for your instructors words, that's the EA method right there. You want to be part of that? Games that fail, fail for a number of reasons that aren't because they did somethings that were innovative. Poor implementation yes, but thats not because they did too many new & interesting things. If you poorly implement an idea your going to fail (or at least
        • by SQLGuru ( 980662 ) on Saturday February 03, 2007 @07:54PM (#17877334) Homepage Journal
          Actually, every instructor I've had works in the industry. Not *DID WORK*....but *WORKS*. Classes are at night. It's in Austin, so there are plenty of studios to pull from. I've had instructors that have worked on games from all eras and genres. Some of the companies that represents: Sony and SOE, Midway, NCSoft, and Microsoft. Plenty who have started their own studios after having worked at bigger ones, too.

          http://www.austincc.edu/techcert/Video_Games.html [austincc.edu]

          It's not a degree program (yet), but I'm not too worried about that since I already have a CS degree. For me, it's more about having fun, learning some new stuff, and making good contacts for when I'm ready to jump into the industry.

          Check out the list of names on the Advisory Board and the list of Instructors. There are some influential names on that list.

          Layne

      • by Evil Pete ( 73279 ) on Saturday February 03, 2007 @08:03PM (#17877390) Homepage

        I read somewhere that in science fiction writing this is called "The Tooth Fairy Principle". Don't introduce more than one exotic technology or idea. I immediately realised that it applied even more strongly to software development. New areas represent areas of high risk, adding even a few to a project can change the risk from moderate to very high. I've participated in a few projects who broke this principle ... as usual commenting on the risk that this implied only made me sound like a Cassandra when eventually the prediction bore fruit.

        However, the major reasons I see for software projects becoming late are: clients repeatedly wanting to change design after the design phase (in one surreal case we had a client change a fundamental design issue 24 hours before going live!), poor resource allocation (a very large subject), management saying yes to unrealistic deadlines, bleeding edge technology (Tooth Fairy Principle - high buzzword compliance).

    • Re: (Score:2, Insightful)

      by iminplaya ( 723125 )
      If I cut a my way through virgin jungle then those who follow have a path.

      And copyright puts in the toll booth.
  • Programmers (Score:5, Insightful)

    by bendodge ( 998616 ) <bendodgeNO@SPAMbsgprogrammers.com> on Saturday February 03, 2007 @06:00PM (#17876532) Homepage Journal
    One programmer is better than two for the same reason that one woman in the kitchen is better than 2. You have to get on a pretty large scale before you need multiple cooks/programmers.

    Software programming in general is hard for 2 reasons:
    1. Computers aren't built for interfacing with humans, thus UI us terribly time-consuming.
    2. The environments people like to drop an app into can be so bizarre, that rock-solid stability is very difficult to achieve.
    • by bennomatic ( 691188 ) on Saturday February 03, 2007 @06:06PM (#17876592) Homepage
      Where do you live? The 50s? You may want to ask some women you know about using that particular illustrative image.

    • By #1 I presume you mean because we don't have telepathic reader devices yet? Voice recognition, typing, mouse, pad, text recognition, visual plotting; what's missing?
    • Re:Programmers (Score:4, Interesting)

      by fireboy1919 ( 257783 ) <rustyp@NoSpam.freeshell.org> on Saturday February 03, 2007 @07:34PM (#17877206) Homepage Journal
      Most cooking projects don't take more than 10 man-hours, but pretty much every programming project does. And, furthermore, mostly when the chef makes a mistake it's obvious to her.

      Neither condition hold for programming. It's for this reason that I think that, in general, *two* programmers can program faster than one. At least, me and my partner can program code that's more bug-free together than we can when we program separate projects, and that makes a difference. If the project is sufficiently large - i.e. takes longer than about 10 hours, the cost of communication between two people is less than the cost of switching. :)

      While we're at it, I think that there's another misconceptions in this interview.

      programmers are programmers because they like to code -- given a choice between learning someone else's code and just sitting down and writing their own, they will always do the latter

      Two of the five developers at my little software company are programmers because they like to figure things out. So we almost always figure someone else's code out before we do anything ourselves. There are varying degrees of this in a lot of the developers we've got there. I would say that none of us will write anything ourselves unless it saves us a considerable period of time.

      But even more, if you had a relative who was always wondering, "What is it that you do all day?" you could hand my book to that relative and say, This is what my work is really like.

      No. I couldn't. My experience as a developer is nothing like what he's described. And he didn't talk about the phenomenon of unknowns that I've noticed - for every project I do, if I estimate how long the known things will take, dealing with unknowns will generally take 60% longer (so multiple time estimates by 3 is generally correct). He didn't talk at all about testing.

      Almost everything he talked about are things that I thought would be true when I started but that have ended up more or less untrue. Discipline coding makes a difference. Automated unit testing catches most problems, and regression testing finds almost all the rest, and not everybody does these things.
    • Re: (Score:2, Insightful)

      by arminw ( 717974 )
      ......Software programming in general is hard for 2 reasons......

      Actually there is only one master reason. So far, there is no mathematical way to prove that a given non trivial software program will actually work as intended. When designing a physical thing, such as a bridge, machine or electrical circuit, there are precise mathematical formulas which can be applied that give a reasonable expectation that the building or machine etc. will perform according to expectations. There are no mathematics that can
  • by euice ( 953774 ) on Saturday February 03, 2007 @06:03PM (#17876556)
    and of course, we are the better programmers, so better fire those other 8.
  • by petes_PoV ( 912422 ) on Saturday February 03, 2007 @06:04PM (#17876560)
    Mostly programmers are trained in the technical details of languages and the libraries/APIs associated with them. They don't gain skills in knowing what users really want and are hurried into producing barely-working stuff, fast.

    Whatever testing is done often only tests that the product produces the correct answers when feed the proper input - no account is taken for how the program reacts to incorrect or incomplete data.
    Changes are requested faster than they can be implemented and often are not communicated very well.

    In short there are systemic failures throughout the whole process, from inception through to delivery. There is no single answer to why software is hard and there won't be until the industry matures and people start to get thrown out of the business for acting unprofessionally

    • Re: (Score:3, Interesting)

      Mostly programmers are trained in the technical details of languages and the libraries/APIs associated with them.

      Mostly, in my experience, programmers aren't trained much at all.

      There is an old saying: if you think education is expensive, try ignorance. Well, we have been, and early results are pretty much in line with predictions.

      • by CastrTroy ( 595695 ) on Saturday February 03, 2007 @11:04PM (#17878318)
        I think this is one of the biggest problems with software today. Too many untrained/undertrained people working on too much software that they are not qualified to be working on. The only reason the term software engineering is a joke to most people is that most people who work on software do anything but engineering. It's not just me either. Everybody I talk to works with people who have no idea what they are doing, and should not be working in the software field. Granted, neither I nor any of my friends that work with these people are perfect, but some of the stories i've heard are almost unbelievable. I'm surprised software ends up working at all in most cases.
  • by Allicorn ( 175921 ) on Saturday February 03, 2007 @06:04PM (#17876574) Homepage
    Programming websites that let you actually view a page without requiring a cookie is obviously hard for the folks at Salon.
  • Software becomes hard when heavy-handed management decisions are made to give too much emphasis to a particular software tool or methodology.
    1. An expert programmer (working with human factors experts) can prototype the new system in a cross-platform scripting language (doesn't matter which one), then can identify the objects
    2. a software team can refactor the system once again in an object-oriented language (doesn't matter which one).
    3. Finally, a period of benchmarking can identify the bottlenecks which can b
    • Re: (Score:3, Insightful)

      by malraid ( 592373 )
      This is also a problem with some programmers. Most geeks place more emphasis in the tools than on the objectives. Some don't even care about the objectives (basically the need of the users) and just want to use a shiny new tool. Or they want to do whatever task in the same tool no matter what (there is a saying that a determined Fortran programmer can write Fortran programs in any language).
      • Yup, it doesn't matter what language I write in, it always ends up looking like C...
      • Re: (Score:3, Insightful)

        by Lazerf4rt ( 969888 )

        You are right on the money. Programmers tend to try put more priority on building re-useable components (tools, modules, etc.) than on actually building the damn product. I know I've been as guilty of that as anyone.

        We're all taught that re-useability, modularity and portability are great ideas. But if you look around at many software projects, these principles are often given top priority, and the cart goes squarely in front of the horse. Few people realize that early architecting can be as evil as early

  • by Herkum01 ( 592704 ) on Saturday February 03, 2007 @06:09PM (#17876620)

    I would say the reason a lot of projects, even small ones take so much time is that requirements cannot be defined.

    Compare building a house to software. Before you build a house

    1. Plans are drawn up
    2. A step-by-step schedule to created for the construction.
    3. Contractors are brought it to complete the work as needed

    Schedule times can slip but you still know where you are in terms of progression.

    If we built this house the way we do software development

    1. Hire all the construction workers
    2. Tell them to build something.
    3. At any point during construction tell them they are not doing it right.
    4. After missing all the deadlines (which were made up by wants/desires of the customer) hire more workers.
    5. Wonder why they cannot get the job done
    6. Cancel the project after everyone realizes they don't want it anymore.
    • by cowscows ( 103644 ) on Saturday February 03, 2007 @06:19PM (#17876694) Journal
      I design buildings for a living, and I've dabbled in programming, and I think architecture and software development have a whole lot in common.

      Your step one in "building a house" can go through all 6 of the steps that you have listed for software development. We get hired by clients, sometimes they have a good idea what they want, sometimes they don't. Sometimes what they want is feasible, sometimes it isn't. It's not unusual for even smaller projects to drag on for years, because the client keeps changing his/her mind. Many projects that cross our desks will never be built.

      Many projects are not the traditional design phase ->building phase. They often overlap, and it's pretty messy.

      I could go on for paragraphs with the similarities that I see between software design and architecture, but I'll save that for another post.
      • by Hoi Polloi ( 522990 ) on Saturday February 03, 2007 @06:24PM (#17876748) Journal
        I think one thing they all have in common is that they are always custom jobs. It isn't like going to a car dealership and asking for model X in dark blue. Software is more like "I'd like a car with extra wheels on top in case it flips and purple stripes and only 1 door...". Standardization is very limited.
        • by Citizen of Earth ( 569446 ) on Saturday February 03, 2007 @11:18PM (#17878384)

          "I'd like a car with extra wheels on top in case it flips and purple stripes and only 1 door...". Standardization is very limited.

          Now imagine if every single weld was a unique, custom job that had never been done before, and if any of them are imprefect, the car crashes.

          • Re: (Score:3, Insightful)

            by Coryoth ( 254751 )

            Now imagine if every single weld was a unique, custom job that had never been done before, and if any of them are imprefect, the car crashes.

            Right, because there are absolutely no "standard recipes" in software. There just isn't anything you could describe as a "Cookbook" [atomz.com] providing standard solutions to common problems that make up the basic nuts, bolts and welds of a lot of software.

      • Re: (Score:3, Informative)

        by dcam ( 615646 )
        I've worked with architects. One thing I think that is different about software is that you can change things once they are in place. That is it is possible to move the whole building 100m down the road. The very changability tempts people to change things, which causes problems.

        Secondly I think people know more what they want when it comes to buildings because the options are more restricted. In software there are no restrictions on what can be built (although you can run into performance restrictions).

        Thi
    • The clients who don't know what they want aren't so bad - most of them will accept whatever you give them. The real problem is with the ones who do know what they want, but can't describe it properly. Or the ones who want the impossible (but those are usually easy to spot early on).
      • Or the ones who say "hey, that gives me an idea... it'd be really neat if that could do X, too!" throughout the project. Feature creep is probably the single largest reason why programs don't meet deadlines.
      • Re: (Score:2, Funny)

        by nomadic ( 141991 )
        Or the ones who want the impossible

        Perhaps you'll tell the Emperor yourself WHEN HE ARRIVES!
    • by Jerf ( 17166 )
      Actually, I say, if you want to see the silliness of the venerable Construction metaphor, show how we'd really build houses if we built houses the way we built software.

      First, building a house is a solved problem, so you'd never hire an architect or builders. You'd go down to Best Buy and buy Microsoft House for $89.95. Any reasonable requirement you can think of is covered by Microsoft House; you have to really try to throw it for a loop.

      Microsoft House is guaranteed to conform to all building codes in all
  • by mrjb ( 547783 ) on Saturday February 03, 2007 @06:10PM (#17876624)
    Yes, writing software is hard, especially writing good software. The hardest part is to make things simple, even harder is to make things simple AND flexible. The need for a thorough analysis is greatly underappreciated.

    Incompetent developers tend to make things more complex than necessary. From that point on, under economic pressure, workarounds are needed to get things done. This in turn makes things even more complex than necessary. THAT is what makes writing software hard. The problem is, it is difficult to be aware of the skills that we lack. As such, a lot of programmers with a huge ego don't deserve one.

    I'm not into Extreme Programming per se, but I've noticed that if multiple people look at a piece of software, chances of problems going undetected get smaller and smaller. Yes, even if you, a master programmer, show your code to a rookie, the chance of bugs going undetected will reduce. In fact, it will inevitably result in more bugs being detected before rolling them out to customers.
  • First Post! (Score:5, Funny)

    by Harmonious Botch ( 921977 ) on Saturday February 03, 2007 @06:15PM (#17876662) Homepage Journal
    Two of us typed this. We thought it might be faster.
  • And as Mattel once pointed out, "Math is hard!"
  • Product managers... (Score:5, Informative)

    by osolemirnix ( 107029 ) on Saturday February 03, 2007 @06:21PM (#17876706) Homepage Journal
    I don't think I can fully agree. I think software development may be hard, but that's never the main reason projects fail. The main reason projects fail in my 10+ years experience is because of product managers, not coders.

    Product managers I have seen (and I have seen many) often don't know zilch about technology, but even worse they usually also don't know much about their market, target audience/users, User Interfaces, project management, etc.
    Consequently they simply don't know what they want and aren't able to explain it in one coherent paragraph of sentences. Once they would be able to explain it, the actual coding would be half as bad.

    So if this guy complains that their projects back in the days at salon went bad, I'm not suprised. He's not a coder after all, he was a typical clueless product manager - started out as a journalist and suddenly he was responsible for a type of product he knew nothing about: CMSs, in addition to having no other qualification in software development or a related area (UI design, project management).

    So am I surprised this project didn't succeed? LOL, of course not.

    You wouldn't let a journalist build a space shuttle or a car now would you? But software? Sure, software is easy, anyone can do it. In the end, it's probably not harder than building a car, but not easier either. it just takes proper skills for all roles in the team, is all.

    • "Sure, software is easy, anyone can do it." ..which is why we get the perennially insightful thread titles about "why software is hard" followed by a zillion-posts saying, essentially, "no shit." Really raises the question of if only one could get management lackeys to simply understand, "hey, software IS hard!" perhaps we could get on without having to perpetually explain why. I mean, you don't go to your surgeon and say "hey, Doc, why can't I just get a bunch of community college kids to swap this heart o
    • by jskiff ( 746548 )
      Product managers I have seen (and I have seen many) often don't know zilch about technology, but even worse they usually also don't know much about their market, target audience/users, User Interfaces, project management, etc.

      As a product manager, I think you're right to an extent. If a product manager doesn't know who the product is being built for and what that customer needs, then the product is doomed to fail. If they know those needs, but can't communicate them to their development team, then the
  • by Peaker ( 72084 ) <gnupeaker AT yahoo DOT com> on Saturday February 03, 2007 @06:23PM (#17876736) Homepage
    The software world is in a very poor state indeed.

    I think that once someone improves the situation of software architecture and programming languages so that programmers don't have to mess with ad-hoc hacks but instead write the logic that they want to implement, then software will cease to suck.

    The main problem is Operating Systems architecture and Programming Languages.
    Due to lack of time, I will only list a few of the Operating Systems problems that weren't solved after more than 30 years of OS development:
    1. Don't allocate resources sanely. One program (even worse when it has many threads) that is wanting more memory and more CPU will get the entire User Interface to a halt, even though guaranteeing the required resources for a smooth UI is so cheap. (i.e: Instead of guaranteeing 0.5% of the memory/cpu to the UI so its always smooth, even this 0.5% goes as an extra 0.5% boost to the program that's already got 99.4%)
    2. Offer an unnecessarily(historically) complicated model to programs, where there are multiple spaces of memory (malloc'able/sbrk memory, and file system space), even though these memory types are actually interchangable and when you malloc, your RAM is moved to disk, and when you use a file, it often allocated RAM. Instead, operating systems should just expose one type of memory, that is always non-volatile and persistent, so that programs don't have to worry about converting/serializing back and forth between these memory types.
      This would also get rid of the unnecessary bootup/shutdown sequence all programs are currently dealing with.
    3. Does not offer a high-level world of network-transparent primitives, that allows all method calls to transparently run over a network. If this existed, we would not see the abomination that is web-forms+AJAX and the rest of this ultra-complicated world that still does not work nearly as well as local GUI's. Instead of extending the web to support GUI functionality (poorly), we should have seen GUI's be extended to transparently reach over the network. The X protocol is similar, but not good enough as it transmits too low-level primitives (pixel data and mouse movements) and is also an alternative and not a standard GUI API that the operating system offers.
    4. The security model, using users, groups and assigning those to objects is of very rough granulity, requires a system administrator to modify the model (users/groups) and does not allow fine-grained control over the access of entities (processes) to objects (i.e: As a non-administrator, I cannot prevent my mp3 player from accessing the network or deleting the files it can read).
      Instead, a capability-security model should be used (not POSIX capabilities, but EROS/KeyKos type ones), which is much simpler to use, verify and much more powerful and fine-grained. This would also facilitate secure movement of components between computers - which could be done automatically by the OS to improve performance. More on that on a later post.


    • Number 2 really isn't desirable. The overhead would be quite painful for a lot of programs that need realtime performance. Also, clearing ram is desirable sometimes, such as when a program fails to maintain an invariant and then goes into an infinite loop. If the variable is persistant and nonvolitile, we have problems. Also, some data *shouldn't* be persistant, like passwords.
    • Um, not everyone programs on and for a Windows machine in C.
    • I like the general idea. In C you have the "static" keywork that makes a variable keep their values between calls of a function.
      I would not propose an "always persistent" memory, because too many calculations are temporary, but a "persistent" keyword in the most used programming language would be a very nice thing.
      However, there is the entire databases thing, they use another paradigm that should be taken into account.
    • by mcrbids ( 148650 ) on Sunday February 04, 2007 @02:24AM (#17879266) Journal
      Addressing your points:

      1) How do you know a GUI application from a non-GUI one? What about programs that are run locally, but viewed remotely, and vice versa? What constitutes a "GUI" application?

      2) But you are allocating different types of "memory"! See Leaky Abstractions [joelonsoftware.com] for more information on this. Your "everything is memory" model sounds nice, but lacks a few key components.... When I fclose() a file, I have a STRONG assurance that the file has been saved and wouldn't go away if the power failed. That's not the case in your "everything is memory" model...

      3) You are either talking about a security nightmare or pixie dust. How does computer B know that it's OK to run code from computer A? See other comments on #4

      4) Capability security requires somebody to set up all those !#@!@# permissions. POSIX, by contrast, is very simple and requires little effort to maintain. Is POSIX ideal in all situations? No. But it's adequate in most circumstances without a lot of effort, and it's usually better to have a "just barely suits" possibility with a decent default than a perfect possibility with a lousy default. Perhaps that explains why your touted EROS operating system died on the vine?
  • by argoff ( 142580 ) * on Saturday February 03, 2007 @06:25PM (#17876764)
    One thing I've noticed about companies is that they try to treat programmers like factory workers. Expect each one to be interchangeable and jump in anywhere on the "assembly line" at any place at any time for any piece of code. However, programming takes understanding, and complex programming takes complex understanding. Even a good programmer fixing a bug may need to analyze surrounding code for several hours before changing a single line.
    Unlike most engineering projects that are completed and done, most programming is a living growing process that is constantly changed modified and improved.

    That implies that there is a need for specialisation and clear boundries, to assign "ownership" or "territory" over certain parts of code. A programmer who understands it and gets the last say on how it's changed and have clear non-arbitrary rules for changing that "territory". Like in open source projects. If you want a kernel fix, you submit it to the proper maintainers, or make your own fork, but no corporate bureaucrat comes along and micromanage how the code is merged and managed.
    • Re: (Score:3, Insightful)

      by zymurgy_cat ( 627260 )
      Unlike most engineering projects that are completed and done, most programming is a living growing process that is constantly changed modified and improved.

      Most engineering projects (and software projects...hell, any project) are like children. They're never completed and done. Once you give birth to one, you're stuck with it for a long time....
  • by Bright Apollo ( 988736 ) on Saturday February 03, 2007 @06:54PM (#17876978) Journal
    Implementing a good design is usually half the battle. Creating a good design is usually the other half, but in practice, a solid design is almost always the part that gets skipped. Let me bore you with a brief anecdote.

    I have a large, global project underway. User requirements are done and have been done, and we're turning those requirements into things we can code or deliver ("View a workorder", "Print asset detail", "Group revisions into single document"). Of that, we have 150 odd deliverable items, not to mention all the fit/ finish work we may have to do, and all of this barely touches on reports, security roles for users, etc.

    The reason we're going to make our date, despite the 1280 discrete requirements we need to test, is that we've taken the time to look at the requirements from a few different angles and come up with a solid design plan, before even thinking about implementation. Each piece will build on another, really hard parts are identified early, blockers and such are flagged ASAP. We know things will emerge that we didn't expect, but we've got the biggest chunks identified and working together on paper. We have the flows mapped out, exceptions and variations listed, and a user group that has to sign off on every iteration of the incremental build (we're spiraling out functions and features).

    The only thing "hard" about all of this is the incessant thinking about the details, and discipline required to focus on the un-fun part of software construction, i.e. the planning and design walkthroughs. The itch to code something already is growing, but delayed gratification means that when the time comes to actually write something, the design will almost certainly lead to a working, if not optimal, solution. We can refactor as we go, but it needs to work completely before it can work efficiently.

    I've been following Chandler off and on, somewhat through Spolsky's references to it and some stray links around the web, and sounds like design didn't go deep enough into what it'll really take to build some of the pieces.

    -BA

  • Once again... (Score:5, Insightful)

    by etnu ( 957152 ) on Saturday February 03, 2007 @07:02PM (#17877018) Homepage
    Why doesn't anyone complain about how hard brain surgery is? Why doesn't anyone complain about how hard building space exploration vehicles is? Why doesn't anyone complain about how hard creating a successful marketing campaign is? Software engineering is difficult because it's a complex subject that takes a combination of intelligent people and training to produce good results. Just because businesses are too stupid to realize this doesn't make the problem go away. You can't throw complex projects at untrained, stupid, incompetent people and expect them to produce quality software. You can't just invent some magic formula for software development that will work 100% of the time to maximize efficiency. Software engineering is NOT manufacturing. Accept it and move on for fuck's sake.
    • by jd ( 1658 )
      Software isn't hard, and neither are brains. They are both soft and squishy.
    • Re: (Score:3, Interesting)

      by aXis100 ( 690904 )
      Software engineering is NOT manufacturing

      I'd argue that most software projects are far from "engineering".

      Here we are, using a construction kit (modern OO language) that is 100% predictable and designable, yet beta software and patch cycle after patch cycle are the norm. You couldn't do that when you build a sky scraper - they get it right the first time. It's the level of design, prototyping, QA and intrinsic belt-and-braces attention to detail that is missing in most software projects.

      That said, I don't
      • Re:Once again... (Score:4, Interesting)

        by kestasjk ( 933987 ) * on Saturday February 03, 2007 @10:48PM (#17878250) Homepage
        The simple fact is there are no analogies for software development, software, or the software business.

        This thread is full of analogy after analogy.
        • Software dev isn't building dev; the building can't be used incomplete, the building won't have to be changed, the building doesn't have to inter-operate with and depend on other buildings.
        • Software dev isn't like engineering cars or spacecraft; there is no finished product in software, and again you can use software even when incomplete.
        • Selling software isn't like selling cars; cars can't be copied
        • Selling software isn't like selling music/books/any other IP; other forms of IP are usually only used once, software is the only usable system that is entirely IP.

        I think if you want to have a discussion amongst people who develop a software you have to ditch the analogies, because none apply. The reasons software development takes longer that you might think, or the reasons software is difficult to create, sometimes doesn't give the expected return, sometimes is buggy, etc, doesn't have anything to do with cars or buildings or spacecraft.
  • Even dealing with a high level language like that in Adobe's Director, there are too many unknowns and edge cases and it is not painfully clear just how everything plugs together. Building a structure for doing all the tasks you need to accomplish is also paramount. As I said, "The structure allows you the luxiry to focus on the details". Build the structure.

    Cheers
  • I think that possibly one reason that software isn't getting better is that hardware hasn't caught up to software in terms of abstraction.

    Why is there a hardware stack with special instructions for supporting pushing and popping data onto it? It's not absolutely necessary.

    Why are there certain data types like int, float, etc?

    Why is there hardware supported virtual memory? Didn't that start as software? Why didn't it stay that way?

    It's because these are extremely useful abstractions over bits. The hardwa
  • No Silver Bullet (Score:5, Informative)

    by KillerCow ( 213458 ) on Saturday February 03, 2007 @07:51PM (#17877308)
    This question was asked and answered in 1986. Why is it on Slashdot as if it was a new idea every two months?

    No Silver Bullet: Essence and Accidents of Software Engineering [wikipedia.org]
  • by Beryllium Sphere(tm) ( 193358 ) on Saturday February 03, 2007 @07:52PM (#17877316) Journal
    Fred Brooks had much the same material in _The Mythical Man-Month_: communication overhead spirals out of control in large groups, project scope creeps out to infinity without a budget, overconfident people try to do too much and fail, it's impossible to know what the customer wants and (in a new area) even what works until you've built something and watched how it fails, only make change to known-good baselines, etc.

    This author had to discover Fred Brooks after he'd started a career of big projects. TMM should have been in his school curriculum.
  • Everybody knows some footballers are worth a million dollars and others are not worth a fig, but somehow hardly anyone realises that some programmers are worth a million dollars but with some others it would be worth a million dollars to get shot them (see the daily wtf [thedailywtf.com] for details).
  • by 3seas ( 184403 ) on Saturday February 03, 2007 @09:12PM (#17877784) Homepage Journal
    Try doing advanced math using the roman numeral system (no translation).

    The analogy is that programming is today being approached in a manner that is far more limiting then it needs to be.

    There are those who claim programming is nothing more than mathmatical algorithims, but it is more as programmers create higher level abstractions to deal with lower level abstractions faster. Sure it can be all boiled down to mathmatical algorithims and even down to binary or machine language but it is the higher levle of abstraction where software is today created.

    The science of software has failed or been distracted from the genuine objective of identifying and defining abstraction physics. For it is Abstraction that is the essence of programming, and there most certainly is a physics that applies to our creation and use of abstractions.

    We create abstractions in order to simplify or automate complexity of lower level abstractions, down to binary.
    The failure is that of not recognizing what we all constantly do, what action constants we apply in our creation and use of abstractions.

    Its like doing chemistry before we came up with the understanding to create the table of elements. We didn't understand the underlying mechanics. But once we understood these underlying mechanics, we created chemical megaplants.

    Though we would not create software megaplants, in understanding abstraction physics, we would do what was accomplished with teh conversion of math from roman numerals to the hindu arbic decimal system. We'd make programming easy enough that the adverage user would do alot more for themselves, just as the general population was able to not only do math for themselves in teh conversion of symbols used (roman numerals to decial) but were able to do more advanced math then the roman numeral elite accountants were able to do.

    Of course the problem is in conversion, as it took 300 years for the conversion to happen. It took 350 years for Galelio to be exonerated....ask the catholic church why... and know why the industry of programming, and regardless of what side of the fence you are on (proprietary or open source), presents resistance to the needed change.

    Programming is hard, because the industry wants it to be. So to keep the elitism, social status and pay scale.

    Of course social demands weigh in on the change happening. As computers today could not have been created using the roman numeral system of math. It won't be hundreds of year for this change to happen, as we already can't keep up using the lessor/harder route.

    Abstraction Physics [threeseas.net]
  • Mythical Man Month (Score:5, Insightful)

    by tedgyz ( 515156 ) * on Saturday February 03, 2007 @09:29PM (#17877874) Homepage
    Read it. Know it.

    Why must we rehash this over and over again.

    If it were easy, then yes, we would have push-button frameworks that magically created programs. What bothers me more than the ignorance of the "no silver bullet" mindset, are the pushers of programming environments that supposedly will solve all our problems. Bullshit. Corba sucks. EJBs suck. All these things suck. Just write good software and stop looking for the golden chalice.
  • by mrnick ( 108356 ) on Sunday February 04, 2007 @05:25AM (#17879806) Homepage
    Software development is hard because of a misconception that knowing how to program in a particular programing language makes someone a good developer. Just because someone can learn a foreign language (i.e. an American who's native language is English can learn to read and write in French) but it does not mean that person will be able to write a good novel in that language.

    I'm currently working towards my Masters in computer science and although I don't intend to ever work as a software developer programing concepts make up the majority of the curriculum. The majority of the graduate program, in computer science, is made up of international students (the university is in the US) of whom most hail from India. After working with these students for some time now I have learned that they obtained their undergraduate degrees from India where computer science is taught in theory. This means that they got their degree without ever touching a computer. Combine that with the fact that they have never owned a computer themselves makes me doubt the quality of their education. For those of you that know some C++ you will understand their level of knowledge when I say that they enter the program without understanding the concept of pointers (dynamically allocated memory instead of compile time arrays), structures, or object oriented programing concepts. Luckily many of these students do not make it past the remedial courses of the graduate program but most of them switch to the IT program under the business college of the university. Unfortunately I am sure many of these students go on to be programmers in their country where there is a big demand due to the trend to outsource coding projects to their country.

    I have spoken with other students in similar programs at different universities and this seems to be a widespread scenario. So with people that barely understand the language they are programing in are being asked to write programs for consumer use. It seems that very little time goes into educating students on how to program. Here I am using the word "program" to mean given a problem utilize one's analytical skills and artistic ability (yes programing takes artistic ability) to conceive a solution and write it in a programing language in such a way that the language compiler and linker produces an executable program. With this in mind you add the fact that virtually no effort goes into teaching people how to move beyond this basic level of understanding of specific programing languages to a point where any competent programmer would consider them to be fluent in the language they are developing in. Would anyone buy a book written by someone that is not fluent in the language that the book is written in? I think not.

    Beyond the fact that the majority of programs are being written by people who are not competent, by any standard, is the fact that programming is not just an application of knowledge of a specific language but an abstract concept that is created from the individuals understanding of the problem and their ability to conceive a solution entirely from within their own mind. This is where the artistic ability comes in. If you give 100 people a problem and ask for a solution in the form of a program you will get 100 different solutions. Out of those 100 people only a handful of the solutions will meet the standards of what could be considered efficient coding.

    Consider that the programmers that I am speaking of here are people that entered into a post graduate computer science program. These people do not make up the majority of programmers out there. What makes up the body of the worlds programmers are people that have not attempted to progress to this level. Most enter the workforce after receiving their undergraduate degree and many have not received any higher education at all. It is my belief that aside from what can be taught to an individual programming requires an inherit ability within that individual for them to be able to produce what I would call quality code.
    • by ralphdaugherty ( 225648 ) <ralph@ee.net> on Sunday February 04, 2007 @06:03AM (#17879940) Homepage
      If some barrier would have kept CPU speeds below 100Mhz then I imagine that by now people would be developing very efficient code and that we would still have the same level of application performance that we enjoy today with our dual core 2Ghz processors.

            Sorry to break this to you, but apps today are not faster than the ones they replaced. We used to write more efficient code, and had more spartan interfaces, because we had to. But I don't think Microsoft Office today is noticably faster than the WordPerfect / Lotus 1-2-3 type apps we wrote in assembler under DOS. All that extra CPU power is more than eaten up by layer upon layer of slower and slower software.

            All this allegedly to make programmers more productive. I haven't seen that either.

        rd
  • by El Nigromante ( 1059332 ) on Sunday February 04, 2007 @05:48AM (#17879896) Homepage
    The article's title gives an indication of this (as some other comments have pointed out): it talks about "programming" but not "software engineering" as a whole.

    Still many companies hire people with not enough computer science knowledge, for performing software engineering tasks. You can do this, but the results cannot be good (at least in the long term).

    I think this is because software is usually successful, in the short term. It apparently solves the problem and the customer gets satisfied. Therefore, why "losing" time and money making documents (where experience gets archived) or performing a good design?

    If you create software, how often do you (your organization) apply these concepts?:

    - Life cycle of a project,
    - Gantt and Pert diagrams,
    - Risk management,
    - Ishikawa diagrams,
    - Code complexity,
    - Software quality assurance,
    - Coding style standards,
    - Documentation standards,
    - Software patterns...

    Have you ever wondered why Linux is still failing to widely conquer business marketplace?

    Satisfaction - both for companies and indivual programmers - should be switched from being creative to getting a good job done. We still have much to learn.

  • It's about people (Score:4, Informative)

    by roman_mir ( 125474 ) on Sunday February 04, 2007 @12:36PM (#17881496) Homepage Journal
    Delivering Good Software is difficult because of the people involved.

    By the way, I read the article and this paragraph:

    Disclaimer: Scott Rosenberg was responsible for Salon's hiring me 10 years ago, was my editor and boss for many years, and is a close personal friend. My daughter baby-sat his twin sons as I "interviewed" him. So I'm utterly biased and completely partial. But "Dreaming in Code" is still a darn good book. - is one of the reasons why software sucks. I have seen too many cases of friends hiring friends not because the people being hired are the best that could be found (or that interviewed) for the job, but because they are friends of the friends. This touches on all avenues of life and business, the best people for the job are not hired, because friends of the friends are hired and those people more often than not are not going to do the job right.

    One should either recommend a person for a position or interview the person, but not both.
  • by mach777 ( 1021209 ) on Sunday February 04, 2007 @01:30PM (#17881850)
    Let me tell you why it is hard.

    A person needs to cross a river, and so believes a bridge needs to be built.

    What he should be doing is ask: "I need to cross this river, can you help me?"

    Instead, he asks someone to build a bridge, a bridge of this and this dimension so his particular vehicle can pass, and a bridge with this and this feature because "it would be nice if..". He also wants a bridge that looks in this and that way, because since he is paying for the bridge he wants it to look the way he wants it to.

    First he was crossing a river, he is now building a bridge. Will the bridge help him cross the river? Who knows? Maybe a bridge like he imagines can't be built, or only be built poorly.

    Its the same thing with software. People want the strangest things. In fact, what they WANT is the only thing they know. They don't know what they NEED.

    So you get an organisation, a business, with an office that has a problem, any problem. The problem will ALWAYS occur because some OTHER process isn't working somewhere else in the organisation. You get bad data from here, and the clerk is expected to output good data out there. In the 50's I bet they wanted more filing capability or better typewriters to solve these problems, in this day and age they want IT. They want a system!

    So instead of using excel, notepad or even a piece of paper like any other sane human being would to keep track of the information, a yell echoes down the corridors: "We need software to support our business."

    So some programmers are hired. They get a description of the problem, which isn't logical in the first place, and they are expected to solve it. Their tool, software, is built using a logical language. It is used to describe the data, and solve the problem by adding a few flows for that data during certain conditions.

    So the programmer (or bridge builder) sits himself down. The first 10% of code/thought he outputs is usually all that should be done about the percieved problem. That is, a description of the Need.

    The next 90% of coding is about the programmer trying to coerce a logical language around non-logical flows and non-optimal solutions, hammering a square button into a round whole, with GUI's, buttons and special extra functions for special extra cases, and those extra wants on top that really describe other problems.

    So we will end up with a mishmash of buggy code that describes the wants of the customer. The Want to solve a problem that should have been solved with organizational changes, or changes in the work processes. But hey now everyone is happy again, software is supposed to be a bit buggy, the organisation is obviously still working non-optimally (software can't fix that), but at least the clerks now have webpages to input the bad data in as long as the servers are up.

    Optimally, the programmer (yes the programmer) should look at the problem, trace it down the whole organisation, yea trace the customer Want to the REAL problem and the REAL need, proceed to make the organizational changes and be done without an IT system at all.

    We all know that won't happen, but that is what should be done. Don't expect a logical function (software) that describes a non-optimal situation, to function in an optimal way.

    Software doesn't work that way, thats why its hard.
  • by steveoc ( 2661 ) on Monday February 05, 2007 @03:30AM (#17887162)
    .. until the release of Vista that is.

    Now, its so easy to get on the internet, do your hotmails and digital photos, and connect with others in new and unexpected ways.

    Its all about the WOW, and thankfully everything is now so much easier - including software development programming.

    And its the existence of people that swallow this sort of shit that contributes to making software hard. These sorts of people, when they involve themselves at whatever unwanted level in the process of developing software, turn out making the whole game look so much more difficult than it really is.

    And then one fateful day when the mangled bastard children of their best creative efforts needs to be interfaced with - then yes, at that point in time, software development truly is a difficult thing.

Where there's a will, there's a relative.

Working...