MIT Studies Software Development Processes 345
IsoQuantic writes "A new MIT study (pdf) looked at SW development processes around the world. One striking difference that the researchers found for U.S. developers is the relatively small use of specifications before development begins. I can already hear my EP-zealot colleague chuckling in the cube next to me. (sigh)"
Not for me. But we learned (Score:5, Insightful)
Specs!? Specs!? I don't need no stinking specificiations!
I used to work in a very loose development shop. The only specifications that were ever written down were protocol specs - and even those were often "documented" in the form of a header file.
I found some parts of this method usefull in that the specs were often written as as pseudo-code comments, and the actual code would be filled in later.
However, eventually the development pool grew, and we got a few folks who couldn't follow this method, and we lost several weeks of work. After that standardized specificiation paperwork was produced for every project from that point on.
Perhaps it's a lesson everyone will eventually learn. Perhaps I'm being an idiot.
Re:Not for me. But we learned (Score:3, Insightful)
Re:Not for me. But we learned (Score:4, Funny)
Re:Not for me. But we learned (Score:5, Insightful)
a) The purpose of the integration (business concepts)
b) The protocol
c) Examples. Lots of examples
makes the whole process a lot easier.
Especially the business concepts are important. They allow me to foresee where changes and extensions may occur, and I then put more work into those parts.
If you're fortunate enough to have a good project leader, use him to communicate with the other parts involved and make him also document all the changes in the protocol. That will save you a lot of time on the phone and quite a few 'tail -f
Re:Not for me. But we learned (Score:5, Insightful)
I highly doubt the "customer" is going to deisgn great software.
for example.
A thousand customers could say I need a tool that lets me input the various factors of my budget and look at what the sums will be.
But how many could say - I need a program with a grid of flexible cells which can hold a value or a formula?
AIK
Re:Not for me. But we learned (Score:5, Insightful)
SPec documents are the "what" the customer wants.
Design documents are the "how"...
Re:Not for me. But we learned (Score:4, Insightful)
Customer says he want a report - but in reality he want to understand his business.
An OLAP server can help him understand his business, a crystal report can start him down a long road of report modifications which in the end will lead him to 1. an unmanagable pile of confusing reports or, b. an OLAP cube.
The design process is to understand the need of the customer - not have the customer specify how or what the solution will be.
If you provide the solution the customer specifies - you will be run out of business when he reads the next business journal anout how to do omething better.
You had beter know what he is going to read next if you intend on being a solution provider for very long.
AIK
RIGHT ON!! (Score:5, Insightful)
Customers do NOT know what they want. Anyone who thinks that they do, hasn't spent enough time architecting software.
What AIK is saying is that you have to dig deeper than the customer requirements. You have to understand the space. You have to look at competitive products. You have to anticipate unstated needs. You should ask, "When I'm all done, and everything is working perfectly, what changes will the customer want IMMEDIATELY?"
I can't stand people who listen for five minutes and start to write "use cases" right away. That works for some dumbass web site, maybe, but certainly not for any involved product design.
Architects need to plot an intercept strategy several YEARS in the future. That's how you build successful software. When the customer puts his Phase N requirements out for bid, and all your competitors run for the hills, your design has anticipated his needs. You've built metatables instead of tables. You've used OLAP where you could have sleazed by with an RDBMS. And so on.
Great thread.
Re:RIGHT ON!! (Score:3, Insightful)
I honestly could not see you winning any bids against todays development shops.
Now by no means am I saying always take the easy way out, but I have seen MANY MANY MANY people like you who
Re:RIGHT ON!! (Score:4, Insightful)
The data that the authors of the articles collected seems to indicate that using either the standard waterfall technique (with its heavy dependence on specs) and using newer methods (what they call sync-and-stabilize referring to the heavier dependence on building product and having that product used by the users) result in the same level of bugs, and the same level of developer efficiency. The real interesting part of the whole article is that it seems that the Japanese are doing something truly innovative as they have bug levels that are a scale of magnitude smaller than everywhere else!
Re:Not for me. But we learned (Score:5, Insightful)
The requirements process is where you get your specifications from the customer about what he or she wants. Design is a completely seperate stage. The requirements are something you both agree on, but its not just something the customer sends to you. You give them feedback about the requirements. Perhaps they are contradictory, perhaps there are better ways to do things. This part is crucial in that the more time you put into this, the more information you will fish out of the customer, and you'll be more confident that you and they will know what the software is supposed to exactly do.
The golden rule in software engineering is the requirements are going to change. You just have to accept it. Why do requirements change? Well, usually its because the customer finally realizes that what they got and what they actually wanted were two different things. And thus you make the necessary changes until the next time they come back looking for you...
How many times have you downloaded a program think it has everything you want, until you use it and then realize theres something more it needs?
Think about something like Mozilla. It's be a sufficient browser for a while now. But once people got it, started using it, they thought to themselves "Now I need tabs!". And thus the evolution of software...
Re:Not for me. But we learned (Score:3, Interesting)
I implemented OLAP cube with an excell frontend embbedded in a webpage such that all users in a multistate company could view the OLAP live from the website - and it took LESS time than any crystal report - and it was precisly what the company had been hinting at for years - only they didn't understand what it was they wanted.
Yes - I'm arrogant if it means anything to you.
AIK
Re:Not for me. But we learned (Score:5, Interesting)
In my experience, RUP seems far better. The customer rarely knows exactly what they want. At best they'll have an idea of what they want it to do and can visualize how they think it might work. So you start with a loose set of requirements, do some quick high-level design, and code it. Then show the customer what you've got. They can identify what is different from what they want and see some of the issues they may not have thought about. This defines some lower level requirements and designs. Repeat this process on a regular basis and you will get good, well-written, and efficiently created software.
I've had nightmares with the Waterfall process. I've had to spend 6 months writing requirements and design docs, formally releasing them, then have 2 weeks left to write the code so the software wasn't nearly as good as it should have (or could have) been. In the end, the requirements and design had some inconsistencies that could not be foreseen until you try to code them.
On another project (that I was not involved in) all of the money went into the requirements and design process and the end software was crap, wasn't user friendly, and didn't get used. But it did meet the requirements.
The problem with writing requirements completely before coding is that you have to basically do the design and coding in your head as you write the requirements, otherwise you can end up with things that are inconsistent. Written language doesn't have to be self-consistent, software does.
On the otherhand, writing software without following any guidance from the customer is definitely worse.
Re:Not for me. But we learned (Score:5, Interesting)
Requirements are what the customer wants.
Specifications are what he's actually going to get
Design is how the analyst thinks it should be done.
Re:Not for me. But we learned (Score:2)
Aren't you counfounding requirements with specification? You can have a requirements specification and a design specification.
Re:Not for me. But we learned (Score:5, Funny)
Not Really (Score:3, Interesting)
My customers don't know what they want. They don't really understand how their own business works and they feel lost because as their business is growing they lose track of the knowledge that was very easy to keep up with when their company only had 20 people.
It depends on your software and your client base, I guess. When I program for hire it tends to be CRM and HR packages for small businesses that have outgrown Quicken, so I guess it's understandable that I see a lot of confused people who honestly jus
Re:Not for me. But we learned (Score:3, Insightful)
Re:Not for me. But we learned (Score:2)
Aren't you really saying that the specs have to handle the "least common denominator"? In other words, they have to be understood by the dumbest person in the group.
Now think about it and I'll bet you produced more and better code with the smaller group!
Re:Not for me. But we learned (Score:3, Insightful)
Honestly, most Dev shops don't have "code librarians" anymore. When they did - this was the person who would make sure that code didn't get duplicated. If a current function could do what needed to be done with very few changes, so be it.
What happened where I was - some people who were not well trained in the code base started re-creating a large number of functions that we already had, and did so in an otherwise incompatible way. Yes, this should have bee
Specs and outsourcing (Score:3, Insightful)
Re:"As little as possible, but no less" (Score:3, Insightful)
If the customer has a million expections, but your Lead or Project Manager has not told the customer what to expect from their current "official" requirements, then you will have a disappointed customer. However, if you let the customer know, right from the start that the project will do exactly what they ask, and no more - then "extras" that were designed in through savvy programming, code re-use - or interface standa
No problemo (Score:5, Funny)
How Ironic (Score:3, Insightful)
Re:How Ironic (Score:5, Insightful)
Requirement are how offshore houses get paid for uninspired work.
You want software which will
a. act like software already created by someone else (knock off)
b. to be a metaphor for existing (and fixed) paperwork based processes
Then you can consider a rigid design - which means you can consider outsourcing - but I ask, what novel piece of software was invented in a developing country?
AIK
(BTW the add on this page flashes my curser and disables my delete key - adds ok - flashy adds which interfere are a bit much)
Re:How Ironic (Score:3, Informative)
I strongly disagree with that statement. I work in an software consulting company in India and I have seen examples of great pieces of software being designed in India. Mind you, these were not your run of the mill software projects, these were complex projects which solved complex business problems at the very heart of the clients business. Just
Napster, Gnutella (Score:4, Insightful)
Re:How Ironic (Score:3, Insightful)
My position is that specifications can hide the need for flexibe architecture.
I like your golden rule.
I worked at a company that basiclly fired me for automating the code generation of mid level tiers because it took
Re:How Ironic (Score:5, Insightful)
The best trick I've ever seen for setting project goals is to sit developers down with business users or the end users of the product and have them observe work in action. This understanding can go a long way when a developer is back in the 'cube' for long hours of coding. Domain knowledge is critical to a developer. Otherwise you end up with some trully epic program that bears little to no pracitcal value.
Requirements should be clear, reviewed often, and adjusted as necessary. Which requires a good project manager to really pull off succesfully.
All in all programming is really a mixture of a lot of aspects and any one not in balance with the others will skew the results. Sometimes you'll still get something usefull, but more often then not the results are hideous. And management will always blame the developers and developers will always blame management, with the only difference being the developers don't have the ability to can management.
Re:How Ironic (Score:4, Interesting)
Requirements that are too rigid are bad. But so too are requirements that are too loose.
I like requirements that are rigid on the edge, that is where they meet the world.
But then give me free rein to change ideas for the internal structure.
Of course, it always helps if you know your customers well enought to anticipate the inevitable change request that will come down the pike. Then, you'll see if your internals can be readjusted elegantly quickly to accomodate the new exposed services they want.
It's important to listen to customers, not just to hear what they're saying explicitly, but to get an idea of where they're going and what they might want in the future based on what they believe is important and valuable. Hint: customers/users may not just tell you explicitly what they think is important and valuable.
Re:How Ironic (Score:3, Interesting)
There's an even better trick: put your development team in a shared workspace. Then make a domain ex
Re:How Ironic (Score:4, Insightful)
Thus illustrating some maxim or another...
Re:How Ironic (Score:2)
One interesting result of this approach is that you'll find projects that become unfeasable due to the fact that their value to the customer falls far below what you're willing to invest in the form of time, staff, etc. Not to mention that, at least in the consulting world, there is a tendency to "meet a client half-way" when doing estimates, rendering them even more inaccurate than they were to begin with.
Re:How Ironic (Score:5, Insightful)
As someone who teaches SE at University, I say "amen" to the parent :-). Grandparent's teacher is either being misunderstood (likely) or an idiot. A post farther down in this thread points out that distinguishing functional from non-functional requirements is, uh, kind of important. So is understanding the idea of a "design constraint".
Of course, "requirements are the enemy of design" sounds pretty cool, and it is true in a way. The important thing to notice is that at the end of the day, requirements always win. That is, no matter how cool the design is, if the product doesn't do what it is supposed to, it is useless. If the product meets requirements, it really doesn't matter what design got it to that point: it's fine. Given that, you might think that it would be a good idea to know what you need to build before you start designing it. You'd be right.
Grandparent, make sure you aren't completely misinterpreting your teacher's position. If you are not, drop the course. In grad school, I dropped my SE course because my (mighty famous) teacher was so ignorant. It didn't preclude me from learning the field to the point where I teach regularly in a professional SE MS program.
Re:How Ironic (Score:4, Insightful)
Funny, I've had the opposite experience: my CS professors always wanted carefully planned projects with specified scope and length, but my boss has me working on an open-ended project that will probably go on basically forever. The more real-world development experience I have, the less impressed I am with any of the "software engineering" doctrine about planning and specifications, and the more value I place on just writing the damn code.
Re:How Ironic (Score:3, Insightful)
Have you looked at the stats on how many software projects actually come in on time and on budget?
As a profession, we'd be better off being honest about the fact. But it doesn't fit tidily into management spreadsheets, so an awful lot of people just make up numbers. Or worse, they are browbeaten into making the est
Re:How Ironic (Score:4, Insightful)
What is needed is for more IS analysts to actually work for the line organization, rather than within the IS department. When the analysts sit within IS, it's far too easy to lay blame on the users.
Re:How Ironic (Score:3, Funny)
Sounds alot like the discussions me and the compiler often have, though I tend to use more expletives....
Re:How Ironic (Score:5, Insightful)
And the Open Source Community best represents this, because a project never ends, but continues to develop in a myriad of directions.
Yes, but thats both a blessing and a curse. The commercial customers want stability, not a build release everyday. Even amongst the Linux community, how many people use the latest release of any software? Most people stick the the most stable release - I still use Debian Woody.
I have no problem with development, but the Open Source community should follow Debian's model and not release something (read Sarge) unless they're really sure its all done, and not release a version for every time feature add or small patch - have the fixes and patches as seperate entities and not as builds.
This was a problem that was told to me by the CEO of a certain reasonably big product development software company - he felt that this lack of stability (or the perceived lack of) is what is scaring corporate customers away. And he was like, if Redhat withdraws support for their old distributions, it is indirectly asking us to upgrade - and that is not stability.
Believe it or not, a lot of people out there compare stability to what IBM Mainframes provided - and sure, its not cutting edge and what not - but guess what? In a commercial enterprise, most often, it just needs to work and work well, for a lengthy periods of time. Period.
Don't confuse development with packaging (Score:3, Interesting)
Maybe I'm misunderstanding your intent here but I'd say the release-early-release-often is a huge strength of open source development.
By showing all the warts, all the problems and solutions as the
Re:How Ironic - Open Source and Specs (Score:2)
Re:How Ironic (Score:5, Insightful)
Simply throwing out all requirements because non-functional requirements are difficult to estimate is absurd. That's why you make requirements but you make them flexible and reasonable. Of course, you can say that the customer doesn't know what they want and even if they do they can't explain it exactly. That's why you iterate through the process.
A Software Requirements Specification document may even be a legally binding document for a development team.
To say that requirements are an enemy of design is incomplete - there is a fine line between requirements that are 'too rigid' and 'nonexistant'. Somewhere there is a nice balance between requirements that are harsh and strict and requirements that are so loose they might as well not exist.
If you get the customer involved, iterate through the requirements (entire software lifecycle) process, and make reasonable requirements, you'll be much better off. Personally, I would rather have in writing what the system should do than just have some kind of vague idea.
Code is specs (Score:2, Insightful)
(For What?)
The only perfect language for software specs is an unambiguated, testable, logical language.
The end result of this logical posativism IS software languages.
Software languages existed before the computer, and unambiguous processes could be described unambiguously only be using these unambiguous languages.
So I suggest the software IS the specification - and its testable.
If you write confusin
Re:How Ironic (Score:5, Insightful)
You can't honestly expect someone to pay you $50-100/hour if you can't give an estimate. So, what is required to make a qualified estimate?
IMHO:
1. Gut feeling
2. Some experience in the field
3. Add enough time to cope for silly things in your code that stops progress now and then (Those really hard to find in-your-face bugs)
4. Realise there's a difference between time worked on the project and calendar time. You'll find you're using a lot of time waiting for others, configuration management (just getting through to a test server might take days, if the people managing the routers etc are incompetent,overworked, unwilling or just plain lazy or you're introducing some new infrastructure (like a new app server) etc.
5. Divide the project into phases. I usually estimate time to make initial end-to-end contact if it is some kind of integration job, then the main development phase, and finally testing and deployument.
6. Google for known problems with the systems you are using. (I once did a job involving MQSeries, Java and Linux. A google search helped me identify the problems I might face.)
6. Add 20%
The true professional plan: (Score:4, Funny)
7. Add 20% (I'm almost there...)
8. Add 20% (Just another two weeks...)
9. Add 20% (Darn these last minute bugs...)
10. Add 20% (Testing takes time, you know...)
11. Add 20% (They want "web based" now...)
12....
Re:How Ironic (Score:3, Insightful)
Requirements aren't the enemy but the boundries of your design. You need the requirements to know what to test. If you don't know how many users a system should support, how fast a user should be helped etc. you can't determine when a project is finished or has reached it's goals.
Requirements and specs form the basis of the box within you create the application and your design. Th
College Professors (Score:5, Insightful)
In my software engineering class, my teacher vehemently states that Requirements are the Enemy of Design.
Unfortunately, a lot of college professors are out of touch with reality. Software development is such a diverse area that you really can't generalize.
For example, I worked for a long time in embedded firmware and digital signal processing, both on mega- and micro-projects. You need to design to requirements for these. There can be creativity in how you impelment a design, but the bottom line is the spec. If you don't design to the spec, the satellite falls out of the sky.
Currently, I am working in multimedia, and we don't really use specs. We have high-level goals, but even these are fuzzy. Here, requirements are more of a hindrance, but we still have to draw the line in the sand for some things.
Gee ... and Specifications? (Score:2)
The Requirements don't have to specify memory and space, but they SHOULD specify functionality. Don't forget the requirements produced by a programmer would (and should) be different from those produced by a manager.
When you are working to provide a customer (boss/division/consumer) with a product (program/library/etc.) you need a benchmark of how to know if you succeeded. Even
Re:How Ironic (Score:2)
Re:How Ironic (Score:4, Insightful)
There is a reason why your professor is teaching rather than doing.
Most of software development in the real world is NOT a free-form pursuit of artistic expression. More likely you will find yourself working on software that is part of a larger system. And because it's part of a system, the components need to be able to work together on a number of levels. Lack of proper requirements management in a large scale software development effort is what leads to lawsuits. It also leads to solutions in search of a problem. VC-funded dot bombs that end up in the waste bin (instead of
Re:How Ironic (Score:3, Insightful)
There are clear benefits to Open Source Development methods, but blindly suggest
Re:How Ironic (Score:3, Interesting)
Good grief. Has your teacher ever designed anything in his (or her) life? I guess it's true what they say about "those who can't do".
You need to have an idea of what you are doing for the project,
In other words, you need to have an idea of what is required.
you honestly cannot know how much space it will take, how fast it will be, etc.
Those aren't requirements (except, perhaps, for an embedded
Re:How Ironic (Score:2)
Its hard to say in advance what better architecture may prove to be.
AIK
And non open source software? (Score:2)
Re:How Ironic (Score:5, Insightful)
Care to point out a tiny sample of "most" Open Source that follows this norm?
In the interest of being taken seriously I will point out a few that I know of that seem rather well thought out:
fltk [fltk.org]
FOX [fox.org]
gcc [gnu.org]
Oh yeah, here [opensource.org]
and others [gnu.org]
You were modded informative yet I see nothing informative in your post. Perhaps because I have contributed to some of these projects and I am jaded? Perhaps because more work gets done in these projects than the corporate arena of closed source that I used to work in and I might be close minded and missing your point? It's a possiblity but I will try to become openminded in case I missed the informative part of your post.
Re:How Ironic (Score:2, Interesting)
The statement is utter crap. A fair comparison would be between an open source program that has something ugly bolted on and a closed source program that doesn't have that feature at all because it's not released and you have no option to get it.
Most open source software does what it's creators want it to do. Should they take the time to design an extensible system for everyone in the world to use for any possible use? Probably not, they just wanted it to do something. The fact that they ma
That's because in the US... (Score:5, Interesting)
Or, in my personal experience, we stick to a formal document for 3/4 of the product then get hit with feature creep for the last 1/4 which makes the product late, buggy, over budget, etc, etc, etc
Sure, I've tried instituting "processes" and management's alwasy keen on the idea. But when push comes to shove, >poof.
The only time management ever stuck with a process was the medical company that, by law, required governmental oversight that demanded process. And you don't want to know how much we skirted process anyway. (Most of the times we built the product first, then wrote the "planning" documentation second.)
Re:That's because in the US... (Score:2)
Oh joy [vt.edu].
Re:That's because in the US... (Score:5, Interesting)
and they wonder why no progress is ever made
Thats why you use Aegis (Score:3, Interesting)
You can't add a new feature without first defining a test for it to pass, you can't fix a bug without defining a test that the old baseline failed and the new baseline passes.
So marketing can walk up and say "release now" or "add these features" and you can do either. But you can't "release these features now" because the system won't let you.
When marketing say "release now" they can only have the bits that
Re:That's because in the US... (Score:2)
Processes should be designed to fit the way you work, not the other way around. Trying to force everybody to do things differently than they have already been done by writing a document is never going to work.
But there are development models that try to accomodate those problems. Try to find one that fits the way your development projects unfold. For instance, Extreme Programming (aka 'Agi
Re:That's because in the US... (Score:3, Insightful)
I personally believe that there are merits and drawbacks to both approaches and that the ideal is somewhere between formal and informal approaches.
The informal, intuitive approach can result in a deliverable tha
They needed an MIT study to determine this? (Score:2, Funny)
Re:They needed an MIT study to determine this? (Score:4, Funny)
From the report... (Score:3)
Cue a lot of M$-related jokes and M$ bashing!
Productivety can jump without specs (Score:4, Interesting)
Of course, this never works in real life because managers want documents to mark the progress against their gannt charts so they always interfere with the "make sure you spec it properly", and the manager on the other team will say "dont do a damn thing until you see a signed off spec"
Shit - its no wonder commercial software costs so much.
Re:Productivety can jump without specs (Score:4, Insightful)
As a gov't employee, I understand first-hand how frustrating it can be to get useful, realistic requirements from our customers (other gov't folks). And I'll also agree that unnecessary beaurocracy can baloon development cost.
But unless you maintain good communication with your customers, you're bound to get it wrong. If you suspect they either don't really know what they're asking for, it's better to clarify that with them than to decide for yourselves what they really want.
Re:Productivety can jump without specs (Score:3, Insightful)
I'm not sure if you meant this to sound the way it does, but you've pointed out my pet peeve:
The customer is not the proper person to write the specs. Specs must be written by a skilled analyst who can observe and communicate with the customers in a manner to determine what is required.
I've seen too many projects go down the drain because they were completely useless. They were completely use
Fun read but ... (Score:5, Insightful)
And of all the praise they lavish on Japan and Indian the conclusion brings it back to reality with:
Just one? (Score:5, Funny)
Shouldn't that be colleagues? :)
Re:Just one? (Score:3, Funny)
Hooray for Freedom to Innovate! (Score:2)
Worthless Study (Score:5, Insightful)
-Subotai
Re:Worthless Study (Score:2)
The rule of 3 is (quick,cheap,quality) - pick any 2.
Re:Worthless Study (Score:3, Funny)
Maybe I just suck, but that sounds awfully like some fantastic brothel deal!
Quick, cheap quality! Pick any two! *waves arm over sea of girls*
Specifications? (Score:5, Funny)
However, they HAVE managed to change the name of the project on me at least three times, and our last two-hour meeting was consumed by a lively debate on what to call a particular form, so it's not like these critical planning issues are being neglected.
It depends.. (Score:3, Insightful)
If your development team is two guys sitting next to each other all day long, there isn't much need for very detailed specs or a set structure. You tell then what your project must have, and they deliver (if they're good).
On the other hand, the larger the team, the more structure is required; you don't want one person breaking what another person took four weeks to complete.
I think in the US, the relative lack of specs is probably because most US firms are in one location where the developers are in close proximity, making communcation quite easy (you don't even have to take your eyes off of your screen to yell over a cubicle).
Lots of big companies (Score:3, Insightful)
It would have been nice if they had included smaller companies in their sample. Probably just as well that they didn't, though, because I suspect those would make the US numbers look even worse.
To a large extent.. (Score:5, Informative)
- In America, they are in a tearing hurry to produce a prototype/model/proof of concept, which of course forms the basis of initial release ("it works, doesn't it? so why re-write it?")
- Gathering requirements is a pain in the wrong end - I've seen all sorts of instances - CFO disagreeing with CTO and re-writing everything, PO Manager introducing something new that CFO promptly over-rules. At the end of the allocated gatherings period you have a hash of what each stakeholder at the company wants. Needless to say, there is a lot of fun in ensuing months - Most American customers do not like to spend time on discussing/analysing progress or answering questions during the development (yes, I agree they shouldn't crop but, but most times they do).
- As a natural progression, the aspirations change over time. By the time it's written and demo-ed they want it to do 10 different things and pipe-up about how they had "already" mentioned they wanted this cool new feature. The signed copy of requirements specifications sure helps
- In Britain though, they pore over every single email the PM/Team Lead sends. Tremendous emphasis at every single aspect ("We are bloody payin you for this, aren't we?")
In India though, life is Sh*t. The documentation team, the process team and the internal audit teams sort of join hands to drown you in sh*tload of paperwork. It's good to have processes, but IMHO, they just get carried away and want copious detailed documents about everything and anything...Unless they develop tools to automate and regulate the processes, it is gruelling and something no-one looks forward to...err, except the audit team
Another good argument (Score:5, Interesting)
I've seen it happen way too often; the expectations of customers can be very unrealistic simply because they have no knowledge about software engineering.
Having a complete design document with two signatures can prevent 'just add this one little feature'-type problems.
When do we need specs (Score:5, Interesting)
who I know well and get along with(and talk a lot with) We can get by with practicly no specifications even with a coding which lasts several months.
You can split up the work by writing header files first and that usually does the trick.Obviously this
cuts down on development time.
However I had on several ocasions needed to join in on a project which has been going on for sevral years, and I found it much much easier to start working quickly on the projects with more specs.
I am currently on a project run by a man who is quite anal about specs and standards and documentation, and organized testing. The result is that I spend more time dealing with standards then programing but It greatly increases the quality of the code, it makes the throwing out a week of work for incompatebilty impossible, And perhaps most importantly it makes getting aquainted with a diffrent part of the project very easy.
As for EP, I have seen it work well and I have seen it fail miserably. I have not yet gathered enogh expirience with EP to identify what makes it work.
Me.
P.s I am not a US resident, nor did I study in the US.
Lack of Specs from Lack of Time (Score:3, Interesting)
Who precisely was studied? (Score:5, Interesting)
But if I look at those that aren't suffering much right now, those doing well, I see that most of the successful U.S. deveopers and shops also specify things out. But, because we don't live and die by the regulation (those being cradle-to-grave parts of much of the world), we can also be more flexible more quickly. And we can prototype from the seat of our pants quickly.
A down side of "flexibility" though is that we often get called "not team players" if we don't instantly cow to the calls from sales and marketing every time a new feature idea pops up. One of my co-workers calls this "chasing butterflies", the lack of focus that results in never finishing anything. That's the downside that leads to bugs and slow development... and failed companies. But many more successful companies, including most of the ones I've been at, have actual product life cycles. There really are two tiers or classes of development companies that way.
The question is, why are there so many undisciplined shops? I think the answer is the easy money of the past. Suddenly we (developers) weren't being managed by engineers and developers, but rather by CFOs, shareholders and sales people. As the crunch continues, I suspect corporate Darwinism will continue and we'll scale back to more methodical practices again.
EP?...EP?!! (Score:4, Funny)
So, crack open a cold one and check out my double nested for-loop ollie nose-grind, dude!!
And who says we're zealots?! ;-)
RRR
Re:EP?...EP?!! (Score:4, Funny)
Thank god (Score:2)
Maybe that is why people offshore ? (Score:2)
Maybe the lack of specs is why people offshore more, they want more definition, they want more quality elements defined up front and don't want the project to run out of control based on trusting IT.
Given a choice between a nice spec, and a bloke from IT who says "trust me I'll talk to you lots"...
Which is a business person going to chose ?
Engineers specify, DIYers just knock it together.
Here's our development process: (Score:5, Interesting)
2. Stop work on it to fix a bug, release the bug fix immediately.
3. Work on the idea again, but slightly changed.
4. When about 80% done, switch to new idea or "gotta have" feature.
5. Code like hell on original idea because it was released in its incomplete form and has now killed puppies.
6. Rinse, repeat.
All the while there's very little documentation, most of it being whatever I do.
Great, isn't it? That's how it was. I've taken control and actually have implemented a release schedule and proper bugfix releasing. I've also just gotten a QA guy, things are looking up. The processes before I got here made me wake up at night. Lo and behold, with the new processes, the phones don't ring as much.
Ahh, the joys of a very small company.
"New study from MIT" ? (Score:4, Funny)
Which goes to show -- even if we had specifications for these things, we're just going to gloss over the details, and do whatever the hell we want, anyway. People don't read what's sitting in front of them, unless it's on some blog, it seems. If it were really important, they'd have made a TV show out of it.
[and those of you with moderator points get to vote if you think sarcasm is funny -- you can select 'troll' to vote no, 'funny' to vote yes. 'overrated' if you'd like to abstain, and 'insightful' if you read the first line, and are just trying to burn your moderator points]
Yet another outcome-data-free paper... (Score:2, Insightful)
Typical logic: PREMISE: It is good for software to be of high quality, in time, and under budget. ERGO, do thing my way. COROLLARY: It is important to use an indentation setting of three sp
Bullet proof specs. (Score:5, Insightful)
I spent many years as a low level civil servant; so none of what I am telling about is my fault
To make a pile of money on a government contract, do the following:
1 - Bid on the contract EXACTLY as the tender is written (no matter how stupid the tender is). This is what the government will stick you with. If somebody else bids on the tender the way it should be and comes in cheaper that's ok. The government will stick them with the stupid specs in the tender and they will lose money! They go out of business and you now have one less competitor. **Include a generous hourly rate for 'extras'. Make sure that 'extras' are cost plus.
2 - Build the project EXACTLY as tendered and bid.
3 - (This is the important step) Discover that the project will not work as designed and tendered.
4 - Fix the project at public expense. This is the part where you apply the 'extras'.
5 - Profit! Note the lack of question marks at any stage of this process.
Specs are fine but they are no replacement for wisdom. If you need to cya then use specs. Otherwise, take them for what they're worth.
Re:Bullet proof specs. (Score:3, Insightful)
In the business world, you must always CYA. Doesn't matter if your customer is the anonymous desktop user, another company, another department in your company, or even your mother. You will get screwed if you leave any opportunity for the customer to do so.
Philosophies in the extreme (Score:5, Interesting)
There was a little back and forth discussion, but basically the requirements were hammered out in a few days. The development team of about 5 coders took these requirements and over the next 6-8 months we used a "release early, release often" strategy. We called it "Build a little - Test a little", basically we had a GUI designer cranking out do-nothing screens, while the rest filled in the guts one function at a time.
We ended up essentially almost on time and almost on budget. The project was WILDLY successful and still in use 7 years later.
We have been funded now to produce Version 3 of this product. Most of the team has moved on, including the chief architect who drove the implementation philosophy and he's been replaced by someone who has embraced specifications like a baby marmoset clings to it's mother [yahoo.com].
Since December, we have done NOTHING but attend meetings and write excruciatingly detailed requirements documents, and crank out UML diagrams. Not a line of code has been written (besides a bit of prototyping).
I'm doing it because I have to. I see a lot of extra labor cost (I estimate at least $250k so far) and I don't think this process has helped us. I think we could have created an entire prototype since December and thrown it all away and been ahead of the game (assuming we'd learned valuable info from our mistakes)
Testing is bad for development (Score:3, Interesting)
As a System Test Engineer, I especially liked this part of the summary
One thing most people don't realize is that 99% of the time, only a small percentage of a program actually gets executed. Much of a program is error checking or handling of unusual events and doesn't get executed if the customer uses the program the way the programmer intended. The only reason that most commercial software is usable at all is because of this. So of course, testing and integration has a negative impact on development time.
Most software companies know this and develop software to be "good enough" since being anywhere near bug-free before shipping the first version requires too much time and effort. Software companies are "gambling" that the bugs that are left aren't bad enough to sour their customers on the product. IMHO, American companies are generally much bigger risk takers than foreign companies. This leads to either a spectacular success or a catastrophic failure
I would like to see the U.S. government start to punish software companies that take large risks with investors capital. I believe a lot of companies die because of poor implementation and not necessarily because of a poor idea. I can't think of anything easier to screw up then software development and it has been considered just an ordinary risk of running a business. I worked for a software company that was run by a man that, from what I saw, didn't give a hoot whether the software worked or not. As long as the IPO produced a lot of money he was happy.
There are many ways to reduce the risk of producing a worthless software program, including certifying programmers, code reuse, and more testing. I know most people don't want to invite the government to enact more regulations but the software industry doesn't seem to be regulating itself. There is a huge crisis brewing on the horizon and nothing seems to be getting better. If the millions lost on government computer systems, like the IRS modernization, isn't enough, we have the potential for a Microsoft virus to wipeout millions of users data.
Developer Veal Pens (Score:3, Funny)
I looked through the report somewhat quickly, but I did not see any mention of the peculiar practice of sticking developers into small, noisy cubicles, with cheap, eye-straining fluorescent lighting, and then expecting them to foucs and produce top-quailty software.
I was walking around where I'm currently consulting, and I noticed that everybody had a set of headphones, and it just struck me as odd that software development should require specialized head gear.
biggest problem not talked about (Score:3, Interesting)
These days, your typical American developer has a narrow stable of technology that he uses. He often doesn't stop to examine whether or not the application being designed is best suited for the environment in which he plans to build it.
I'm probably going to get flamed for this, but I believe we now have "vanity languages" and platforms that are driven more by marketing than fitness for a particular purpose. In the last several years I've seen lots of programs written in stuff like Java, Cold Fusion,
I would suspect a significant share of development disasters are due to the people involved choosing the wrong tools and then making things ten times harder for themselves later on.
Going overboard with design documents (Score:3, Insightful)
He observes that only in-experienced craftpeople plan out things down to the minute detail to begin with. This causes them to get lost in the details and not able to recover from an problems during the building stage.
Experienced people all employ processes that may start out with a rough high level design, but the detailed design only gets determined as the construction process matures.
The idea is that a more fluid design allows you to both absorb errors or any problems or new insights that may occur during the actual build process.
The same thing applies for software engineering.
There is a nice balance to be found somewhere between the bondage-and-discipline approach and the XP style design-as-you-go. The type and size of the project also have to be taken into account.
Why no specs (Score:4, Interesting)
We're too busy coding.
The truth is that commercial development schedules are unrealistic. If it would take two years to do a project correctly, only eighteen months will be allocated. This leaves the developer in a quandary over which part of the process to skimp on. US developers choose to skimp on specifications, while German developers choose to skimp on implementation.
That's my experience anyway. I've seen specs from Germany that are so padded I think the author must have stock in a paper mill. And I've seen the incomplete software that arose from it. In one instance a product was shipped that was completely unusable, whose only sales the first year were to the sales department as demos, but which won a corporate award for adherence to the process.
It's simply a different way of working. To the US developer, if you can't do it right, at least make it work. To the German developer, if you can't do it right, at least go through each step of the waterfall model thoroughly and methodically until your time is up.
Just about every corporation views the process as more important than the product. But their individualistic and rebellious nature means that US developers will work on the product anyway. But German developers will just do what they're paid to do.
and now that I've read the conclusion (Score:3, Funny)
page 20 conclusion
"It is important to remember, as well, that no Indian or Japanese company has yet to make any real global mark in widely-recognized software innovation, which has long been the province of U.S. and a few European software firms."
Cute babes program? (Score:2, Funny)