A Decade of Agile Programming — Has It Delivered? 395
snydeq writes "InfoWorld offers a look back at the first decade of agile programming. Forged in February 2001 when a group of developers convened in Utah to find an alternative to documentation-driven, 'heavyweight' software development practices, The Manifesto for Agile Software Development sought to promote processes that accommodate changing requirements, collaboration with customers, and delivery of software in short iterations. Fast-forward a decade, and agile software development is becoming increasingly commonplace, with software firms adopting agile offshoots such as Scrum, Extreme Programming, and Kanban — a trend some see benefiting software development overall."
No (Score:1, Insightful)
No
Captain Obvious (Score:4, Insightful)
And in other news, despite potential pitfalls, the pope agrees that Catholicism has benefited religion overall.
Re:A point of view (Score:5, Insightful)
The original manifesto's key message was "people over processes" which I still agree with. On the other hand the word "Agile" now means just as many processes as waterfall or RUP meant in the past. Shame.
Re:Maybe they did it wrong... (Score:5, Insightful)
In the end Fred Brooks got it right decades ago: "there is no silver bullet". Software development is just hard. Anything promising massive gains in success or effectiveness is snake oil.
Why don't they ask (Score:4, Insightful)
if waterfall has delivered?!
It seems most projects work in spite of waterfall, not because of it.
I'm not saying it doesn't work. That is for small/medium projects with a very tight scope it does.
What's the name of that FBI project again?! Virtual case file?! Oh well...
Re:"Agile", no -- "agile", yes (Score:1, Insightful)
Basically it means they are not using any specific methodology but doing things as they think it's better in specific circumstances. The question is then if we shouldn't just drop all the crap about methodologies and just go ahead and do things the best way we can.
Re:Maybe they did it wrong... (Score:5, Insightful)
The something is possible to define and explain but is typically different in different cases.
In this case it seems that the problem is changing requirements. That will kill any project no matter what the methodology.
I think.. (Score:3, Insightful)
Bits and pieces of agile are good. I think if you take the spirit of agile.. that is doing things because it's a good idea, not because you have a compulsive need to fill binders with documentation.. it can work well.
I like unit tests for regression testing (that is, verifying that the software still does what it used to).. but I think test driven development is more hassle and riskier than it's worth (now instead of just changing code when the requirements do a 180, you have to change code _and_ unit tests, ($cost + $time) * 2).
I like eliminating "because we need documentation" documentation.. but I think there is great value in documenting stuff that is complicated or weird. Having a binder full of "the UserAccount class represents a user account. It is comprised of a username representing.." is useless. The same goes with diagrams.
The worst is when agile is implemented as a buzzword. "We are agile! We have a binder full of documentation describing the rigid agile process we follow!" (not saying agile or processes in general shouldn't be documented, but in my opinion the whole point of agile is to be flexible).
Re:"Agile", no -- "agile", yes (Score:5, Insightful)
The original Manifesto:
"We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more."
I think it is hard to disagree with the original statement. What is frustrating now, is that "Agile" degraded to a bunch of buzzwords and processes (SCRUM, XP, TDD, BDD, etc.) which it was going against originally. Of course it is more business in selling these (by consulting companies) than the vague "do what fits you best" statement.
Also, the problem is not with particular practices, or processes but the mindless, inflexible application of them to every project and team on earth. Teams are different, software are different, why should we all use the same processes then?
Re:No (Score:0, Insightful)
Yes!
Look! Boxes of software falling from the sky!!!! (Score:1, Insightful)
Sweet, more supposed justification for the cargo-cult bullshit that passes itself off as modern software engineering practices. "Yes, someone said that every morning we must huddle and Scrum and the Scrum must take place like this or it won't work right! If you let someone sit down it won't work right !! Now make sure you don't do anything until you've jumped up and down together and given a handy to the dude to your left or it won't work right !!!"
Re:Maybe they did it wrong... (Score:5, Insightful)
In this case it seems that the problem is changing requirements. That will kill any project no matter what the methodology.
This is exactly one of the key points that the Agile community keep trying to drill into us! The Agile Manifesto explicitly states "Responding to change over following a plan", which is translated into the principle "Welcome changing requirements, even late in development."
Re:Maybe they did it wrong... (Score:3, Insightful)
Software development requires Creativity, Discipline and many more things apart from Agile, and the process goes through lot of iterations, training, discussions and confidence building among developers to motivate them to follow process in the company. Also, the process to be adapted should be close to the type of Software company is focused on and what developers are comfortable with. Many process designers/implementers forget this fact and blindly throw something at them and expect results overnight and get surprised when it ends up disastrous.
It takes time to show them the processes are here to improve quality. Agile or any other Software Engineering process is not bad. The process of process implementation could be bad.
Also, process implementation is a difficult task if upper management believes that it is job of juniors.
So yes, I agree with you that Maybe they did it wrong way. If somebody believes Agile is going to work like a switch, they are mistaken.
My 2 cents.
Re:Maybe they did it wrong... (Score:5, Insightful)
Agile principle #2 (from http://agilemanifesto.org/principles.html [agilemanifesto.org]): "Welcome changing requirements, even late in
development." Other methodologies typically draw a line in the sand to recognize the costs of changing requirements late in development. They don't often forbid crossing that line, but they encourage people to honestly evaluate the trade-offs involved in changing requirements.
Paired programming might get you cut! (Score:1, Insightful)
I post anonymously for a reason. But the next person that decides they want to sit down and "pair program" with me might just get stabbed. Yay almost 1/2 the productivity!!! 1 person sitting the other thinking:
- You smell.
- God I can't type while someone watches.
- Fucking sip that coffee one more time bitch!!
- I have to fart.
- I don't want to hear about your kids.
and the biggest..
- Touch my screen one more time!!! daaare you. :-(
I swear, the ones that are the most interested in this programming paradigm feature are the ones that can't code and want to feel part of the process while still acting in their typical ineffective fashion all the while flying under the radar.
Agile can work ... (Score:3, Insightful)
In my experience, Agile works for two things:
1) Maintennance and support development
2) Extremely strong engineering teams
Formal processes with more up-front planning and review like waterfall allow you to turn out high quality product with average teams. You can't use the typical Agile processes with average teams and do substantial new development, but I've seen it work well enough for incremental development or support engineering/bug fixing.
A good manager will know his or her team and use the methods appropriate for success with that team, and not chase buzzwords.
In my experience, though, these days there are a LOT of bad managers.
Re:Maybe they did it wrong... (Score:5, Insightful)
I don't know enough about Agile to make a judgment myself, but you've practically said it yourself: your first experience wasn't Agile, it was just something that someone called 'Agile'.
Re:Maybe they did it wrong... (Score:5, Insightful)
To take a car analogy - using Agile is like using a GPS: it is a methodology that tells you were you are, and can adapt if you take a wrong turn or if you change the destination. Like the GPS, if you cannot make up your mind where you are going (or do follow the indication of the GPS gives - been there, ...) you will not get anywhere.
Re:A point of view (Score:4, Insightful)
"People over process" to me also means acknowledging the value of each individual's skills and abilities. That starts with resourcing: instead of posting jobs for "2 junior programmers, 1 senior programmer, 1 encryption specialist (part time) and 1 PM", one would think "Together, Alice and Bob are up to the task of writing this module, Alice knows enough about encryption to cover that part of the job, and Bob is a good team lead". No two people are alike, and if you value people, that means that you account for their differences as well. Yes, it also means that if Alice calls in sick, you have a problem on your hand, but it's by no means an insurmountable problem.
And it goes beyond that. "People over process" to me also means that you let people do what they are good at beyond the call of their assignment. (as long as it benefits the company). As an individual, it means that you sometimes take responsibility for stuff that is not strictly your job. If someone contacts me with a problem I can solve, I do not answer with "Contact the helpdesk"... if it's likely the problem will get kicked down 3 layers, taking 2 weeks to come up with a non-answer. Instead I will take an hour off my regular work to provide a helpful answer (of course with the suggestion that they contact the helpdesk, and copying in support as well). This helps everyone, including the company.
Yes, accounting for individual differences makes budgeting, resourcing, managing continuity, and people management a lot harder. That's why managers hate "people over process" and why Agile IT gets loaded down with more process: resources are so much easier to manage than people.
Indeed, THERE IS NO SILVER BULLET (Score:5, Insightful)
IT, or at least some people in IT suffer from what I call the "Oprah" syndrome. No, not the weight issue. The idea that a problem is just one problem and that it can be solved with a single solution. Or the silver bullet that will solve all your problems at once.
Software development is bloody hard, partly because it doesn't deal with a real product and testing a software product is extremely abstract.
If I design a better brake pad, I can test this fairly simply AND thorougly. We know exactly what it is supposed to do, how it does it and how we can compare performance. There is no magic, if the brakepad reaches 198 degrees on a sunday, the trunk will pop open. Also nobody will ask a brake pad engineer to add a christmas special to it, on the 25th of december, at 8 in the evening.
Web development especially suffers from the idea that if only we implement magic method number #45324521, all the hassle of web development will go away. Writing specs is long boring and not very sexy. So lets do agile and not write them anymore. Problem solved... eh no because spec writing is long boring and not very sexy because getting the requirements out of a user is like sucking cock. The user might enjoy it but you are just left with a bad taste in your mouth and a desire to throw up over the user. Or maybe that is just my dates.
"But if you do agile right", but if you do documentation driven development right! Any method, done right will most likely work. That is the definition of doing it right. Why is it so hard to do agile right? Why are there so many variants already?
I see a lot of "magic" solutions in web development.
Database abstraction, so we can magically switch database!!!
Question: When have you EVER switched database on a web application and HOW easy was it? Is there ANYONE out there who only use pure SQL that is 100% understood by all databases the same? Then go stand in the corner, because your code lacks any optimization. Real developers optimize their code for the specific environment they are using. This includes making use of database specific features. Don't use Mysql auto-increment and PDO and think you are database independent.
Frameworks take all the hardship out of writing code!!!
Question: How smart do you think it is to hire a coder who hates to code? It is like hating a singer who hates to sing, a spokeperson who doesn't like talking.
This is where a lot of the silver bullets come from, from people that don't actually want to be a developer but just cash the paycheck. You don't hire a cook who only knows how to nuke frozen meals do you? If you go to the supermarket and buy a microwave meal thinking wow, this makes cooking easy, then here is a hint: IT AIN'T COOKING. It is heating up food.
I see time after time some framework or tool promosing a complete working website without writing a line of code... EEK! That is my JOB! Luckily they are all wrong because what at most they do is create a straight CMS for your database, which is rarely what the customer wants. After all, there are already plenty of tools to graphically maintain your database content.
No, Agile hasn't delivered... for those looking for a silver bullet to the problems of development. For others, it is a valid method with its own problems that puts it along side more traditional models, not ahead of them.
We're ALWAYS doing it wrong (Score:3, Insightful)
Reminds me of an old poster [globalnerdy.com] from my CS department. It features legendary crotchety old fart John McCarthy [wikipedia.org] angrily looking at the camera, with the caption "Programming: You're Doing It Completely Wrong."
Re:Maybe they did it wrong... (Score:4, Insightful)
Uh, that's not Agile. Nearly everything you mentioned there is so anti-agile I can't even start breaking it down. Slapping an Agile tag on it doesn't make it agile. No mater what most companies think.
I have seen frequently referenced that the switch to agile takes up to 18 months to produce results. And even then there have been studies that find no significant enhancement in productivity.
As far as I can see there is no actual benefit in productivity at all moving to agile. It just gives greater clarity to the current state of development and should produce software with lower levels of faults through automated testing and continuous integration.
Re:Maybe they did it wrong... (Score:5, Insightful)
Your post reminds me of a Muslim friend who told me once (dead seriously) that Muslims didn't attack the World Trade Center. They weren't REAL Muslims, you see.
Agile - The fad (Score:3, Insightful)
In my experience plenty of teams "do Agile" without understanding what it's supposed to achieve. People just cherry-pick and adopt well known bits of some Agile methodology or other (for example the stand-up meeting) but then don't do it properly or miss the feed-in practices that are necessary to make it work.
It's all about "Being Agile" (i.e. fashionable) instead of "Having a way of developing software that can consistently cope with fast-changing requirements minimizing wastage, chaos and cross-team overhead".
Just as an example, in the last 7 or 8 years since Agile became fashionable, having been in and out of multiple teams/companies that use Agile to a smaller or greater degree (I'm a freelancer), I have yet to see a proper Use Case which describes in a consistent and well defined way a feature that the system being designed must provide, including handling of error conditions. In fact, 9 out of 10 times, people forget to account for errors (even user errors). Use Cases are the basis for each development cycle, the point of communications with those defining the requirements and the basis for prioritization of work and yet most teams can't even get those right.
Then there's the flaws in analysing the requirements to do make sure the system has all the data that it needs (if for example one Use Case says "The user will select one value out of a list of options for field X" you need to have those options coming from somewhere, potentially ending up with a whole bunch of other Use Cases dealing with maintaining those options).
Agile doesn't solve the essential problem in IT which is a lack of understanding of the software development process, lack of preparation and lack of method - it just provides cover for those developers which do everything in an ad-hoc, reactive way and will then excuse themselves as being "Agile".
There are simply not enough people around that try to understand the process by which people create a software system or application from a "not so well defined set of needs from the users" but lots of people focusing on the quasi-aestetic details of some language or other - too many people asking "How?" not enough asking "Why?"
Re:Maybe they did it wrong... (Score:3, Insightful)
Firstly, I think Agile has delivered. The real problem is that most companies (people) have used the term, "agile" to mean I don't really have to have discipline because I'm being so agile. In my experience, management loves the "code done in two weeks" part, but they hate that the rest of it has to be done in two week increments part. In my opinion, Agile methodologies have had the least affect on developers. Most of the rest of IT is used to a leisurely pace where they have meetings for six months to decide what programmers should do in two weeks. Using an agile methodology, these same folks are under the gun to deliver and, in my experience, they don't like it.
Re:Maybe they did it wrong... (Score:2, Insightful)
You also have to be not an idiot, and you must not have idiot managers and idiot customers.
Therefore, it is always doomed to failure, since at least one of those will always be true.
The most successful model will be one that assumes all of those are going to be true.
Re:I think.. (Score:3, Insightful)
Bits and pieces of agile are good. I think if you take the spirit of agile.. that is doing things because it's a good idea, not because you have a compulsive need to fill binders with documentation.. it can work well.
Documentation is like code -- it doesn't work unless you do it right. An example is kubuntu's documentation. Pages and pages about how to customise the K menu, but not a single word about how to start the editing program. I had to have someone on slashdot to tell me you left click the K menu.
We're not mind readers. Just because you think something's obvious doesn't mean it is. Bad documentation is as bad as bad programming. The trick isn't less documentation, it's BETTER documentation.
It seems Microsoft has been studying this, because Windows help system is even worse than kbuntu's documentation. At least most stuff is documented, and accurately, even if a few things someobody thought was obvious were left out.
I like eliminating "because we need documentation" documentation.
Indeed; sometimes it seems you need less than three digit IQ to be a manager. Document code for the people who will have to maintain it, not for documentation's sake. Document the software for the end users who need to know how to find a feature, not for the sake of documentation (like Microsoft seems to do).
The worst is when agile is implemented as a buzzword.
I saw nothing but buzzwords and marketspeeak in the linked FAs.
Re:A point of view (Score:3, Insightful)
If you read the original manifesto, it is very carefully worded. It does not say "abandon processes", it says that the priority should be on the developers as creative individuals, instead of mechanized drones.
Also, it is not against "processes" (with small "p"). A build system is itself a process, a source control software also enforces a process. The manifesto is against Process with a capital P. It is hard to explain, but easy to give an example what I mean about this:
http://www.ogcio.gov.hk/eng/prodev/es3.htm [ogcio.gov.hk]
The above link is about the SSADM Process. Read it and you will immediately understand what I mean.
Also, lot of people misunderstand "Working code over documentation" and think that documentation is not important. In fact, it should be read primarily as "project documentation", the things that most old fashioned processes mandate. Again, look how many and what kind of docs SSADM needs.
All things turn to process in wide use (Score:3, Insightful)
The original manifesto's key message was "people over processes" which I still agree with.
This is I think, a totally key point.
Agile is supposed to describe a way in which people can work together to improve productivity. If you think about it, what it's really doing is attempting to document how a successful team (or one kind of successful team) interacts, like Goodall studying mountain gorillas.
So then other companies want to have successful teams too, and study what was done. But instead of saying, here's an approach, everyone read this and lets try something similar, instead of that companies all over have managers reading agile books and then using that as a template into which they cram all the developers they have, ignoring the differences in the way a given group interacts with each other.
And that's because it's easy to do one thing, but very hard indeed to adapt something for the individuals doing it. The payoff is enormous if you can get individual tailoring right, but it's so hard to get right I don't know that you can even try except to try and recognize signs of success in groups and encourage them (which in turn leads to cliques if you are not careful).
It's something that happens to any methodology, in short order it will ossify into rigid methodology because the communication of how something works spreads to most people in the form of text that looks to managers like a template for how to arrange "resources".
Agile Misinterpretations (Score:3, Insightful)
Re:Maybe they did it wrong... (Score:1, Insightful)
Your post sounds sarcastic although your friend makes a lot of sense. To most Muslims, highly radical types are horrible people. Of course, to radical Muslims, the normal people aren't real Muslims in turn.
Re:No (Score:3, Insightful)
This is pretty much the ultimate "get off my lawn" post.
You took the word from my mouth. (Score:2, Insightful)
Total failure from my experience (Score:2, Insightful)
Re:Maybe they did it wrong... (Score:1, Insightful)
You were fired because ... they did it wrong.
1) You were forced to take the ticket: wrong
2) You were denied help when you asked: wrong
3) You were held to an _estimate_ which happened to be off by a factor of 4: wrong
It is never strict Waterfall vs. strict Agile (Score:3, Insightful)
Yes, anyone adhereing to a strict waterfall model today is probably being silly.
If you are doing internal-only development you can get away with constant change. When there are real outside users there is real, external documentation and marketing materials. There is a real date for a product launch, and things have to be stable for it.
Marketing materials need to be prepared and printed. If they take a bunch of screen shots and sent stuff to the printer Marketing will not be happy with the announcement that those are the "old" screens and the "new" ones are much better now.
An really funny scenario is some marketing type is going to give a demo of the product to some big customers (prospects, really) and no longer understands how the "new" product works after the latest round of changes.
Yes, I have seen that happen. The result is there are some new developers working on the product and the launch is delayed. Sometimes for a year. Sometimes the original developers have a hard time finding a new job.
Change is good, stability is good. The intersection of these two is really great. Anything that is too far away from the intersection tends to be bad, especially at the outer edges of either.
Re:You took the word from my mouth. (Score:3, Insightful)
Of course, there is always the exception (Score:4, Insightful)
While you have a valid example of why database abstraction CAN be useful, you have not actually proven that it must be done every time just in case you might switch sometime in the future.
Seen to many developments throw speed out of the door for a migration that will never happen.
Re:A point of view (Score:3, Insightful)
Also, it is not against "processes" (with small "p"). A build system is itself a process, a source control software also enforces a process. The manifesto is against Process with a capital P. It is hard to explain
I don't think its really hard to explain, though I think the best explanations I've seen are in books that focus on "lean" methods rather than "agile" ones (which, really, aren't all that different; the principles are largely similar, though "lean" seems to focus on an enterprise wide approach that lean software development applies to software development, while "agile" seems to focus on software development with applications to the broader enterprise particularly the interface between software development and business):
It is important to have clear processes that people follow, because otherwise you have wasteful churn as extra effort is spent doing everything ad hoc. But you have to respect the people doing the work to the point where they are expected, on encountering a problem with the existing process, to call a halt, propose and test an alternative, and then that alternative, if it is an improvement, is adopted as the new process.
That is, just as software being developed needs to be developed in small increments and able to respond to emergent discovery of requirements, so the processes used in software development (and, really, throughout the enterprise) need to be able to be adapted rapidly in response to emergent requirements. And, to do that, the people that do the work have to own the processes by which the work is done.
This is opposed to devotion to enshrined Processes that are treated as received wisdom and not subject to question or revision by the people actually involved in doing the work.
Re:Indeed, THERE IS NO SILVER BULLET (Score:3, Insightful)
Frameworks take all the hardship out of writing code!!! Question: How smart do you think it is to hire a coder who hates to code? It is like hating a singer who hates to sing, a spokeperson who doesn't like talking.
There are no silver bullets, but there are lots of wheels out there - they don't need to be reinvented for every project. Question: How smart do you think it is to hire a coder who hates to use frameworks, and loves to roll his own for every new project?
Re:Maybe they did it wrong... (Score:2, Insightful)
You are completely right.
I remember when extreme programming (part of agile programming) was still new and being promoted. I have not kept abreast of developments but at that time there was only one project for which extreme programming had been systematically used: the C3 project at Chrysler. It turns out the project was a resounding failure. However, every time the extreme programming snake oil peddlers had something to say about it, they'd defend extreme programming by stating that the reason the project failed was not due to to the process but due to internal politics.
Yeah, and I have a process to get cold fusion from ingredients found in my kitchen. What? It did not work? That's because a ghost interfered.
I went and read the article and found that even to this day the defenders of agile methods pepper their statements with disclaimers:
Well, duh... is this not the case for any method? This is just setting up the stage for again pointing fingers at anything except the method when projects using agile methods fail. The whole agile thinking starts from the axiom "agile methodologies are intrinsically successful". So any problem is seen to come from outside.
Re:Maybe they did it wrong... (Score:1, Insightful)
Since agile is non-prescriptive there is no "doing it wrong".
Agile just wants to define every Agile failure as "not-Agile", and every success based on people somehow working it out for themselves as "Agile".
Re:Frameworks = bad practice? (Score:2, Insightful)
Frameworks aren't "automatically generated code" (typically), they're libraries of code written and tested by other teams in the past, along with a skeleton code structure to fill-in.
I agree with your sentiment, but it doesn't apply to frameworks.
Re:Maybe they did it wrong... (Score:1, Insightful)
And that raises the pertinent question of what the hell sort of process relies on having some superman genius as a critical part of it?
Processes are supposed to make successes repeatable and help eliminate failures. If step 1 is "hire superman", then it's hardly surprising the failures.
Re:Maybe they did it wrong... (Score:4, Insightful)
Well, there's a consistency there, then.
One of the basic tenets of Agile is that you don't use inexperienced people.
Much of the agility comes from the fact that your team consists of pros who don't have to hack so much as lay down code and keep going.
If you give someone a task for which they aren't trained, you're shoving logs under the bogies of your train. The whole thing will stop, but not all at once, and the spot where it starts stopping will be the least happy about it.
Re:No (Score:2, Insightful)
Re:Maybe they did it wrong... (Score:3, Insightful)
Straining the analogy to the limits you can mitigate a changing destination if you plan your route well. Most changes will not be of the "make a legal U-turn" variety, but rather "let's go to that other shop/restaurant in the same city." If you planned your route well and stick to major roads like motorways then these changes can be incorporated without massively disrupting the journey.
In programming terms that means building in flexibility where possible, normalising your databases and so on. If you are writing an inventory management program and the company brings in a new procedure as long as your database is in good shape and the code not monolithic it usually isn't too hard to accommodate. Actually a lot of business software is just a human interface to an SQL database with a bit of automation thrown in.