Why Your Users Hate Agile 597
Esther Schindler writes "What developers see as iterative and flexible, users see as disorganized and never-ending. This article discusses how some experienced developers have changed that perception. '... She's been frustrated by her Agile experiences — and so have her clients. "There is no process. Things fly all directions, and despite SVN [version control] developers overwrite each other and then have to have meetings to discuss why things were changed. Too many people are involved, and, again, I repeat, there is no process.' The premise here is not that Agile sucks — quite to the contrary — but that developers have to understand how Agile processes can make users anxious, and learn to respond to those fears. Not all those answers are foolproof. For example: 'Detailed designs and planning done prior to a project seems to provide a "safety net" to business sponsors, says Semeniuk. "By providing a Big Design Up Front you are pacifying this request by giving them a best guess based on what you know at that time — which is at best partial or incorrect in the first place." The danger, he cautions, is when Big Design becomes Big Commitment — as sometimes business sponsors see this plan as something that needs to be tracked against. "The big concern with doing a Big Design up front is when it sets a rigid expectation that must be met, regardless of the changes and knowledge discovered along the way," says Semeniuk.' How do you respond to user anxiety from Agile processes?"
Agile is not a golden bullet (Score:5, Interesting)
Developers hate Agile too (Score:5, Interesting)
Re:doesn't work (Score:5, Interesting)
"Proper software engineering" does work, its called "Systems Engineering", is well established and successfully used for large-scale mission-critical projects in almost every industry outside of IT - which seems to be blind to anything invented outside of IT.
Systems engineering has its own professional accreditation organization: http://www.incose.org/practice/whatissystemseng.aspx
Re:doesn't work (Score:5, Interesting)
"Proper software engineering" doesn't work.
You're right, but you're going to the other extreme. The problem with all methodologies, or processes, or whatever today's buzzword is, is that too many people want to practice them in their purest form. Excessive zeal in using any one approach is the enemy of getting things done.
On a sufficiently large project, some kind of upfront design is necessary. Spending too much time on it or going into too much detail is a waste though. Once you start to implement things, you'll see what was overlooked or why some things won't work as planned. If you insist on spinning back every little change to a monstrously detailed Master Design Document, you'll move at a snail's pace. As much as I hate the buzzword "design patterns", some pattern is highly desirable. Don't get bent out of shape though when someone has a good reason for occasionally breaking that pattern or, as you say, you'll wind up with 500 SLOC's to add 2+2 in the approved manner.
Lastly, I agree that there is no substitute for good engineers who actually talk to and work with each other. Also don't require that every 2 bit decision they make amongst themselves has to be cleared, or even communicated, to the highest levels. If you don't trust those people to make intelligent decisions (including about when things do have to be passed up) then you've either got the wrong people or a micromanagement fetish. Without good people you'll never get anything decent done, but with good people you still need some kind of organization.
The problem the article refers to about an upfront design being ironclad promises is tough. Some customers will work with you, and others will get their lawyers and "systems" people to waste your time complaining about every discrepancy, without regard to how important it is. Admittedly bad vendors will try and screw their customers with "that doesn't matter" to excuse every screw-up and bit of laziness. For that reason I much prefer working on in-house projects, where "sure we could do exactly what we planned" gets balanced with the cost and other tradeoffs.
The worst example of those problems is defense projects. As someone I used to work with said: In defense everything has to meet spec, but it doesn't have to work. In the commercial world specs are flexible, but it has to work.
If you've ever worked in that atmosphere you'll understand why every defense project costs a trillion dollars. There is absolutely no willingness to make tradeoffs as the design progresses and you find out what's practical and necessary and what's not. I'm not talking about meeting difficult requirements if they serve a purpose (that's what you're paid for) but being unwilling to compromise on any spec that somebody at the beginning of the project pulled out of their posterior and obviously doesn't need to be so stringent. An elephant is a mouse built to government specifications. Ok, you can get such things changed, but it requires 10 hours from program managers for every hour of engineering. Conversely, don't even think about offering a feature or capability that will be useful and easy to implement but is not in the spec. They'll just start writing additional specs to define it and screw you by insisting you meet those.
As you might imagine, I'm very happy to be back in the commercial world.
Agile isn't what I hoped it would be. (Score:4, Interesting)
I've done agile for the past few years and always waterfall before that. Was looking forward to agile because it sounded more flexible, but that's not what I've observed in practice. Everybody is so touchy about finishing up all of the sprint tasks by the end of the two week period, that there is a strong disincentive to bring anything new in, when you're finished up. No one wants to skew the metrics.
Between the aribtrary development cycle and the constant meetings, I find it far less productive than waterfall. Especially since so much waterfall is implemented in an interative fashion anyway.
Re:doesn't work (Score:4, Interesting)
"Proper software engineering" doesn't work.
As a Software Engineer in the formal sense (Engineering is regulated profession here in Canada) I can assure you that "proper software engineering" works great - and it's often Agile. Software Engineering is just like any other type of engineering, you have to pick the right tool for the job. That said, a lot of under-qualified people go around claiming to be Software Engineers and think that generating piles of paperwork will make their crap code (and crappier designs) smell better. They are just as bad as these people [youtube.com]
Re:Are you nuts? Don't talk agile with the custome (Score:5, Interesting)
If anyone thinks that there is complete, 100%, nailed down design for a 75-story skyscraper before any digging starts, and that the design for the 69th floor is similarly nailed down before the 3rd floor is finished, they need to spend some time on a construction site. The overall shape of the building, the structural design, the very bottom and top floors, and the allowable parameters for design of the later floors, sure. But the exact design of floors 50-72? No. Plan for what happens when the selected elevator supplier goes bankrupt, the ship carrying a key delivery of structural steel sinks, the developer finally signs a tenant for 68-70 and he wants an internal staircase and private elevator for his offices? No. Look up "fast-track" in a construction dictionary.
sPh
Re:I tell them I feel the same way! (Score:5, Interesting)
If you're not doing Xtreme Agile, you're not doing Agile right.
That is correct comrade. Failure is always caused by insufficient ideological zeal and purity.
Re:doesn't work (Score:5, Interesting)
Until the thing is built or the software is shipped there are many options and care should be taken that artificial administrative constraints don't remove too many of them.
Exactly, and as someone who does both hardware and software I can tell you that that's better understood by Whoever Controls The Great Spec in hardware than in software. Hardware is understood to have physical constraints, so not every change is seen as the result of a screw-up. It's a mentality. I'll also admit that there is a tendency to get sloppy in software specs because it is easier to make changes. Hardware, with the need to order materials, have things fabbed, tape out a chip, whatever, imposes a certain discipline that's lacking when you know you can change the source code at anytime. Being both, I'm not saying this is because hardware engineers are virtuous and software engineers are sloppy, but because engineers are human (at least some of them).
I don't hate Agile (Score:5, Interesting)
I work in a government office as a developer. We get lots of projects that come from management, who may or may not actually understand the business they're managing. If they do, they can give you some idea of what they want. If they don't... they can give you a very vague, buzzword-laden description of what they want.
What would tend to happen is people would go off and try to use something like waterfall. They'd go gather requirements, build a big specification, design, etc etc. The managers in question would sign off. We'd build it. We'd then learn at the end that it actually doesn't do what they need it to do, and their business actually doesn't work the way they said it did. Why? Because managers and users suck at dealing with this type of stuff (in my experience). In house, consultants, it doesn't matter. This has been done wrong so many times that we decided to try something more agile.
And it's worked for us. We build the pieces that people actually do know we need (either because they're based on a paper process that we can look at, or because some user can articulate it). We get that into people's hands. They tell us what they hate, what they still have to do manually, and what would make their lives easier. Then we go off and build those pieces.
It's not actually saving us time, but the users have been MUCH happier with the end products. And since I like happy users and dislike spending time building things that are pointless just because two years ago someone thought it should be in the requirements, this works out pretty well for me.
It's not the right way to do every project or in every environment, but my users certainly don't hate it. (Quite the opposite, we get a lot more feedback from people asking for improvements now because they've seen it acted on more readily.)
Re:Developers hate Agile too (Score:2, Interesting)
As I understand it, in a stand-up, one is supposed to say what one did yesterday (I don't care), what one is going to do today (again, I don't care), and what road-blocks, if any, you have (and, unless your problems affect me doing my work, I still don't care). And it never takes 5 minutes. You also have to factor in the time of just getting everybody in the same place at the same time.
If you discover a road-block at, say, 2pm, what do you do? Just sit there twiddling your thumbs for the rest of the day because the next stand-up at which you can bring this to anybody's attention isn't until tomorrow? You should just either walk over to me or e-mail me now. If you do that, then maybe we can solve the problem by, say, 3pm. Why wait until the next stand-up? If everybody did this with problems, your claimed reason for needing stand-ups evaporates.
Stand-ups are both annoying and pointless. Agile is nothing more than modern-day snake-oil.
As a project lead I found a lot of value in stand up meetings. Not all devs like them, that is for sure. The thing to remember though is just because an individual developer may find little value in it for them personally, that doesn't mean it doesn't have value for the success of the project as a whole.
If you encounter a road block at 2:00 by all means, try to resolve it, but the truth is not all developers will and often they don't have to, it can wait and they move on to something else. Then there are developers who will naturally put things off anyway (sometimes too long) or will bang away at a problem forever when they could simply ask someone else. The beauty of the stand up meeting is that problems are quickly identified even if the dev doesn't specifically mention them. If they've spent 2 days on something that should have taken a few hours, you as the project lead know there's trouble.
Just the fact that a developer on a daily basis has to stand up in front of their peers identify what they did and what they are currently working on helps them stay on task and helps you as the project lead nip problems in the bud. It also keeps everyone involved informed as to who is doing what and the current state of things as a whole. All by holding one short meeting. And if you're really doing agile, the developers are located in the same area and assembling them shouldn't take anytime at all.
Re:doesn't work (Score:3, Interesting)
You've fallen into the trap of using their terminology. As soon as 'the problem' is defined in terms of 'upfront design', you've already lost half the ideological battle.
'The problem' (with methodology) is that people want to avoid the difficult work of thinking hard about the business/customer's problem and coming up with solutions that meet all their needs. But there isn't a substitute for thinking hard about the problem and almost certainly never will be.
The earlier you do that hard thinking about the customer's problems that you are trying to solve the cheaper, faster and better quality the result will be.
Cheaper? Yes, because bugfixing that is done later in the project is a lot more expensive (as numerous software engineering studies have shown)
Faster? Yes, because there's less rework. (Also, since there is usually a time = money equivalency, you can't have it done cheap unless it is also done fast.
Higher quality? Yes, because you don't just randomly stumble across quality. Good design trumps bad design every single time.
Agile causes anxiety in the clients because the first thing you do after the requirements have been gathered is to ask the client to prioritise them. Naturally the client is going to start imagining worst case scenarios, where the developers can't do everything they've been asked to do*, and so they push back, saying that everything is important, and then the developers want to know what is the most importantest (sic). The solution is to do technical prioritisation. If I go to Honda to tell them how to build my car, and then tell them that the tyres and steering wheel are the most important things to me so they should start building those first, Honda will politely tell me to let the experts do their thing. By analogy the whole thing of making the clients prioritise the features really only works when you've already got a car, and they're choosing what options to add (read as: additional features after you've done everything that was on the initial requirements)
*Not unreasonable, given that even the Agile evangelists say that 60% of software projects fail.**
**Before Agile became the dominant methodology the failure rate was widely reported as 30%.***
***Make of this what you will.
Agile can also cause anxiety in the customer for other reasons. Imagine if you wanted a wooden chair and your developers all ran into the forest and each started madly chopping down a different tree. When you tell them the particular type of lumber you want it to be made out of they tell you to bog off because they're not at that stage of the project yet. Of course, that's not actually what happens. In fact everyone is chopping madly into the same tree (but all at different heights ... )
Disclaimer: none of the Agile projects I've ever been on have failed. But then neither have any of the waterfall or RAD projects, or even those with a null or ad hoc methodology. The common denominator for success is not the methodology.
Re:I tell them I feel the same way! (Score:3, Interesting)
Of course there's a big difference. Software is essentially the most malleable and fungible thing humans have ever devised. The big problem in software engineering that doesn't arise in automotive engineering is that in software engineering, the requirements are frequently in flux. If you can fix your requirements as precisely as the requirements for a car, than yes, there is no difference. But in general this isn't true. Unless you're writing software for NASA or something similar, most such requirements often can't be so easily nailed down. You get a giant spec, build the thing, and then realise that the process the software is designed to integrate into doesn't quite work the way the spec says it should. I've been in the software business for close to 20 years and except for a brief stint in embedded systems this has always been the case in my entire professional career. We build something and halfway through the design we find it doesn't quite work the way they customer wants it, even though yesterday they said that was what they wanted it. Are you going to tell the customer: "fuck you, you said it should work that way and that's the way we're going to build it!"? That is not the way to treat any customer, most especially when you can, with some additional effort for which you should be duly compensated, accomodate their desires. Defective brake pads can cost a car company billions in recall and replacement. Similar bugs in software can be fixed almost painlessly by comparison. Software can also be far more complex than cars. A typical car probably has a few thousand distinct parts. Software like the Linux kernel has distinct parts in the billions. Try to tell me again how there is no difference between designing cars and designing software.
Re:I tell them I feel the same way! (Score:5, Interesting)
I've worked in lots of places that claimed to do Agile, but few really did. Often it meant doing daily standups (or even sit-downs in one case!) and not having any good specs. Right now I'm working at my first contract that really is Agile, and it's fantastic. It is chaotic, but not because of us; it's the reality we're facing. We're doing several projects at once, the designs had to be sent back several times because they were wrong, business isn't really sure what they want, and we're being productive in spite of that.
We tell business what we need from them in order to do our work, instead of the other way around. We don't accept issues that don't meet our standards, and as a result, more and more issues do meet our standard. When we uncover a misunderstanding, we can change direction on a moment's notice, and because business, admin and others show up at our standups and our sprint demo, we discover these misunderstandings pretty quickly.
Not all is perfect. Not everybody around us really gets Agile, and particularly our tester would be much more at home in a Waterfall setting, but for me personally, it's working very, very well.
Re:Agile is not a golden bullet (Score:3, Interesting)
Unfortunately, iterative development is expensive, probably twice as expensive as waterfall for the same result: "refactoring" is another word for "rework,"
This is just nonsense.
Software is done after X hours. X does not vary on the fact that you do it iterative or in one big bang. So the cost is the same.
However experience has shown that small iterations/increments are easier to handle.
Agile in practice is typically waterfall without a project plan That is completely wrong. Every agile (Scrum(Kanban/XP) project has a plan. Otherwise it would not fit the description of being agile.
On top of that most agile methods are _not_ waterfall like. Except if you consider an iteration to be a mini waterfall (which it sometimes is, but often is not).