A Decade of Agile Programming — Has It Delivered? 395
snydeq writes "InfoWorld offers a look back at the first decade of agile programming. Forged in February 2001 when a group of developers convened in Utah to find an alternative to documentation-driven, 'heavyweight' software development practices, The Manifesto for Agile Software Development sought to promote processes that accommodate changing requirements, collaboration with customers, and delivery of software in short iterations. Fast-forward a decade, and agile software development is becoming increasingly commonplace, with software firms adopting agile offshoots such as Scrum, Extreme Programming, and Kanban — a trend some see benefiting software development overall."
Maybe they did it wrong... (Score:5, Interesting)
"Agile", no -- "agile", yes (Score:5, Interesting)
I've worked in 100% waterfall, and in good agile environments, and I've found the key is to keep things small-a "agile", not to concentrate on capital-a Agile. Some places embrace Agile as a process, and fill binders with process documentation around their Agile process - at which point it's no better than any other.
I think the key to success is summed up in this line from T3FA:
First decade? Not likely (Score:5, Interesting)
The truth is that electronic delivery created agile development, at least for software companies. Internal departments have had the connectiviety for 2+ decades to deploy new releases often, but it wasn't till the late 1990s or early 2000s that a "boxed" software company could provide regular working releases to their customers (who wants to mail a CD or a stack of floppies every 3 weeks, and what customer would actually apply them?) Web-based apps and self-updating software just brought dedicated software development in line with corporate development practices, and then it was given a name.
Where it works and where it does not. (Score:3, Interesting)
2. It at least notionally asks the management to consider resources while committing to features.
3. It allows some kinds of software project to managed better. Things like merging products when companies merge, or when porting software to a new platform etc.
4. It works when the project has a large number of people with identical or largely similar skill sets. If the project has large number of specialists (like "Frank here is the only one who can even touch the turbulence modeling code") development will be slow, agile or not. At least the management should know not to commit to too many turbulence modeling features.
5. Rally proponents broad brush all skeptics as "people not willing to change".
6 Too many Rally concepts come from manufacturing and some of the examples and analogies don't even translate correctly. Take the famous, "burnt toast" example. A company decides to burn all the slices, and then scrape off the charred crumbs to get the color desired by each customer. Supposedly Rally will toast them right in the first try saving all the efforts that go into scraping off charred crumbs. Well, in software it does not cost me any money to char a toast and scrape the crumbs. Often times it is perfectly ok to render all the pixels of each body, even if another body is going to come back and override a lot or most the pixel later. You waste time trying to predict the final pixel value to render it just once.
7 Some times it is funny to see the Rally proponents not using Rally methods to develop their presentations, not using Rally for their own internal websites! Too many of them recite from a text book or a holy book instead of using actual code/project examples.
Re:Maybe they did it wrong... (Score:5, Interesting)
I fail to understand why that failing was related to the methodology?
"Due to changing demands"
It doesn't matter which methodology you use, if your goal is constantly changing you won't get anything finished, almost "by definition". If it wasn't that bad, maybe they weren't familiar enough with the methodology?
In fact, I'd say that agile software development methodologies work best for such projects, because they are aimed at constantly changing demands.
That's why almost all startups use agile software development, because *every* startup goes through a process like (very grossly, mind you):
1. search a business model
2. execute business model
3. realize 1) wasn't realistic, change it
4. goto 1
The transition process (usually referred to as pivoting) needs to occur *rapidly* and *continuously*, hence the agile software development.
Of course, if you keep changing the initial goal, you will never get finished, doesn't matter which methodology you use.
Scrum is *not* a replacement for good management (Score:5, Interesting)
I appreciate good management. I can live with no management, but I can't handle bad management.
SCRUM has sort of become a device for a manager to avoid managing. The human in the picture is replaced with a sprint chart and backlog tracker. It lets bad managers get by, while good managers are really thrown out of the picture.
I hated scrum in my old job. But the new job just throws out a list of features to implement, ranks it and throws it at one of us. There are no punishments for missing a day, no tracker to guilt-trip you and most importantly, the stand-up meetings are just before lunch. And mostly, serves to keep our communication channels open across the team.
I hated the time-keeping TPS report style scrum, but I'm cool with the new approach. I love the idea of sprints and taking a week out of a month to hammer something out. But this system only works with motivated teams with a fair scrum-master.
But I repeat, it is not a replacement for good management. It is as good as a way of letting me manage my tasks,but please (for the love of God, please) do not use it to manage me.
Re:Maybe they did it wrong... (Score:5, Interesting)
Re:Maybe they did it wrong... (Score:2, Interesting)
+1 yes
Verily I say unto you, Agile is the development methodology of mystics. You don't fail or succeed on this basis alone. You also have to be not an idiot, and you must not have idiot managers and idiot customers. Agile doesn't prevent anybody from being these things. Nor does waterfall turn you into these things.
"You're doing it wrong" is a phrase you hear uttered by proponents of faith healing, psychics, homeopaths, practitioners of feng shui, gingers, etc when their particular brand of woowoo fails to deliver on its promises.
Now, that's not to say someone *isn't*, in fact, doing it wrong. But I am saying that you can't necessarily take next step and conclude that there is a *right* way.
Re:"Agile", no -- "agile", yes (Score:3, Interesting)
A related problem is that you don't look ahead and don't see the showstoppers until it's too late to avoid them. When you do, you often end up having to rewrite large parts of the code or implement major kludges.
At least with a traditional methodology, everything is reviewed for feasibility and risks before you invest too much time driving down the wrong path. And a change in scope means a forced review of how this affects everything else too.
But there are benefits to agile too, not the least of which is that it shows progress, which gets management off the developers' backs, which in turn increases productivity.
My preferred method is parallel programming, where two or more units are set to design and do the same task, and halfway through, you present both solutions, axe one of them, and merge it with the remaining unit, with focus on doing QA They don't know the other unit's code, but will know many of the inherent problems, and see ones that the "winning" team are blind to.
Your initial velocity will be slower, but you'll end up with both a solid product, and one that management has actually picked themselves, and thus feel obligated to stand behind.
And no, the hours when you work double aren't wasted -- any more than the hours spent on scrum, six sigma or other "necessary overhead" are wasted. The competitiveness ensures a better product, and you even have fallback code and coders, whereas for administrative overhead the hours spent has zilch value later in the game.
Re:No (Score:4, Interesting)
From languages I've seen, programming has regressed in the last decade. When Grace Hopper wrote the first compiler, her dream was an english-like language that computers and people could both understand easily with a minimum of training. She co-wrote COBOL as the second compiler.
Older languages, like the mainframe DBMS NOMAD and the PC DBMS dBase were IMO brilliant. They needed little documentation, and no curly braces or other such nonsense you find in modern languages like C or Java.
The "language" I hate most is MS Access. Convoluted and unintuitive, you don't really write anything, and what's produced is "visual basic" and the equally unreadable SQL.
I hate it. I'd rather program in assembly, so I can know what registers are being accessed, whether I pushed or popped a stack (and which stack), etc.
I hate programming in a language that doesn't let me feel the metal. Hell, I'd tale old fashioned 1980 BASIC over the newer languages; it doesn't take long to learn not to write spagetti.
Re:Maybe they did it wrong... (Score:2, Interesting)
Alright, I'll bite. What do you mean that you're doing it wrong. Agile is supposed to be many things but as long as you follow a few key ideas, it'll work better than waterfall.
1) Continuous delivery. Deliver something every two weeks.
2) Quickly fail. If a problem is found in a design or a project, find it early and save tons of money.
3) Small teams. No 80-person teams here.
4) Small tasks that you should accomplish quickly helping with visibility
5) Highly visible tasks and burndowns to help with "buy in" from upper mgmt
6) Open communication meaning that the team has the responsibility of fixing things, identifying poor performers, and helping people to succeed.
No manager... just scrum masters.
Just these few key points make a world of difference and can be key to success. I haven't seen it fail but maybe you were in a company of design-by-personality...
Agile is almost meaningless (Score:1, Interesting)
Agile has had the misfortune of being a buzzword that anybody can write about and sell books for. Lots of places then use it as the buzzword process that they are following when in reality the only thing that has changed is what they are calling it.
Other places genuinely try to implement it and find that by getting some things slightly wrong they cause themselves more problems than by sticking with what is familiar to them. This naturally sours everybody involved to agile.
TDD and XP are hard to get right.
However, TDD - Test Driven Development - is something that if your not doing then you probably should looking at implementing it. It raises the very valid question that if you don't know how your going to test something then how are you going to know when you've got it working. It encourages modular code, well factored code because otherwise writing the tests is going to be a pain the ass. Hint - if your finding writing your tests difficult then it probably has something to do with the design of your code, and that is why writing the test first works to improve design.
XP is even harder because it's a superset of TDD. Not only do you need to be getting TDD right but now the process also covers project management and client interaction.
In my mind, the most important part of agile, the part that makes it agile is not any of the above. XP and TDD may be good things but the essential part of Agile is the weekly retrospective where the team sits down and spends an hour on discussing what's going well, going badly or is just worrying and then comes up with ideas for how to improve what they are doing at that point in time.
The idea that the process that your team is working by is not fixed but is evolving is the most important idea of agile.
Re:"Agile", no -- "agile", yes (Score:3, Interesting)
If you interpret the manifesto in the original context in its own age, you will understand that documentation means there not "developer documentation" or "user documentation" but the various "project documentation" artifacts that were the holy grail of that age.
The same is true for processes. It is not about the small scale ones (source control, review processes, whatever), but the overengineered project processes prevalent in that era.
Re:Maybe they did it wrong... (Score:3, Interesting)
The reason it looks like magic is because it requires at least one person who Truly Gets It. It's easy to get business people to buy flash, because they cannot measure substance, nor can they differentiate between skill and luck with past experience.
For Agile methods to work, you need at least one person who sees the whole big picture, and given enough time and/or other good people to work with, could implement everything from scratch. Once you have that person and a commitment to make the project work, you'll succeed. If you don't have that person, your odds are just as good as any other poorly managed software project.
I worked on a project where we had 6 months to do more than others had done in 18-24 months in the past. The requirements were subject to change on the fly, but the big picture was understood well enough by everyone to make sure that we kept going down the right path. We bought three off the shelf software components that were supposed to be the building blocks of the project. In the end, we used one, abandoned one, and had to do a lot of work to make the third one usable for our purposes. From a functionality perspective, buying the software covered 10-20% of the work that needed to be done to create a viable solution.
There were several times where the management direction would have badly crippled us further into development. However, our management trusted our recommendations and later in the process learned that what we were advocating had been right all along, but wasn't obvious until core requirements changed later.
Without someone who has a deep understanding of software development and all of the things that go into it, the whole process looks like magic.
Re:Indeed, THERE IS NO SILVER BULLET (Score:5, Interesting)
"Question: When have you EVER switched database on a web application"
Earlier this year, because expenditure needed to be kept to a minimum during the recession starting in 2008 so the initial build used MySQL with the goal of moving to MS SQL Server when profit forecasts were a lot safer and more stable, which they were (and still are!) earlier this year.
"HOW easy was it?"
Effortless, precisely because I'd used database abstraction. It was done and tested in around 20 minutes thanks to the well written unit tests accompanying our abstraction layer demonstrating that the switch worked as required.
"Is there ANYONE out there who only use pure SQL that is 100% understood by all databases the same?"
Yes, I did, for this particular project, for precisely this reason.
"Then go stand in the corner, because your code lacks any optimization."
That's a rather broad statement, there's plenty of optimisation you can do even with just standard SQL, whilst you may lose some additional DBMS specific optimisation features, you may still be able to reach suitable levels of optimisation with ease-
"Real developers optimize their code for the specific environment they are using."
What was that about magic solutions? Optimisation is something you prioritise like everything else. If your application runs at no more than 10% CPU usage and the load isn't going to increase because the userbase is fairly static and this provides plenty of room for increases your application would be expected to see then it's far better to ensure your code is maintainble, than it is to sacrifice maintainability for unnecessary and time consuming optimisation and save the company money by focussing on the priority that best suits the task at hand. "Real developers" should recognise that they simply don't need to optimise at all if in the specific case they are considering it is an unnecesary task.
To use a typical Slashdot car analogy, even the car industry gets this, this is why not every car is in a fight to be the fastest in the world, car manufacturers understand that when a lot of people are limited to a set speed limit anyway, there's no point optimising the car beyond that point and ensuring other things, like having enough room for children to sit in the back, are a priority. The speed of the car just has to be good enough to fulfil the end user's needs not be able to reach speeds which the vehicle will never ever be pushed to in practice at an unnecessary and possibly unaffordable level of cost to the end user.
I actually agree with pretty much everything else you said, but your viewpoint seems a bit contradictory- on one hand you seem to be arguing the basic principle that it's all about using the right tool for the job, but on the other you then seem to be arguing against ways of doing things that are perfectly valid in some scenarios. If you hadn't taken a pop at database abstraction and insisted that all "real programmers" optimise their code then I find little fault with everything else you said. As it stands it sounds like you're saying "You should use the best tool for the job, except in a few cases where I don't like that tool". Sometimes database abstraction has value, sometimes optimisation doesn't.
Re:Maybe they did it wrong... (Score:2, Interesting)
Yes, some things should be able to change.
No, some other things should NOT be allowed to change without a complete project renegotiation.
You guys without an external paying client can open your eyes again now.
Re:Maybe they did it wrong... (Score:3, Interesting)
It all depends on the level of change as well. Agile, by design, is supposed to have constant product demos throughout the development process. The customers should always know what is being built, and the changes should come in little bits after each demo instead of a massive turnaround near the end.
I've done nothing but proper agile for years now, and even when it's done perfectly a massive about face on a project will kill pretty much anything....agile or not.
Agile tends to avoid game-changing changes at the last minute because the customer can watch the product evolve and can provide proper feedback along the way. It also puts a lot of emphasis on business value vs. wild desire.
Re:Maybe they did it wrong... (Score:3, Interesting)
it is a methodology that tells you were you are, and can adapt if you take a wrong turn or if you change the destination
If you constantly take wrongs turns, maybe you need a better driver. If you can't decide where you want to go, nothing can give you directions on how to get there. None of this relates to the effectiveness of the GPS device.
Translating this back out of the car analogy is left as an exercise to the reader.
Re:Maybe they did it wrong... (Score:3, Interesting)
Yes, but does a Brazilian saying "I'm a Scotsman" make him a Scotsman?