Microsoft Lauds Scrum 299
under_score writes "According to eWeek.com Microsoft is adopting the agile methodology called Scrum to get software built faster. Is it working? They seem to be claiming that Scrum and Extreme Programming have helped them get recent releases such as SQLServer out the door faster with better quality. Many other large organizations are also adopting agile methods including Yahoo, and Google. Are agile methods the next big thing in software development?"
But will our management buy it? (Score:3, Interesting)
Re:Doubtful about the speed (Score:4, Interesting)
However hopefully if they continue to use this flawed business plan, they'll continue to slowly lose customers. Well, I can dream, can't I?
So how do you write tests for .. (Score:3, Interesting)
- applications that are a constantly moving target "this would be cool to have"
- applications where the moving-targetness lies in the presentation, while at the same time some customers bitch about any change in presentation
- applications with changing data sets - you can run your tests fine on the standardized data set, but then when it hits the real-world data, all you can say is "Sorry my application is perfect, it just doesn't work with with that data.".
But that's typical. (Score:5, Interesting)
Take a look at one of the Agile Poster Children and his proof [agilemodeling.com] that it works.
Quote: "Because of the newness of agile methods there simply hasn't been sufficient time to prove that they work in a wide variety of situations."
Thats a wonderful way to dismiss anyone saying bad things, and it's rubbish, because the burden of proof for any claim is independent of its age.
Quote: "the question "where is the proof" is typically asked by organizations that fit the late majority or even laggard profiles ... Because agile techniques clearly aren't at that stage in their lifecycle yet I believe that this question simply isn't a fair one at this time."
So the act of asking for proof these things work means you're not ready? Ad hominem alert.
Quote: "Are they really interested in finding an effective process or are [they] merely looking for a reason to disparage an approach that they aren't comfortable with? Are they realistic enough to recognize that no software process is perfect, that there is no silver bullet to be found? Are they really interested in proof that something works, or simply an assurance of perceived safety?"
Ad hominem again.
Then you look at the project that started Agile, the Chrysler Comprehensive Compensation (C3) project. It was lauded as the first agile program and a success, however by February 2000 with the system was failing when paying 76,000 of the company's 86,000 employees. It was cancelled. Apparently this failure is now the new success [c2.com].
Every methodology has rapid followers who will hear not evil said of it, but when looking at these things you have to remember "He's NOT the Messiah ... he's just a very naughty boy."
Re:big things (Score:3, Interesting)
My guess is that XP is working only because it banished the "good on paper" metodologies and because of the refactoring formalism. All the other points you cite are a description of "good management", and as so, don't change just because the company adopted another metodology. But getting ride of a lot of useless paper (by letting the programers decide what is usefull) is the way to go.
Developer's opinion of scrum (Score:3, Interesting)
Previously management just did whatever felt right.
Sometimes this was very good, sometimes it was very bad. Sometimes it was just inconsistent.
Now management has a defined methodology that they follow. There are some rules. The rules need not be particularly great ones (although I don't mean to suggest scrum isn't good), just so long as management is thinking about them and makes a concious decision to be consistent and let develoeprs know what to expect.
Specifically, scrum has helped us overcome the "holy shit, its a big customer bug!" panic that happened occasionally. We still panic, but its not the entire organization jumping onto one bug, its just a single scrum team.
Posting as AC, as my coworkers AND management read slashdot and will recognize me.
scrum experience (Score:3, Interesting)
Meetings went something like this:
Go around the room, and say #1 - what am i going to do today, #2 - whats in my way of getting #1 done. One two people were allowed to talk, the person who's turn it was, and the manager in charge of the meeting. If another person in the meeting was the cause of someone's #2, the manager would turn to them, give them (and only them) them the chance to respond. Lather rinse repeat.
There was no "I did this yesterday" because a) we supposedly heard about that the day before, b) the assumption was that you got it done.
Even with at least three different projects going on, and maybe 15-20 people in the room, we were out of there in 30-45 minutes. Any major issues were taken offline so that the rest of us could get back to work.
We usually had only one meeting a day, sometimes two. I found it worked extremely well with a minimal amount of thrashing. We might have been using a modified version of scrum; can't remember - those were dotcom days, everything's still a blur.
Agile & Scrum work for us (Score:5, Interesting)
We don't, however, do that much pair programming. And the whole completely open office space works for some, and definitely not for others. For myself, I'm way too easily distracted -- so I need a nice quiet and private cubicle in order to achieve the state of "flow" where I can write code. In my experience, pair programming works for debugging and integrating code -- and not so well for creating it. YMMV.
Sketchbook has come in on time and on budget, and with extremely high quality. Agile and Scrum had something to do with it. I think the fact that we had a clear vision, a small and very experienced team, a really good working relationship with our usability team and research team, great QA, and excellent management had at least as much to do with it.
As a process, its the only one I've seen in 20 years of doing this that actually makes the life of a programmer better, not worse.
Shorter release cycle? (Score:3, Interesting)
No. No one wants such a short release cycle but you and your shareholders. If I may borrow one of your favorite words, there is not enough "innovation" in most of the technologies you purvey to justify an 18 month release cycle. You managed to pull it off for Windows XP and did nothing but piss off those of us who bought your line about Windows 2000. To us, it was clear that XP was nothing more than the finished version of Windows 2000. We had just spent a fortune on upgrading to the future only to be told 18 months later that we weren't worthy of free utilities, functionality upgrades, or even comprehensive service packs since we weren't on the latest release. As far as I'm concerned, you can keep your interim versions to yourself. Anything shorter than 3-4 years for operating systems and server products is lunacy.
Re:But that's typical. (Score:5, Interesting)
The Unified Process precursors were initially developed by consultants helping improve the code quality at Ericsson. The work was mostly in the area of voice switches and network management equipment. These consitute a specific field of software design which is quite different from the rest of the industry.
The most important difference is that there is an existing specification to which the software must comply. The specification already defines what the software is supposed to do for each allowed input, what are the error conditions, how to handle errors and this is usually defined as a set of simple and very strict rules. This type of task can be very easily expressed as a flow chart. The data objects are mostly defined in the protocol spec so there is no data design work to wrestle with. The spec rules are trivial to map to elementary state machines and are usually very small and well defined. They can be easily written with test cases and unit tested. And most importantly there is plenty of system resource to implement them.
While the methodology behind this type of work can be applicable to other fields there is no justification whatsoever to state that it is the only correct methodology.
It is not applicable to systems whose behaviour is mostly determined by a resource constraint.
In order to apply it to a system you have to define the specification first and express it in terms which are suspiciously similar to a Telecom switch spec - trivial actions with well defined input and output.
It is not applicable to systems where the conditions determining the change of execution are complex and cannot be expressed in terms of simple rules. Best example is possibly heavy duty math. It is nearly invincible to UML, UP and Agile attacks.
So on, so fourth.
We use scrum, and it's working for us (Score:3, Interesting)
But like any process, scrum won't work unless you have buy-in from every level. I think it took us almost a year before we really got into a groove with scrum and started getting really big benefits out of it.
Developers now work without meddling from management for at least the duration of a sprint (a month). We can focus and get lots of work done.
Transparency has built trust between the developers and the other stakeholders, like testers, usage experts, and management. There's far, far less tension between these groups. And whatever tension that does exist is kept off the shoulders of the developers.
We were a small company, bought out by a very large one, and now our group and our process is starting to be viewed as a model for other groups in the company to emulate, because we're (apparently) far more efficient, and we're getting a lot more work done.
We don't use XP (although we do have a lightweight code review process). The benefits of XP weren't quite as evident to us, so it's not something we mandate - developers can do it sometimes at their discretion.
Re:So let's get this straight (Score:3, Interesting)
You've given us a rocking-horse analogy, and you've given us a purported example of a failed project that used it. But software projects go massively over budget all the time, and they get cancelled all the time. Given that people often learn the wrong lessons from failures, the negative opinions of the methodologies by GM executives may not be a compelling indictment of the methodologies themselves.
Or, to put it in a more Slashdot-friendly way: Who cares what a bunch of incompetent blowhards in suits think?
Seriously, though. What specific problems have you had using these methodologies?
I've tried XP, though in an academic setting on a small project (about two months), and I thought it was the most fun I'd ever had programming.
The real question is... (Score:3, Interesting)
From my experience in corporate culture I suspect the latter. What often happens is a group with in a company adopts a methodology which works for them. They then become the poster children for 'organizational change'. The processes are then rammed down the throats of other departments who may be resistant to change, or for whom the processes are inappropriate.
Managers will go to be trained in the new metods and learn the form but not the true spirit of the reform. 1/2 hour scrums will begin to creep up to 2-3 hour daily meetings. In order to take advantage of the accelerated release cycle, products will be scheduled for release sooner, and testing/QA will suffer. Quick fixes will be prefered over true fixes and, inevitably, release will slip. That is my prediction.
Truly changing an organizaqtion takes years. Everyone originally working under the old system essentially has to be fired, quit, transferred, retire or die (one article I read, though I cannot find the reference, said it takes about 20 years for a large company to change).
This might be interesting to watch, in a morbid sort of way. MS is looking for a magic bullet, and we all know how well that works.
Re:So let's get this straight (Score:2, Interesting)
that's the amazing thing about XP (Score:2, Interesting)
I think the lack of meaningful design and analysis is the real killer of XP. XP relies on a methodology called continuous integration to overcome this deficiency. Continuous integration is simply a way of saying continuously change the design of the architecture to meet the needs of the day. And so you end up continuously rewriting the same code to meet new usages. XP has another concept called test driving development which dictates unit tests are written before the actual functionality. So when you continuously integrate, not only do you have to continuously update objects that use your code that you're changing all the time, but you also have change and maintain the unit tests already created. This whole series of chasing your tail can be avoided with a little forethought (aka design and analysis) to what you're doing.
XP can be a useful methodology in environments where the requirements are changing daily, there are a lot of unknowns and risks in the project such a dependency that doesn't exist yet or that is unstable and so changing a lot or projects whose future from a business perspective is questionable. But overall, XP is as big of a joke on the development community as scientology is to religion. Both are pushed by zealots detached from reality and neither has any evidence to back its dogma.
Since I bashed XP above, I'll enumerate some of the good things about it.
1. Continuous integration if done in a more sane manner like coming up with a design before hand, but not being afraid of changing it when needed and having tools to manage changes (like a continuous integration server)
2. quicker release cycles
3. continuous input from the project champion and business users.
4. having unit tests
5. daily meetings on the issues of the day
6. making a business case for refactoring code instead of doing so because it's cool or technically correct.
7. building only what you need and nothing more
Scrum worked well for us, but not a cure-all (Score:3, Interesting)
For us the weakest part of the scrum process ended up being the time tracking (which is really cool in theory and draws pretty pictures for sr mgmt on progress). This isn't due to a fault in the concept, but the nature of our workload. Many of the groups were still heavily into the 'R' side of R&D at this stage, and its very hard to predict what you'll turn up and how much that will cost you when you're still in research and design work. From a mgmt point of view this looked like us slipping daily on the charts, which caused some bad feelings.
Once things moved to implementation and testing in a given group the scrum stuff worked brilliantly. As one of the team leads I generally dislike excessive meeting time (preferring instead more informal 1:1 or 1:n in the hallway or on IRC), but these got short enough and had enough value they were worthwhile.
It really does help to force everyone to think about what they've accomplished in the past day and 'promise' what they expect to accomplish in the next day to their team. We didn't have any real slackers, but just spending the couple of minutes planning out your day enough to tell everyone else what you'd be up to was very beneficial.
Generally the 'what help do I need' part of the meeting was the least useful, as most people would IRC or email around directly (perhaps at the cost of some NMI style distraction) and not really ever come to a meeting needing anything. It was still IMO worthwhile.
Scrum only worked when we could break down implementation into bite sized chunks (no more than 2 days I think is the guideline in the book); at the risk of repeating myself it really didn't work well going into a big problem and trying to work out a plan and design.
Re:Agile & Scrum work for us (Score:4, Interesting)
No, I would submit that those qualities had everything to do with it, and probably a lot more than anything that implementing Scrum could ever do. Consider yourself very fortunate to work with such people, in such an environment. I know I do.
Scrum and other such overarching methodologies are simply ways to enforce standardized positive behaviors upon otherwise mundane workers, or good workers lorded over by second-rate managers who themselves need some kind of framework to tell them how to handle their particular herd of cats. Managed-incompetence, you might say.
On the other hand, compact, tight-knit development groups (like the one you mentioned) have usually already worked out highly efficient methods of getting their jobs done, methods that directly apply to the type of products under development. That happens almost naturally when you have intelligent, motivated people with good communication skills who truly want to cooperate with each other. The group I work in is one such: after working together for years we all know each others strengths and weaknesses, and when given a directive we automatically assign the best person to the job, and if it requires more than one of us, the person best suited to take the lead just assumes the role. And that happens because our manager trusts our judgment and doesn't treat us like children, as some do. Granted, most of my coworkers have at least twenty years in software development. A team composed of fresh-out-of-school programmers would be a different matter entirely.
I guess what I'm saying is that if a company is fortunate enough to have a development group that is already fast, efficient, produces quality work and has that certain esprit de corps that is the hallmark of a good team, don't mess with it. That kind of team is a corporate Golden Goose, and it is surprisingly easy to kill. I'm not saying that there isn't always room for improvement
We don't, however, do that much pair programming. And the whole completely open office space works for some, and definitely not for others. For myself, I'm way too easily distracted -- so I need a nice quiet and private cubicle in order to achieve the state of "flow" where I can write code. In my experience, pair programming works for debugging and integrating code -- and not so well for creating it. YMMV.
If I had mod points I'd give you a +5 Right on the Money for that one. Actual programming is a solitary effort, and a work environment should reflect that.
Read a little (Score:2, Interesting)
Re:So let's get this straight (Score:2, Interesting)
Also look at Apple's software before Steve came. MacOS was absolute crap. Copland, which was supposed to make MacOS a modern operating system, was a late, dismal failure that was eventually killed because Apple was too incompetent.
Apple is not a good example of what to do. Not every company can be a personality cult.
Re:that's the amazing thing about XP (Score:2, Interesting)
That's pretty much exactly not continuous integration. I have my doubts about your understanding of XP.
Re:This is a new thing? (Score:3, Interesting)
Waterfall make work in theory.
In practice I've yet to work for a business willing to wait 2 years for their product. Where the business environment, aims, markets don't change for that period of time.
Pragmatically Waterfall doesn't work. It often almost works, and most developments muddle through. But my customers want new functionality next week, not next year. They'll wait until next month, but any longer and they start complaining.
Since I know of ways of successfully delivering next month, I'd be failing them to force them to wait until next year while I kick off a full waterfall process. Especially since they wouldn't give it the support it needs to work properly.
I can deliver every 2 months. I can deliver approximately bug-free code in that timeframe. I can deliver genuine business advantage, meeting actual business need and keeping customers happy. And I can repeat it.
Other engineers don't write software the way they build bridges, design power stations, construct roads. Oddly enough some of them have tried - and if they'd succeeded they'd be winning ALL the business. Different environment, different constraints, and therefore different approaches needed.
Incidentally, "every step must be done in order" is very wrong. Test before you build - how else do you know you built the right thing. (And test again after.)
You also forgot various steps, but that's a whole other discussion..
Re:XP level of testing leads to brittle code (Score:2, Interesting)
I'd also have to say that you (self-admittedly) sabotaged the unit testing process [slashdot.org] automatically diqualifies you from providing an objective opinion on unit testing.