The Programmers Who Want To Get Rid of Software Estimates 347
An anonymous reader writes: This article has a look inside the #NoEstimates movement, which wants to rid the software world of time estimates for projects. Programmers argue that estimates are wrong too often and a waste of time. Other stakeholders believe they need those estimates to plan and to keep programmers accountable. Is there a middle ground? Quoting: "Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software. Also: Managers have a habit of treating developers' back-of-the-envelope estimates as contractual deadlines, then freaking out when they're missed. And wait, there's more: Developers, terrified by that prospect, put more and more energy into obsessive trips down estimation rabbit-holes. Estimation becomes a form of "yak-shaving" — a ritual enacted to put off actual work."
Simple methodology (Score:5, Funny)
Good managers get my best guess.
Bad managers get my worst case.
Horrible managers get my resignation.
Tag: WORKS4ME
Re:Simple methodology (Score:4, Insightful)
One would hope that a good manager would have enough practical and direct experience in writing software to at least come up with a half-decent estimate, no?
Most shops I've seen lately have the scrum masters spend a part of a planning session simply asking individual contributors "Here's a rough outline of the proposed project [...] now how long do you think doing that will take", and they come up with an estimate adjustment from there... most of the time, it's fairly close. PMs pad things a little of course, but the results tend to be fairly close.
YMMV of course... depends on who is posting the final estimates - is it devs, or is it the MBAs.
(If it's the latter, run like hell.)
Re:Simple methodology (Score:4, Insightful)
Then something changes and blows the estimates out of the water. But the MBA's think since only one line in the spec changed, the schedule should stay the same.
Re: (Score:3, Insightful)
Re: (Score:3)
Then something changes and blows the estimates out of the water. But the MBA's think since only one line in the spec changed, the schedule should stay the same.
More often in my experience the managers and the users refuse to believe the estimates and force new "more realistic" estimates to be submitted.
Because "It's Simple! All You Have To Do Is..."
Re:Simple methodology (Score:5, Funny)
My methodology:
If someone gives me their estimate for a software project or task, I double it and add 30.
If someone asks me for an estimate for a software project or task, I rough it out, then double it and add 30.
It's really amazing how much stress that avoids (oh, and it also does a passable job of converting Celsius to Fahrenheit.)
Re:Simple methodology (Score:5, Insightful)
I've been programming for over thirty years, and I can't do estimates. At all.
Even when I think something is easy, it turns out to be hard if there's some snag I didn't foresee. If I think it's hard then I may find out it's not so bad after all because there are already tools to help put. Something that seems intellectually easy may take a long time because it involves a lot of tedious programming or detailed testing. Writing documentation takes me forever, I can not do that quickly at all unless someone wants a plain text file (which no one wants anymore). One big problem here is that I almost *never* program something that's been done a hundred times before, it's always something new, something that requires learning a new system, new hardware, something requiring reading through a lot of documents first, something with a huge amount of unknown variables, and so on. I also want to deliver something that works with minimal to no bug fixes required later, as opposed to shipping on time followed by a long sequence of updates.
Now certainly for some things I can say that I'll be done that night, because it's something really easy, like reverting a change, fixing up an obvious bug, and so on. But even then I occasionally hit a snag that's unpredicted.
The big problem is that the people who want the estimates don't understand this. And they come in all types. There's the person who thinks that a little bit of pressure makes people perform better (ya, you gave us someting to do in 3 months that needs 9 months). There's the person who thinks the estimates are accurate and golden and then sells the product before any work has begun. There's the person who thinks that your estimate means you will be working 100% of the time only on that single project and nothing else. There's the person who has a preconceived idea of how long it will take and considers that any longer estimate is "sandbagging". There are the people who will put me on two of the companies most important projects of all time, at the same time.
At one place, I gave an estimate of a simple task to be "two weeks after the main project finishes I will have the integration finished". So the sales rep wrote down a specific date based on that. The main project was late, but when that date arrived the sales guy comes up and asks me for my piece, which I had not started yet and could not start yet, then he panics because he's promised it to the customer (before even the product I am integrating with even exists). Ultimately I ended up with only two days to work on it.
I seriously wish that people would switch to a model where they announce a product after it is finished, and schedules are not based upon when the next conference or trade show is.
Re: (Score:3, Insightful)
I've been programming for over thirty years, and I can't do estimates. At all.
I've been programming for over thirty years, and I'm pretty good at estimates.
It comes from a practice that I learned at Microsoft. Before Microsoft, I did the usual "Oh, about two weeks" estimate for everything that I was shown.
Design your feature or product. Then figure out how you're going to build it. Then break down that build into tasks that are no less than half a day and no longer than three days. Add it all together and divide by the fraction of a week where you're actually going to be producti
Re:Simple methodology (Score:5, Insightful)
Paint my car black.
So you do all the prep work, car's primed with a dark primer, black paint is mixed and ready to go, then this change request comes in:
Paint my car white.
Well, now you've wasted the paint you just tinted black, and you can't paint white on top of dark primer (well, you can, but you need many more coats), so you've got to redo the prep work. That means waiting while the primer fully cures, so you can sand it off properly; otherwise, it'll gum your sandpaper. then re-prime, then you can paint. Assuming you don't see
Paint my car forest green.
in the meantime.
That was one word. Yes, one line matters.
Re:Simple methodology (Score:5, Interesting)
Most of the delays I encounter are caused by someone else; either the need to refactor someone else's shit code (that I wasn't allowed to review before providing an estimate, of course), delayed approval for the work, delayed access to resources, any number of external forces. Very rarely do I exceed my estimated *hours* for a project unless changes are requested (but then I'm not exceeding the estimate, either, since I make the client either agree to a new estimate or accept a refund of any portion of their deposit not already applied to the hours I've worked), but all too often I find myself completing projects well past their due date because some resource wasn't made available to me until after that date had lapsed.
Fortunately, I foresaw that unforeseen things would happen when drafting the boilerplate language of my SoW and covered all of those cases. I go over the entire SoW with my clients before starting a project and make sure they know what the triggers are for a re-quote, what will cause the project to be delayed, and under what circumstances they're entitled to a refund of any deposit they pay (e.g. if they back out of the project once I've started work, I'm deducting my hours from that). As a result of that, and perhaps a bit of luck, I haven't had any disputes over project scope, budget, or timeline, and the one client I did have back out of a project simply said they no longer had the budget (they were being sued) and told me to hold on to the remainder of their deposit as they'd be back to finish the project after they lawsuit ended, hinting that, even if that didn't happen, the small sum would make no difference going forward (of course, I'm sitting on that money for now, and if they fully back out of the project I'll insist that they either accept the refund or sign a document releasing the funds to me).
That was one thing that really pissed me off when I was working for someone else; I had no control over external interruptions or delays and it was usually the person interrupting and delaying me who was holding me accountable for all of them. I'm not out to scam anyone, but I always felt like I was when dealing with my previous employer.
Re:Simple methodology (Score:4, Insightful)
No. That is the whole point, which you have seemed to miss. I'm the software engineer and even I can't come up with a reasonable estimate; why the hell would some manager several layers of indirection distant from the design be better at it?
Re: (Score:3)
Well a good manager, will work with the developer and give them clear deliverables, adequate milestone achievements even if these milestones may not look like much. Then we can give a good estimate.
A Bad manager will toss out a pie in the sky grand vision that does something. So the programmer will need to go back and rework idea over idea, going back and improving on the design.... So this will take a worse case scenario.
Re:Simple methodology (Score:5, Informative)
You can't really give a good estimate up front on most projects, because even if the requirements are well defined (and they rarely are) they will change.
That's all OK. That's the whole point of Agile, if you ignore the Agile BS that consultants sell. You can quickly do coarse-grained task breakdowns (say, to 2-week tasks) to get a reasonable estimate of the work as you currently understand it: that estimate is generally within a factor of 2, if the requirements don't change, so you can at least see if you have 1/3rd the staff you need or something way off.
Then you measure real progress against that first-take estimate. Usually by about 6 weeks in on a team-sized project, you'll have the real multiplier. "We thought the first set of tasks were 20 programmer-weeks, but really as we understood it it was 30", and multiplying that initial "6 months" estimate by that measured 1.5 multiplier is a damn good estimation method. And it only gets better as work goes on. I can reliably give fairly accurate estimates from project completion now, on Agile projects, by the time we're about 1/3rd of the way done (and that's so much better than some teams are used to that it seems like magic).
Selling that all properly to management is a different thing, but that's a whole job skill that any senior engineer just needs to have. Just like showing management the cost of each requirements change, as well as the skill to minimize that cost of change (which is the real point of Agile- the rest is hype).
Re:Simple methodology (Score:4, Interesting)
A well defined project can be estimated. Change Orders estimating needs to be done properly and BILLED - the cost of re-analyzing is a real cost and the business needs to see it. Once you get that across the number of change requests decreases dramatically.
Software engineering is Engineering... some of the costs are inverted, but otherwise it's the same project management as other engineering projects.
Re: (Score:3)
A well defined project can be estimated. Change Orders estimating needs to be done properly and BILLED - the cost of re-analyzing is a real cost and the business needs to see it. Once you get that across the number of change requests decreases dramatically.
Software engineering is Engineering... some of the costs are inverted, but otherwise it's the same project management as other engineering projects.
Correct. Our rule was "Give away the initial project because we'll retire on the change orders..."
Re: (Score:2)
Re: (Score:3, Insightful)
Exactly - that is why there was been a growing recognition that Agile isn't getting the job done. At my last company a bunch of Agile zealots came stampeding my way trying to say that we had to change all our development processes to be "agile" because, well, just because. So I went through a simple requirements exercise with them. The first and foremost requirement is that when sales/marketing sales that they want 100 features in the product and there is no way that we can doo all 100 features, we have
Re:Simple methodology (Score:4, Insightful)
Sorry bro, but if you got silence in response to that question, those weren't "Agile zealots." They weren't even "Agile newbies." They were ignorant cunts.
Agile does NOT say "start coding and figure it out as we go." Agile says "Take some small subset of the functionality you've declared "the most important," estimate that, and deliver that."
Somebody has a list of 100 features? Great, estimate business value and level of effort - use story points, t-shirt sizes, etc. That alone gets you a pretty good way to prioritize your list - things that are "small-to-medium LOE, high value" go to the top of the list. Then you estimate those specific features, and start working on them - to start delivering business value as quickly as you can. Then as you go down your prioritized list, you continue refining and estimating the other features and pull them into your work queue as they reach the top of the "Effort x Most Valuable" ranking.
If you're a true Agile team, you're cross-functional, which means that proper representation from ALL of the organizations needed to deliver the product is present, embedded in your team. While devs are banging out designs and code, Testers are working on test plans and test automation given the specs that the devs produce, and Support people are building operations documentation and support matrices etc. - all in parallel, all coordinating with one another as the process goes. A team with proper representation from all functions SHOULD be able to provide necessary dates and timelines, because there is no "Dev team" just throwing dates over the wall to Support and Manufacturing.
Agile DOES answer these questions - if you're able to sit there and pretend that you've never heard the answers, then you're incredibly ignorant, or incredibly invested in somehow feeling like you've "discredited" Agile by sneering at it.
Re: (Score:3)
The first and foremost requirement is that when sales/marketing sales that they want 100 features in the product and there is no way that we can doo all 100 features, we have to be able to prioritize them
Yes, that's exactly how all Agile development works. Did that not work out for you?
Now, please tell me how Agile is going to tell me how much effort is going to be required to complete a particular project when the whole Agile approach is to start coding and figure it out as we go?
That's not quite Agile - you need enough of a plan and a design to know WTF you're doing before you start, you just don't need an in-depth design of stuff you won't even touch for 6 months. But you know what the components are, and about how big each one will be, at least relative to each other, and you know where the dependencies are. So how do you know ho long it will take? You measure you're progress! When you know yo
Re: (Score:3)
Re: (Score:3)
Re: (Score:2, Interesting)
Yeah. Estimating the time to develop a simple web page is one thing. Estimating the time to design and engineer a new enterprise system is another. It seems that there are "managers" who think the first can be translated into the second. I'm sure Cheops asked his engineers to specify how long it would take to build his pyramid... The answer? "It depends!". Depends upon how many workers we can get, how far we have to go to get the stones, what the weather will be like...
Re:Simple methodology (Score:5, Informative)
I had no particular problem with estimates.
At a minimum, you could break out easy "construction/recognized pattern" work from risky new stuff.
As far as managing programmers... it was humorous.
Few liked giving estimates. So they would say it couldn't be estimated.
So I would ask, will this project take 2 years... and they would say, "oh no- of course not" and after a bit, we'd get down to 6 months or 6 weeks or 6 hours...
So then we'd time box it to what could be done in a month and move any risky items up to the front so we could establish if a new technology wasn't going to work before we put a lot of work into the project.
Then, I recorded over/under for every project and found (over about 24 programmer data set) that programmers consistently overshot or undershot their estimates. So after a few projects, I had a pretty good idea of their deliverables.
Finally status reports and status meetings with function points and overall percentage delivered kept things on schedule or let us know well ahead of time there was a problem with the estimate/schedule.
Programmers were not my problem- executives were.
They...
a) pushed us to violate standards.
b) ordered overtime without ordering it. As in assign 80 hours work and then insist it be completed when everyone knew it couldn't be completed. Made worse by the fact the indian contractors said "I'll do my best" for "no- you are batshit crazy" and then things fell apart when the indians were unable to deliver. Of course, the indians were very good at delivering to the (crazy/incomplete) specifications on time. At least enough to be testable. I'm not sure if it is because they were contractor types or that they were indian- perhaps a bit of both. I learned in a contracting shop, you always say you can meet the estimate (to get assigned the work) instead of giving a realistic estimate. Then renegotiated it later when it wasn't going to make the schedule. If you didn't, then the three other people bidding on the work would get the work. Executives seemed to have zero memory for the fact that you delivered on time on estimate while the other people were usually late.
c) made everything priority 1a. they had no ability to prioritize as far as I could see. Which really just pushed prioritization down to me or the programmers.
d) cancelled projects without warning.
ignorant hypocrites (Score:5, Funny)
so, estimating time is a waste of time, but complaining about estimating time is not?
GET BACK TO WORK, MONKEY.
Re:ignorant hypocrites (Score:4, Interesting)
No, you complain only once, whereas software estimation has to be done over and over for every task... for many years. Software coding/design is similar to solving a maze -- you just can't give an accurate estimate how long it will take to solve the maze. Estimates are good for repetitive or non-creative tasks that have been done for many years and you know exactly how long some task takes.
Estimation in the software world is a scam perpetuated by managers and management to get developers to work extra hard.
Re:ignorant hypocrites (Score:5, Insightful)
Holy crap! Are people actually buying into this BS?
Software coding/design is similar to solving a maze -- you just can't give an accurate estimate how long it will take to solve the maze.
WTF? Yes, you can give an ESTIMATE on how long it'll take to solve a maze. Go get a book of mazes; Do 10 of them; Time yourself for each one; You now have min, max, and average times for a maze of that size/complexity. Next time someone shows you a maze, you can make an educated guess about how large/complex it is, then give your best case, worst case, and normal/average estimates. Do the same for programming.
Estimates are good for repetitive or non-creative tasks...
They're also good for creative tasks. I went to college with a major in fine art and worked for years as a monument engraver (etching portraits/landscapes/etc on tombstones). If someone asked about how long it'd take to etc a 5x7 portrait, I'd have a VERY accurate estimate for them. Is that not creative enough for you? How about every single project for every single class for every year of art school? ALL of those had timelines, and every student became quite good at estimating how long each project would take so they could get them all done on time. And you could ask most of them (at least those with higher than a B average) how long it would take them to do a certain thing (ex. sculpt a bust in clay from a live model) and, if they were familiar with that medium, then they could give you a very good estimate.
Estimation in the software world is a scam perpetuated by managers and management to get developers to work extra hard.
If management is asking the devs for their estimate, then how in the hell is it management fault for any of those timelines? Let me put this another way... devs, BE SMARTER. You've been asked for estimates on more than one occasion. Most that are new to development will low ball (stating a best case estimate). If/When you do that, you're just setting yourself up to fail. The best outcome will be that you are on time, and every other outcome sucks for all involved.
If you're not good at estimating, just try, come up with your figure,then double or triple it. If you always come in under the estimate, then you manager may start adjusting your estimates... who cares? Let them. It's their fault if you don't meet their manipulated figure. However, if you're not giving a large enough estimate in order to get your work done, then it is YOUR FAULT if you fail to hit your own estimate!
All that said, there are cases where providing an estimate is unreasonable. On one end of the scale, if a manager is asking for very accurate down-to-the-hour estimates on vague but somewhat small tasks, then it's asking for too much (almost more work estimating doing the damn thing). On the other end of the scale, if it's some grand idea for a giant project with no plan yet, you'll be pulling figures from your ass just as much as he's pulling the project plan out his ass... but it is what it is. Just go big.
A lot of it is just about setting expectations. If you give them high estimates, then you're more likely to meet or exceed their expectations. They may dislike the figure at first, but when you come in under budget they will be very happy and forget all about how high the estimate started out. A high estimate is not "padding", it's setting expectations that you can meet (see the maze estimate above... use "max" and you'll be able to meet or beat your estimate every time, which is what all involved really want out of your estimate).
Re: (Score:2)
Hate replying to myself, but I hadn't RTFA before posting. I'm not entirely against what they're saying/proposing. It's a different way to do things, rather than just some folks whining and complaining and avoiding estimates for stupid reasons.
Re:ignorant hypocrites (Score:5, Insightful)
The difference between software and mazes is that with mazes you are doing the same task every time.
I know, for example, that I can take cold iron and have it running most of what makes it a useful production server machine in about 4 hours. I'll sign my name in blood on it. Because I've done that same task over and over.
On the other hand, software is a creative work and creativity implies that you are NOT doing the same task every time and that at best you are guessing.
The problem is, everyone else is second-guessing, and they're guessing wrong. And they often will not accept a realistic estimate. They'll push for a "realistic" estimate, then blame you when you cannot meet it.
Actually developers guess wrong too - they usually think it's going to take half the time and work it really does (based on stats I've seen). But developers could simply allow for that by doubling what they think.
Except that in many cases, as I said, they cannot resist the pressure from outside.
Re: (Score:3)
Estimating the size/complexity of a software project is exactly where all the error lies. The original quote was correct that it's like solving a maze, you just assumed they were talking about mazes on paper that you could estimate size and complexity at a glance. No, this is a full sized maze that you walk through and have no idea how deep and complex it is.
Re: (Score:3)
I am absolutely an expert. Sometimes a lot of of my time is wasted by people needing me to help them out, delaying me from getting my own work done. And I can't estimate. Sometimes because I'm not good at telling people to go away and leave me alone, but usually because unexpected things happen, or I'm being shuffled around between two many projects and can't multitask well, or I'm doing something new I've never done before (but that's ok, people know I'm smart and assume I'll get it all figured out).
And
either Fred Brooks (Score:2)
Well someone has to do it (Score:5, Insightful)
Business can't plan or talk to customers or have any strategy whatsoever without at least some estimate...that's just the real world. If devs don't give estimates, managers have to make estimates. If managers don't make estimates, business makes estimates. You want devs to do the estimating.
Re:Well someone has to do it (Score:5, Funny)
WHY CAN'T WE HAVE BLANK CHECKS
I thought this was funny, but it probably isn't - especially now that I have typed in all lowercase words to get pass the yell filter.
Re: (Score:2)
If a manager can't bring in a project in on time and on budget he is useless.
Re:Well someone has to do it (Score:4, Informative)
IF managers listened to the developers.
My last major project:
Manager: What resources do you need for this project (Replacing a huge, entirely file-based factory product manufacturing system)
ME: At minimum, I need two more people, plus someone to take over most of my smaller duties. And a couple new DB servers.
Manager: How long will this take?
ME: Two years minimum, since we are replacing an entrenched 15 year old system.
Manager's report to director: He can do it by himself in 6 months.
3 months later:
Manager: Why isn't this finished? You started over a year ago!
ME: I started three months ago, you gave me nothing I asked for and you've given me several other projects since then that you said were higher priority.
Manager: Well stop wasting time and get to work! I told the director it would be done by now!
ME: You gave me nothing. At least let me have a CS temp.
Manager: There's a woman in finance who has a sister that might want the job. She knows Excel really well, she can probably learn databases and programming.
Aaaand I think we've all been there.
Fortunately the sister didn't want the job and I got a great college grad as a temp. I would not have finished the project (in two years) without his help, we hired him after a year, too.
Re: (Score:2)
I would not have finished the project (in two years) without his help, we hired him after a year, too.
First thought: your manager was a tool, and generally a waste of space--actually he wasn't even THAT useful, since he actively made things worse overall.
That said... the above quote is a bit damning. You claimed you needed two additional people, an empty task list, and two years. You did the job in two years, with other tasks encroaching on your time, and with a single new grad that you only had for 21 of those months. Either your project was a death march (not ruling that out, mind you), or your estimat
Re: (Score:3)
Re: (Score:2)
Business can't plan or talk to customers or have any strategy whatsoever without at least some estimate...that's just the real world. If devs don't give estimates, managers have to make estimates. If managers don't make estimates, business makes estimates. You want devs to do the estimating.
I suspect the problem is not so much the estimates, it's that a manager demands an estimate from their underlings, tells them they don't like that estimate and give them a quicker estimate, promises their customers certain completion by the quicker estimate, and then complains when the programmers can't finish on time or have buggy code. That's without the obligatory change in the project requirements halfway through. At some places the estimate is nothing more that bending over for the manager.
Obligatory D
After which managers toss the "bad" estimates (Score:3)
My experience has been that management comes to the developers for estimates. They provide those estimates to the end users. The end users bitch, whine, and complain that they need it to be done in half that time.
Management then comes back to the dev team and tells them they've agreed to get the project done in half the time that was estimated.
Then both management and the user community bitch when their "estimates/targets" aren't met, and who is blamed for the issue?
The developers.
The developers
Re: (Score:2)
Re: (Score:2)
The only way to provide a reliable estimate is based upon past experience. A reliable estimate would hence be the same time it took for a similar prior project to be completed.
No understanding of programming is required.
Re: (Score:3)
Without an understanding of programming, you can't reliably tell how similar a prior project is.
Re: (Score:3)
An understanding of 'similar' is required though...and without knowledge of the programming there's no way you'll know that. When the programming is all 'magic,' everything is similar.
Re: (Score:2)
In this case, please inform me what the fuck that useless manager is here for? If I have to do his job, he's essentially a waste of precious oxygen.
Re:Well someone has to do it (Score:5, Interesting)
I was a technically literate manager, having done lots and lots of programming myself. My job was simple: run interference with the client so that my team stayed funded. The team was very happy to let me do that job, which required a lot of travel and unpleasant politics.
In turn I trusted the team. I asked them for realistic estimates to give the client. If the team thought they weren't going to make it on time, I asked them for a heads-up as far ahead as possible, and I would take the news/new estimate for the client. I did not criticize the team, either to them directly or to the client.
They knew they were asked to do their best but software being what it is, they were not held to preliminary estimates. (The only issue was with downright incompetence or negligence, which was very rare.)
It's interesting that I was considered a good manager by the staff and the client, but not by my own management, who said I wasn't hard enough on the staff. Well, sorry, I got results and I kept us funded.
So, was I a "useless manager"? I hope not. I didn't produce anything tangible, and that bothered me, but I hope I played a useful role.
Re: (Score:2)
In this case, please inform me what the fuck that useless manager is here for? If I have to do his job, he's essentially a waste of precious oxygen.
And this is a surprise?
Re: (Score:2)
Yes. The manager's estimate would be preferable.
Without estimates you can't budget... (Score:2)
If you can't, based on the available facts (the design, requirements, etc.) , determine how long it will take to implement a project then you need to find someone who can. And you have to understand that there ARE external factors you can't control and if they change, the estimates change.
And management needs to know that your estimates are based on specific things (the requirements not changing). Any changes to requirements, etc. will impact schedule
If your company puts the budgets into stone, that's a di
Re:Without estimates you can't budget... (Score:5, Funny)
Lets see... What would they say? This is the one-sided conversation, since it doesn't matter what you say anyways.
"Ok, we can accept that estimate."
"Ya, ya, ya, whatever."
"We'll have that information to you by the start of the project."
"The information isn't ready yet, we'll have that by the time you need it."
"I thought we had that to you already. We'll have to check with the information source."
"The PMs have some changes."
"Here's the information, but there are some small changes."
"No, those are small changes, they won't impact the timeline."
"No, you can't have more time, we already made commitments."
"The PMs have some changes."
"What do you mean you won't have it in on schedule? You agreed with the initial estimate."
"You're going to stay here until it's done, I don't care how long it takes."
"I don't care that you've been in the office 30 hours straight, this is your fault."
"We're hiring an off-shore company to help you with the project. Get them up to speed."
"The PMs have some changes."
"Since we have the off-shore team, we need to cut your department back."
"I read an article saying Java is the future. Redo it in Java."
"What do you mean we're waiting on the off-shore company?"
"We fired the off-shore company. You're good, you can get it done in time."
"Ok, hire more people into your department, but we're only offering half the salary, and no more bodies."
"Why is this project so far behind? Don't you know what you're doing?"
"The PMs have these changes."
"Why aren't you done? We're weeks from the deadline!"
"You didn't meet the deadline. Don't you know deadlines are firm. We have commitments."
"I don't want excuses, I want results."
"You and your idiot team are fired. Get out of my building."
[2 months later]
"We need you to come back and finish the project. We need it by next Monday, that should be plenty of time."
"Here's all the new specs. They should be easy to do."
"What do you mean total rewrite, it's only a few chances. You are an idiot. Get out."
[1 month later]
"We need you to come back and finish the project. We need it by" {click}
"We need you to come back and finish the project. We need it by" {click}
"We need you to come back and finish the project. We need it by" {click}
"Why do you keep hanging up on me?" {click}
Good method for improving (Score:5, Insightful)
This is a trick I learned from agile (although I'm sure it's been around longer than that). Each time you make an estimate, pay attention to how accurate it was, then next time adjust and make a better estimate based on that. Estimating shouldn't take much of your time.....if it does, then you're doing it wrong.
These guys seem to be complaining that their managers aren't doing anything with the estimates. Which probably means they don't understand what the managers do (but it could also mean their managers are bad).
Re: (Score:3)
Re: (Score:2)
This is a very insightful post
Thanks!!
Re:Good method for improving (Score:5, Insightful)
Good managers are almost as rare as honest politicians.
Then teach your manager to be better. Reject the corporate structure (it may shift many times while you work at a company, even though not much else changes. That is an indication it's not particularly real), accept that improvement can come from even the lowliest programmer, and then help your manager do better.
If you think that your manager is completely bad in every way, you're not going to be able to help him. You need to be honest, recognize his strengths, and help him overcome his weaknesses.
You'll probably also need to overcome some of your own weaknesses, unless you are perfect. I can tell you one way to not accomplish anything: start a twitter campaign telling the world how bad your boss is. That will not make your system better.
Parkinson's Law (Score:4, Interesting)
Predicting the future is hard (Score:4, Informative)
Joel Spolsky has a take on this problem, called Evidence-Based Scheduling [joelonsoftware.com], which tracks past estimates against their deliveries, and uses that to improve future estimates.
Re:Predicting the future is hard (Score:5, Insightful)
I've seen that sort of thing used fairly effectively. If you keep track of how long past projects took, it's fairly easy to estimate a new project based on an estimated ratio of how complex the new project is compared to the most similar past project(s). This takes practice and a bit of a knack to do well, but it doesn't take much time, and it's certainly better than nothing.
What doesn't seem to work well, in my experience, is breaking down a project into microscopic detail and individually estimating each detail, e.g, the "Microsoft Project" approach. I can understand the appeal of that sort of thing, but it's almost impossible to use a data-driven approach at the microscopic detail unless you use some sort of "Personal Software Process" tracking tool - which very few people want to actually do because it feels like you're instrumenting yourself as a lab rat in someone's twisted experiment.
In any event, having a plan and a schedule, though inevitably imperfect ones, helps motivate everybody and helps keep things on track. The more enlightened managers of the world will allow these things to be revised along the way as needed, as their imperfections get revealed.
Re:Predicting the future is hard (Score:5, Informative)
I work for Joel Spolsky and we have a single estimate: 6-8 weeks.
Estimation is basically useless because it has the unspoken assumption of "perfect design", which is an invalid one: software development is not a repeated exercise, it does not have consistency.
Having a single estimate does this to your projects: much smaller things, just do them; much bigger things, they are too big so don't do them, things vaguely in that scale, do them iteratively but define what you are trying to achieve in writing so you know when to stop.
No project managers, product managers work at a way higher scale (decide which are the projects worth working on).
Re: (Score:3)
So is the system in the linked 2007 article no longer used, never was used...?
Re: (Score:2)
Unfortunately most of the projects I've been on are of the "We want to replace old system A and make a new system B that also has features X, Y, Z" variety. What they want from the new system is usually okay, it's somewhat documented already, you've identified some stakeholders, you can show them prototypes, you can ask for clarifications and their testing validates the feature. Developing new code for genuinely new features is actually quite easy and fun.
The old system on the other hand is more like softwa
Re: (Score:3)
Very often the original system was hacked out in a couple of days/nights by one or 2 heavily-medicated (caffeine, alcohol, whatever) people.
It more or less did the job even though it has one or 2 really awful (but infrequent) bugs and is difficult to maintain or expand.
Then one day someone decides it's time for the Second System (see Fred Brooks).
They put together a team and a schedule and - like as not - spend a lot of money on fashionable faddish tools and consultants and a lot of time coming up with bell
Depends on what "delivered" means (Score:5, Insightful)
On the flip side we have so-called "agile" development teams who simply define a deliverable as whatever they've completed on delivery day. These developers rarely tell management until the 11th hour that what they're going to deliver is below spec. Agile development has its strengths, but this aspect is a giant weakness. The solution is not to eliminate schedules. It's to adapt them to changing conditions and be proactive about slippages.
Re: (Score:2, Interesting)
I always get my coder to give me an estimate and I keep track of both the agreed upon and expected delivery based off history.
I always negotiate any changes I make to anything with the coder IN WRITING as stated in the contract.
If I'm changing the spec the coder gets to change his estimate of both time and price.
As a result I make damn sure my spec almost never changes.
It incentivizes me to know what the hell I want and not turn into one of the shitheads I've done projects for.
My coders work with a contract
My rule of software estimates (Score:4, Interesting)
Part of this is the stupid human trick factor - if we think something is going to be difficult, we approach it differently and with more caution. As a result we're more able to identify things to make the project go smoother. Conversely, if we think something is going to be easy, it's because we've only determined one path to approaching it, and if that path turns out to be non-viable, we have an immediate delay as we scramble to find other solutions that will work.
There's also the psychological factor at play for the customer - if we say something is going to be ghastly and difficult and will take us a long time ( because we think it will) but then turn around and deliver it ahead of schedule and under budget, we look good and feel good. This does not work if you say something will be ghastly and difficult but secretly think it will be easy and fast, however.
First, get your story straight... (Score:5, Insightful)
...Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software...
...developers' back-of-the-envelope estimates...
From the above two quotes, it looks as if the author does not even know if too much or too little time is spent on estimates.
.
No wonder his managers are frustrating with the time estimates provided. The guy has not a clue.
It's research (Score:4, Insightful)
Programming is basically research and development You are building something which has never been built before. As such how do you estimate something if you have nothing to compare it to?
Actually though I have a limited amount of experience using Function Point Analysis. Some of the aspects of it are old, it was created before web apps became pervasive, but that doesn't matter. The principles are the same and there are dimensionless constants you can use to tune the FPA model to your organization. Eventually you end up with an empirical model of your programming team, development environment, tool suite, management efficiency, complexity of the problem domain, people quitting or dying, etc. You need to avoid over fitting however and recalibrate from time-to-time.
Agile keeps story points and velocity which serve the same purpose as the dimensionless constants of FPA. Unfortunately managers try to translate them to mythical man hours. But if you keep velocity or function points, whether you are using Agile or not, you should be able to get some idea. You have to be a bit scientific about it.
That and 'T-Shirt Sizing' is about as good as it gets.
I wish I had training (Score:2)
I have similar feelings about estimates in my work: It's never accurate, so why bother. Of course, I understand we need estimates.
I wish I had some training in how to make good estimates.
Recentely, I've been giving my managers a range: between 1 and 3 weeks for this feature. Half a day up to a day for this bug fix. Of course, manamament just takes the average, add ups the numbers and doesn't tell the client about the probability distribution.
Not just software (Score:3)
Projects that deal with a small workload and don't have changing requirements are much more likely to stay on budget and on time. This is how things should be broken up. Build small pieces and deliver the pieces as they become complete. Don't set out to build an entire 5 year task as a single project.
Hmmm .... (Score:5, Insightful)
On the one hand, it's pretty much impossible to do any project planning with no estimates.
On the other hand, some things are impossible to estimate until you do them.
Years ago I worked with a manager who kept repeating that bad estimates were a project risk and we should give good estimates. We kept telling him that an estimate is, by definition, based on incomplete knowledge and before you have done the work and that if he had a time machine we could give him better estimates.
If I knew exactly how long it would take, and what unforseen things I'd be running into ... it wouldn't be a frickin' estimate, now would it?
People treat estimates like you're expected to have perfect knowledge of the future, and then build their world around those estimates.
I have seen a tremendous amount of bullshit and stupidity from people who do not understand what an estimate is, and how it's done.
I don't think you can get rid of estimates entirely ... but management needs to stop being so stupid about how they interpret them.
If we could tell you for a fact exactly how long it would take, it wouldn't be a fucking estimate.
I rank how people do estimates right up there with how some PMs want you to track your time ... once had a PM say he wanted me to account for my time in 5 minute increments. And I told him in no uncertain terms that would mean 2 out of every 5 minutes would be spent documenting what I'd done the last two minutes, and there would be an additional 1 minute of lost time in each 5 minute increment doing to context switch back to what I was working, and that effectively 60% of my time would be wasted on his stupidity.
And then I told him to piss off.
Re: (Score:2)
I agree. Estimates aren't a problem for teams that understand what "estimate" means. It isn't exact. It can't be exact. As long as you respect it for what it is, then it is a powerful planning tool.
The next time a business person gripes about estimates not being accurate enough, ask him/her to estimate (to the minute) how long it will them to drive home during rush hour traffic. When they start complaining about how an unexpected accident would cause the estimate to be very inaccurate, then a light bul
Re: (Score:3)
A couple of observations.
There is a hellacious difference between an estimate and a deadline. A lot of the problem is abusing estimates and quietly or unconsciously turning an estimate into a deadline. The process for producing the two is totally different, or at least should be.
One very good manager I had would never directly ask me for an estimate. He would ask me how long it would take to figure out how long it would take to code x. If you can make a reasonable estimate of the size and complexity of
Selling ourselves down the river (Score:2, Interesting)
Research into procrastination has noted that people have much less concern about their future selves than their present selves and are willing to sell their future selves down the river for the sake of present ease. But when the present marches into the future, and we are confronted with the work that our past selves refused to do, we pay the price in unmet deadlines, all-nighters and general torment.
http://www.nytimes.com/2015/01... [nytimes.com]
The result becomes one of people providing estimates that allow them to scr
"It's hard, so we won't do it" (Score:4, Insightful)
Part of being a software professional is understanding how long it takes for you to do things
If you don't know how, go read a book. It's not hard. The money that's used to pay you depends on your estimate being somewhat correct.
Imagine how upset you would be if you asked for your paycheck, and payroll said "we're not sure when we're going to pay you, but it should be sometime soon."
Re: (Score:3)
Re: (Score:3)
Yeah, and tell us, what is the track record of the financial and executive teams ability to prognosticate?
Yes, you need to be able to estimate to run the business.
But let's not pretend the average CEO, sales wanker, or marketing idiot has ANY better track record at making guesses about the future. In fact, in my experience, they're overly optimistic, not founded in anything real, and mostly pulled out of their ass of based on what
Re: (Score:2)
Re: (Score:2, Insightful)
I don't know about you but I'm not making cookies. Sounds like you're not doing any design either...or implementation in the real world when bugs appear, refactoring is required, etc.
Tilting at Windmills (Score:5, Interesting)
I used to tilt at this very windmill myself. It took me a long time to realize that people really, really need to hear an answer to "how long?". When we were in startup mode a long project was a matter of a few weeks. Everything had to go into production immediately. So we got used to banging stuff out in small chunks and doing it as quickly as possible. Entire project timelines were completed in less time than it takes to draft a proper requirements document.
But as we grew in size, the new people were not happy with our development team. Even though they would ask for a new report at 10am and have it in production before they returned from lunch, they still felt like we were not responsive. It was one of those reports that began to help me understand the psychology.
There was a problem with a very complex Crystal Reports document one morning. The Director of the department called to let us know about it. I told him we were already working on it and would have it fixed as soon as possible. "When will it be fixed?" he asked. Well, I had no clue. We had only learned about the problem 10 minutes before and hadn't figured out what was broken. So I explained this to him and told him that we had this as our top priority and it would be fixed as quickly as possible. I certainly thought that should make him happy. Well, it didn't.
He was rather pissed that I wouldn't give him a timeline. The day before we had made some changes for him that took about 2 hours, but he was upset that we didn't let him know how long it would take. Being an engineering type, when I hear "how long will this take?", I hear a request for a certain degree of precision. The problem with short projects like this is that by the time you have enough information to give an honest estimate, you are pretty much done. Maybe you have 15 minutes or an hour to go, but nothing worth reporting to anyone. Well, after explaining my position a few times and just making him more angry I finally gave up and just gave him a made-up deadline of Thursday afternoon. He was perfectly happy. I had just spent 15 minutes trying to explain to him that we were working feverishly and would be done as soon as we could (which would likely be a couple of hours, but who knows) and he hated that answer. "We'll have it for you in 3 days!" made him very happy. Even though his production was impeded by the lack of this particular report. From a human psychology standpoint he would rather know that it will be done in 3 days, barring delays, than not know when it will be done and have it in two hours. I personally think that is a dumb way of doing things, but I am the outlier, not the director.
After that learning experience we began implementing a more comprehensive SDLC process and providing timelines for projects. Everyone was much happier with the development team after this. Even though their projects went into production much more slowly. They loved the perceived control that having a timeline gave them. We developers know that these things are basically fictional documents - just educated guesses really - but it provides real customer satisfaction, so we keep with it. In fact, we kept evolving this idea into more and more involvement from the business unit as we moved into Agile and SCRUM methodologies.
I would say that unless you are working in an organization of less than 25 people, providing timelines is an absolute requirement from a purely human standpoint. This comes from hard experience - even though I think that everything about a timeline is probably B.S. and all of the effort that goes into preparing one would have been better spent solving problems and building something useful. In the real world you can't ignore the psychological needs of the group.
Re: (Score:3)
From a human psychology standpoint he would rather know that it will be done in 3 days, barring delays, than not know when it will be done and have it in two hours. I personally think that is a dumb way of doing things, but I am the outlier, not the director.
The psychological issue is that you don't know, but you have a hunch, you have some insight. You know it's probably going to be a few hours.
But for non-techies, all this stuff is a total blackbox. When you say "I don't know" they panic, because for them that means anything from a day to a month or maybe infinity. Uncertainty is a horrible psychological state and people try to avoid it. It's an instinct. When you don't know if that shadow is a monkey or a lion, it's better to panic just in case.
By saying "th
Typical Estimating (Score:2)
If you need an estimate real bad, you'll get a real bad estimate.
The real problem with "time writing" is that people end up lying on the reports to meet goals or metrics and then the estimates created from that data are useless. Managers want both though. They want accurate estimates AND time tracking that meets productivity goals. Sadly their productivity goals would require robots to meet...
There's One More Pitfall (Score:2)
Let's say you've figured out pretty well how long something will take and you give your realistic estimate. The game from the managers then becomes, "That's too long, you need to give a more realistic deadline."
This pressure on the other side is often why developers set deadlines that are too short and then miss them. The penalties for that are less of a problem if you come off as sandbagging (even when you aren't). Managers who have no clue what the complexity of the problems trying to be solved not trusti
Estimates are like contracts (Score:2)
They have to cover every detail and be iron clad. But by the time you've come up with all of these specifications, most of the work is already done. So how do you recover the cost of an estimate if the other party says no?
I came from the web development world as a one-man department of a larger computer store. Quoting projects was an absolute nightmare and the specifications always changed even when they somehow manage to fit the definition of the words in our estimate.
Estimates are not the problem... it just sucks (Score:2)
Incorrect estimates are the problem. When non-experience people estimate, or when management force to under-estimate that is the real issue.
Estimates are necessary in software development, but of course that "estimating" sucks just like "deadlines", "taxes", "death", "breaking up with your girlfriend in person", "no wifi", "cancer", "divorce" and many other things.
I have a solution, why don't you go to your boss and
Estimates are easy. (Score:2)
Get rid of time estimates? (Score:5, Funny)
Peter Molyneux is that you?
Every project will be behind schedule (Score:2)
By definition. When you look at our current corporate culture, you know it has to be. For a simple reason: Companies bidding for jobs. And more often than not, the cheapest offer gets the deal.
Who is the cheapest? Usually the one that cut the most corners and underestimated his cost (i.e. time) to deliver the most.
The corrections and collaborations are the problem (Score:2)
Often the estimate includes an estimate of how many times the client will change something or whatever. And really THAT has to be made a part of the estimation system.
Estimate a time if they basically don't say anything more and provide required information promptly.
Then say ANY change to that what so ever is going to change the estimate in UNPREDICTABLE ways because you don't know what they're going to do.
Here someone is going to say that this is well understood and that developers deal with this all the t
Wow! (Score:3)
Engineers think project managers and deadlines are a waste of time and a pain in the ass, while project managers think they are essential. Now that's what I call news! Whodathunkit!
This is business. Management wants to quantify everything to manage resources, manage spend, control cost, maximize profit, etc. It makes perfect sense at the same time that it doesn't really jive with how engineering works a lot of time. One thing for developers to keep in mind, though, is that *doing* something is never as important as *telling people* about how you did it. Metrics mean way more to the people who sign your paycheck than the code you write does and you should design your metrics accordingly.
The other component is PMs themselves. How many really good PMs have you worked with in your life? Grand total of 1 for me. Most PMs are people who don't really understand technology and have created a whole system of super-important metadata to "add value" to the process. When it's done properly a PM can help a lot, but mostly its just blustering and wasting everyone's time. These people want to protect their jobs, and their jobs are defined by timelines and metrics.
Deadlines and Milestones instead of Estimates (Score:2)
I have found that the best middle ground is to work with the developers and project team to set deadlines and project milestones. While down to the tenth of an hour estimates are not necessary, there have to be goals to hit.
The best managers are going to let the developers provide their own estimates, and then calibrate the timelines accordingly. Some people are good at estimating time. Others are horrible at it. The project manager needs to know their team well enough to account for those factors.
The o
In related news (Score:2)
The city announced work on a new interchange involving the major arterial road running through the city, significant delays are expected while construction is underway.
When asked for a timeline on when the construction would be completed the lead engineer answered "Who knows? We generally underestimate these things by months or years so I might as well not bother."
Work is expected to commence sometime after they finish their current set of maintenance roadwork.
Good night, this was your 11 o'clock news at 11
Use it as a weapon (Score:2)
Use your estimate precision as a bargaining chip for things your need. Typically what impacts my estimate is requirement uncertainty. So give an estimate with an error +200/-10% error. If we could just finalize these particular requirements my estimate would improve. This way a manager will focus on getting the requirements fixed.
I was once given a "project" and I asked for the requirements document. They said there are no requirements so I happily said great I'm done designing!
Estimates (Score:2)
I've been programming for about 30 years and it took me at least 20 years to learn how to make good estimates. If you can't estimate how long something is going to take, then you probably haven't been doing it long enough. I don't care what your field is, or even if it's engineering or not.
Programming is just like any engineering endeavor. Be a professional and admit that not every job is a weekend hack. If you work somewhere where you have to lie about work taking less time than it takes, then quit that jo
Wasn't this the main point of "Agile"? (Score:2)
Find a compromise between predicting too much of the future and just managing a project by the seat of your pants; get into a rhythm where you check how good your estimations and learn to get better at them.
Of course you can't develop every project this way; I've used Agile and it's worked for me. I've used waterfall and it's worked for me too. You have to try to be sensible; you can't completely wall of other people's need to know when you'll accomplish certain things, nor can you build a solid plan based
Re:Estimates are for planning - not penalizing (Score:5, Insightful)
People do not predict well and it is not smart to use it against them.
You can improve. Most programming we do is boring, and similar to something we've done before. For things like that, you should be able to make your estimates reasonably accurate.
Sometimes projects have large unknowns, and it's impossible to know how long it will take. In that case you need to propagate that information up to the people who need it, either managers or salespeople or whatever.
Re: (Score:2)
You must be working in a shop that does the same thing over and over again, year after year.
No. I don't think many people work in that kind of shop.
No excuses. Don't use that as an excuse for bad estimates, it's something you can improve.
Re:Estimates are for planning - not penalizing (Score:4, Funny)
If you penalize me for going over time and budget, my next estimate is 30 eons and 200 trillion USD.
Watch me come in well inside the budget.
Re: (Score:2)
https://www.youtube.com/watch?... [youtube.com]
You didn't tell them how long it would REALLY take did you?
Re: (Score:2)
He was a wise man...
Re:Is this really a problem unique to devs?? (Score:4, Insightful)
You have three choices. You can give me enough time to finish the project correctly and receive an awesome result, but you will pay for that extra development time. You can give me too little time to deliver a quality result, but you will pay me for patches. You can give me too little development time and opt out of paying for updates, but you will pay for it when there are problems. Choose well.
There's a very old, very over-used saying about technology that people must be forgetting. It comes with three possible qualities: good, fast, and cheap. You may choose two.
Re:Is this really a problem unique to devs?? (Score:5, Insightful)
No, it's a very common problem in engineering in general, and not unique to software. But the reaction "let's eliminate estimates" appears to be.
As an engineering manager, I learned the hard way many times how estimates turn into deadlines. Your estimate is reported to the manager's manager and so on up the line, and someone uses it in planning their shit.
Your estimate, in which you did not build any schedule margin, then becomes an item in the critical path of someone else's plan, someone who didn't build in any margin either, or —worse— who was pressured to make a completely fictional "plan" which is really just a backwards-calculated paper justification to "prove" that a job could be completed in an impossibly short period of time by assuming nine women can make a baby in one month and things like shipping, reproduction, and quality assurance take place in zero time. This "plan" makes upper management happy. Temporarily.
You, leader of a small team that is working merrily away, accomplishing real work and solving the occasional unexpected problem (OEM pinouts were wrong, widget zeta delayed in shipping, amplifier stage behaving like oscillator, etc.), are asked for a status update. Because of your unexpected problems, your estimated completion date is now two weeks later than your previous estimate.
Now the middle manager, who knew he wasn't going to meet the "plan" he was forced to develop, now has someone to place the blame on. He knows he's going to be in the path of a metric fuckton of shit, but he's placed himself uphill of you.
It's clear even in TFS that the real problem isn't estimation, it's poor program management, lack of requirements management, and often also marketing-driven decision-making.
In other words, the same old shit.
Re: (Score:2)