Ask Slashdot: Why Is It So Hard To Make An Accurate Progress Bar? 736
hyperorbiter writes "How come after 25 years in the tech industry, someone hasn't worked out how to make accurate progress bars? This migration I'm doing has sat on 'less than a minute' for over 30 minutes. I'm not an engineer; is it really that hard?"
Uhmmm. (Score:5, Insightful)
because (Score:5, Insightful)
Things are asyncronous. You wait for things from disk, ram, user input, over the network etc. How long it will take is non-deterministic. So a task composed of a bunch of these little pieces will be non-deterministic too.
Various reasons (Score:5, Insightful)
Another reason is it is difficult to estimate time left. If you look at some old FTP programs, they'd estimate the rest of the download's time based on how fast the previous has taken. Future lag, fragmented files, etc aren't taken into consideration.
There's a bunch more reasons, but namely the progress bar's main purpose is to show you that the whole system isn't locked up, which they've been doing well for the past 30 years or so.
Re:because (Score:5, Insightful)
Yes (Score:4, Insightful)
It's very hard to predict how long something will take, particularly in relation to other things, if what you're writing is going to be on any number of platforms with different processors, storage, memory and network situations.
You can be reasonably accurate with it, far more than my favorite 99% in 1 second, the last 1% in one hour scenario. There are cleverer and cleverer ways of making it ever more precise, but those methods usually involve spending time on getting it right, and not many people do it.
Can't Go Backwards (Score:5, Insightful)
Comment removed (Score:5, Insightful)
consider a rocket launch countdown (Score:5, Insightful)
Re:because (Score:5, Insightful)
Personally, I like the "list of what I am going to do + check them off as they are done" approach, if the "progressing task" is of varying descriptions.
That said, for some tasks, you have to consider the "Heisenberg effect" a bit, depending on the work being done - do you want your progress bar pestering the "worker" for progress reports all the time, or just doing its job.
Re:because (Score:5, Insightful)
The list/status bar solution is nice for another reason: if your program does hang your users know where in the process it happened. You get more useful feedback from the users, sometimes they might be able to troubleshoot it themselves (oh it is dying when connecting to my fileserver, perhaps I"m not connected to the network). Heck it is useful for course grained optimization you see the logical steps that take the most time and can drill down into speeding them up as much as possible.
Re:Can't Go Backwards (Score:5, Insightful)
What? Why the fuck would a progress bar go backwards? A progress bar shouldn't be pegged to the fraction of time of the job done. It should be pegged to the actual progress. If progress starts going slower, then the progress bar should start moving slower.
Re:Various reasons (Score:5, Insightful)
The spinning asterisk or swirlie just tells you that the thread updating the asterisk or swirlie hasn't frozen -- it often doesn't tell you anything about the underlying activity the user is waiting for.
Re: Can't Go Backwards (Score:5, Insightful)
Realistically a large IO bound task is something you hand off to low-end process to get it done as fast as ppssible. Would you over-ride the O/S to transfer that 10gigs in 100 meg blocks and report each bit back to your high level UI so that th progress bar is accurate?
We will have an accurate progress bar when ... (Score:4, Insightful)
... when we solve the halting problem. I'm not entirely joking. The main problem with progress bars is that, quite often, it is not possible to accurately estimate how much time is needed to complete a problem (i.e. for the program to halt).
The 99%... (Score:5, Insightful)
The public opinion of the Progress Bar would be considerably more favorable if programmers would simply treat 100% as if it were 75%.
In other words, do all the stuff you have to do, measuring progress and whatnot, but when you're actually at 80%, report yourself at 60%. Likewise, when you're at 95%, say you're at 70%.
Then, only when you really are completely finished, you jump from 75% to 100% in under a second.
Complaints gone.
You know what I'd like to see? (Score:5, Insightful)
I don't want to finish the sub-process I'm currently doing (which has probably stalled)... just FREAKING STOP.
If you (programmer) want to close connections, or save the changes to the disk, do it in the background. Making me sit there for another 10 minutes while you're "cancelling..." is not helpful. I will force close your program. Failing that I will hard-reset the computer. Seriously.
Re:Crappy software (Score:5, Insightful)
Give this man some upward moderation.
The problem is that the question is wrong. It's trivial to make a progress bar...just sum up all the things you have to do, and move the bar each time a "thing" is done, rounding to the nearest pixel. It doesn't matter if the "thing" is a byte to copy, a file to install, or any generic task. As long as you can add one to a counter each time you have done another "thing", you can then display it graphically.
The actual complaint is about displaying accurate time remaining to complete the task, which really has nothing to do with the display of the progress bar. Instead, it involves guessing about how long each remaining "thing" will take to complete, and then displaying that sum of those times. This is hard because no matter how accurate the data used to make the guess, something outside the control of the program can disrupt the processing.
It's a Prediction of the Future (Score:4, Insightful)
"How come after 25 years in the tech industry, someone hasn't worked out how to make accurate progress bars? This migration I'm doing has sat on 'less than a minute' for over 30 minutes. I'm not an engineer; is it really that hard?"
Yes, because all progress bars are inherently a prediction of things that will happen in the future. If there is any error condition, unusually large blob of data or weirdly structured hard drive to read from, varying bandwidth bottleneck, fritzy peripheral not responding as expected, etc., etc. times a million, then the unusual event will make the prior prediction incorrect and look silly in retrospect. As long as there is any "if-then" clause or error handling in the branches in the system, then the unexpected can happen and make the prediction (progress bar) invalid.
It's analogous to weather prediction. It can't be perfect, it's an extrapolation, but people will always complain about it.
Re:Can't Go Backwards (Score:5, Insightful)
To put it another way, it is impossible to make an accurate progress bar because it is impossible to predict the future. That's all.
You can make assumptions, extrapolation, use past performance,etc but you are never going to get it accurate.
This is like getting pissed at your GPS because it told you you will reach in 22 mins when it actually took you 35.
It is either a dumbass expectation from the user or a dumbass design decision. You might have as well used an hourglass so the user could spend their time shaking it trying to get the sand to fall faster.
Re:Can't Go Backwards (Score:5, Insightful)
Yeah, but then users complain because the bar is weirdly scaled, like 95% taking up a second and the remaining 5% an hour. You can try to profile it, but the actual time is determined by the local hardware after all: Processor speed, number of cores (which affects only some operations), network speed, free memory - there's no way to predict how much of the time will be spend on each task.
(And if the progress bar isn't an indicator of execution time at all, then the arbitrary completion percentages assigned by the developer mean little. In that case, you may as well go back to hourglasses and spinning beachballs...)
Re:Can't Go Backwards (Score:5, Insightful)
Suppose a bar can never back up. Then its speed may vary enormously, making the visual how-done-am-I representation valueless.
It is HARD.
Define "fraction of job time done" numerically. Suppose you have to download 100 packages and then install them. How many % of the total should the download represent? Does the guy have cable or dialup? And how many % does the install represent? Does he have a slow USB hard drive or a ridiculously fast SSD? What if the first file downloads at 1 MBps and the 17th one at 139 Bps? I've seen that happen (varying server load, per-file mirrors). Which installs faster, a package containing 250 files totaling 10 MB, or a package containing 10 files totaling 100 MB - hint; it depends.
Generally, might as well show a rotating propeller or a cylon helmet moving dot, neither of which has a beginning or an end. Just an indication that something is going on, and the system doesn't look frozen. Maybe a couple of numbers under it showing # files done out of # total files.
Simple solution (Score:4, Insightful)
Show progress of the work, not the time. You never know what other things the machine is doing, and is better to give no estimate than a bad one.
And don't suffice with a bar, also provide information on what the process is doing.
Re:Can't Go Backwards (Score:5, Insightful)
What I like is the Step X of Y types of progress indicators. Sure step 5 of 12 might take way longer then the rest but at least it is honest. It also helps to tell the user what it is currently doing. You know that downloading drivers may take a while when the internet is slow (or down).
Re:Can't Go Backwards (Score:4, Insightful)
No, no it isn't. A second progress bar flickering up and down all the time tells the user nothing. Just put the one general progress bar there, and you're done.
Re:Crappy software (Score:5, Insightful)
Except even the number of tasks is often variable over the life of the task.
Take for example loading a web page. It starts out as 1 task: Get a page from the server. Once you've done that, how many more requests will that first request generate? Impossible to tell. It could be none. It could be hundreds, and some of those can generate their own requests. (etc, etc.)
The answer is this: Some feedback, no matter how incorrect, is better than no feedback at all.
Re:Can't Go Backwards (Score:2, Insightful)
Define "fraction of job time done" numerically. Suppose you have to download 100 packages and then install them. How many % of the total should the download represent? Does the guy have cable or dialup? And how many % does the install represent? Does he have a slow USB hard drive or a ridiculously fast SSD? What if the first file downloads at 1 MBps and the 17th one at 139 Bps? I've seen that happen (varying server load, per-file mirrors). Which installs faster, a package containing 250 files totaling 10 MB, or a package containing 10 files totaling 100 MB - hint; it depends.
I think two things are being conflated:
* amount of work done, and left to do
* time to do that work
The "percentage done" (and graphical bar) should represent the amount of work, as measured by the number of bits copied (or installed). The estimated time is calculated (in a separate thread?) depending on the rate at which the work is progressing.
Once bits are copied, installed, or downloaded successfully move the bar forward proportionally to the total amount you're going to move. As each package/file is done, check the start and stop timestamps and do a calculation.
outline format (Score:4, Insightful)
Re:Can't Go Backwards (Score:4, Insightful)
I hate those, they're totally pointless. "I'm possibly doing something, I don't know how much of it I've done, and I don't know when it will end, and I won't show you when I started".
To be honest, progress bars shouldn't be used for indeterminate timescales. If you can guess a time remaining, then say that, as this can be changed, and it is visually more useful than a context-less progress bar that can go in reverse.
Well, they are not pointless. They let the user know "Something is happening, but I have no idea when it's going to be done.". If you don't do something like that, then the user thinks your app has locked up and they kill the application. It's called "User feedback".
Now on to the original theme of the story....
Progress bars shouldn't be based on time, they should be based on quantity of work. Typically when they do the marquee style of progress bar (where it just show animations, but no actual progress), it means the quantity is unknown. This happens in a lot of situations where the total number of steps to be processed is unknown due to the data, or the amount of time it would take to calculate the total number of items would take too long.
Progress bars should also be a different subject then Time Estimates as they are not the same. I have 100 things to process, so my progress is between 1 and 100. How long that takes depends entirely upon what I am processing. If I'm processing Invoices say, then how long it takes depends a lot on what is involved with the invoice (line items, calculations, other look ups).
Just because it took 10 seconds to process the last one doesn't mean it's going to take 10 seconds to process the next one. In order to determine how much time is left, what most people do is take the quantity left and multiply that by the average time it's taken so far. You could apply smoothing to the average (so it doesn't jump around a lot), but other then that, how else would you estimate the time? If the amount of time it takes to process items is fairly even, your estimate will be very accurate, if they are vastly different between items, then your estimate is going to jump all over the place and be wildly inaccurate.