Big Challenges for Vista Bug Hunters 213
The New York Times is reporting on the final rush to bug fix Windows Vista. Even with massive numbers of testers and five years of work behind them, the folks in Redmond are pushing it to the wire in order to make sure it releases soon. From the article: "Vista has also been tested extensively. More than half a million computer users have installed Vista test software, and 450,000 of the systems have sent crash data back to Microsoft. Such data supplements the company's own testing in a center for Office referred to as the Big Button Room, for the array of switches, lights and other apparatus that fill the space. (A similar Vista room has a less interesting name -- Windows Test Technologies.) This is where special software automatically exercises programs rapidly while looking for errors."
special software (Score:5, Funny)
and this software, folks, goes by the name "internet explorer".
Re: (Score:2)
Re: (Score:2)
What platform did they develop this 'special software' and why don't they rewrite Vista in it. What errors would you get if you ran it on itself.
Re: (Score:2)
Re: (Score:2, Flamebait)
Re:special software (Score:5, Informative)
You'll see this kind of attitude in every bigger software company. I've had personal experiences like this in Adobe and Macromedia with their flagship products. Features are dropped, specs constantly changing and inconsistent between teams.
In some cases, you can spot the same feature implemented twice in source, with different interfaces, in different locations, and code linking randomly to one or the other, or even both (imagine updating this).
The bugs to be fixed are selected first for how obvious they are (likely to occur) and not how critical they are. This is why it's common that bugs that can totally wreck operation and lead to data loss may be left, if the occurence is rare or unlikely.
Everybody is in stress and the main goal is that you get the reviewed bug off your shoulders: if it's mildly reminiscent to something else, it's marked duplicate. If you can't reproduce it quickly, it's marked as fixed or not reproducible. Tricky bugs are marked "fact of life" or "deffered".
Successful companies and their products grow, but the way the products and resources are managed does not scale. Instead, programmers are expected to churn a major release every X months, screw everything else, and keep the cash flowing, the investors happy.
With Windows, we have a successful product that supports a huge ecosystem of applications (including legacy support), localization, usage cases etc. It's natural that in time, updates will become more rare, and will be much slower and more expensive to produce.
The trend of software-as-a-service is not coincidental with this situation. In 5 to 10 years the base software we use might be so complex and tough to work on, that the only way it can be sustained is by small, regular payments, and the updates will be small, incremental, security/performance oriented. No more big releases, no more rushes to fix bugs in the last moment.
This is the way evolution works. The other route is, of course, revolution...
Re: (Score:2)
I think the interesting part will come when we see large, old code bases that started in the software-as-service world. A few years back I s
Re: (Score:2)
Re: (Score:2)
1) Most open source apps do not suffer feature bloat. They aren't trying to entice new customers so they usually just do what they are supposed to do.
2) Updates are often done just to improve or optimize. Companies who are paying their programmers to churn out sellable products often can't afford to optimize every little bit of the app. Volunteers who just want to software to be its best can.
3) A
Re: (Score:2)
Time (Score:4, Insightful)
my $0.02
Comment removed (Score:5, Informative)
Re: (Score:2, Insightful)
Though to be fair people will have a go at MS if it's late OR if it has bugs, so they can't win. That said, either way people will be forced to use it.
Re: (Score:3, Insightful)
I don't know. Few people upgrade their version of Windows
unless they're getting a new machine. However, lots of people
are discovering that a 3 year old computer is perfectly capable
of doing what they need it to do and so doesn't really
need to be replaced unless the hardware is failing.
Even more interesting, for the first time ever, Apple's
offerings are starting to be percieved as a real alternative
that is, arguably, comparably priced.
It will be interesting to see w
Re: (Score:2)
Re: (Score:2)
Bill, is that you?
Re: (Score:3, Insightful)
At the time it's released, Mac will have another OS out but that's beside the point. That only matters to people that are `on the fence` OS wise and not a significant number. In the halls of the OS engineers, it matters as it proves what insiders at MS have said that "Microsoft isn't able to ship products anymore [blogspot.com]."
When SP1 is released, there will be hoopla and hype that Vista will have even more features, be more stable, and even more secure.
Re: (Score:2, Insightful)
All a bit academic really, the adverage person will get it good or not and slashdoters will complain it crashes and is insecure. Its like the circle of life.
Re: (Score:2)
Re: (Score:2)
There was never a time when Windows ME was the only version of Windows on sale.
Re: (Score:2, Informative)
Huh.. (Score:5, Funny)
Color me unimpressed.
Re:Huh.. (Score:5, Insightful)
If I had been running a beta version of the operating system I would have gone ahead and sent, on the grounds that it might have been a bad interaction between app and OS. In the event I said no.
You need to know more about what is triggering the crash reporter.
Re:Huh.. (Score:5, Funny)
Re: (Score:2)
Wait a minute, that mainboard is still in my_MQ\ZY61?'DK7N.DP#+U^4:.$NO CARRIER
Re: (Score:2)
"450k ought to be enough for anybody." - Bill Gates [wikiquote.org]
Re: (Score:2)
Some of them were probably trying to get it to crash, and that can distort the numbers.
I know a lot of people on here have Windows XP systems that have run forever as long as updates are made, but what's the average uptime for a Windows system? It could be that almost all Windows machines crash at some time during the beta period timeframe, in which case the 450,000 crashes would be expected.
I might expect most people to send the data to MS because you as beta tester wan
Re: (Score:2)
Uptime on the Vista beta boxes is likely to be low - on a test system, you're going to have frequent reboots, I would suspect. Besides, this is the home-user, desktop version - Longhorn Server is coming out later.
My Vista install has crashed a few times - 90% due to DivX not liking Vista, a few due to Media Center not liking DivX, and the rest video drivers crashing. All but twice Vista RESTARTED the driver and booted my game again, meaning I only saw two bluescreens.
All the bugs, minus the DivX relat
Re: (Score:2)
At any rate, my personal experience with Vista is that it's as solid as XP once you run RC1 or later. Apps do crash (usually older ones), but the OS itself is very stable.
And before you make a
Re: (Score:2)
Re: (Score:2)
why not sometimes you get feedback which identifies the problem and a solution. You don't have to like microsoft to send in a crash report. It is in your own interest as well as microsofts
got to admit i thought it was a one way street, but i know my brother got some feedback and identified a problem.
Re: (Score:2)
Plus, if I were running a beta, then I would definitely send in crash reports; that's what betas are for, after all.
Re: (Score:2)
Code signing certificates are expensive (Score:2)
Doesn't this require the third-party application's developer to have signed the application's code using a VeriSign code signing certificate? At 499 USD per year plus wha
I wouldn't want to be the guy (Score:4, Funny)
That is the kind of information that can get people fired...
Re: (Score:3, Interesting)
The real world is probably worse than the statistics suggest.
I tried to install Vista on three PCs, all of which passed Microsoft's Ready for Vista testing tool, but all three failed before they were able to sent any crash data back to Microsoft. Two installs hung due because Vista didn't like my SATA / motherboard combination. The other got its knickers twisted over my partitioning scheme. And that was before I got a chance to find out if any of my other hardware (printer, scanner, TV card, firewire,
Does it mean anything? (Score:2)
Statistics! (Score:5, Insightful)
Re:Statistics! (Score:5, Funny)
Now the world will be destroyed, and we'll find out it was really Steve Ballmer's plan all along...then he'll throw a chair at something.
Begun the dark times have.
Re: (Score:2)
I think crash data can also apply to applications that crash. Still, if only 10% has a system crash, that's a lot!
Re: (Score:2)
Perhaps it was because 1 in every 10 couldn't get it to install/boot/work
Re: (Score:2)
Yes, in fact, I would. If I were beta-testing an OS, I would make sure I did that on an isolated machine, until I was certain that it was ready for being hooked into a network. And an isolated machine won't be able to send bug reports.
--
*Art
Re: (Score:2)
Or it could be that 1 out of every Vista system crashed 9 times.... and the remaining systems went into BSOD before the reports got dispatched
Did you notice ALL the chairs AND Tables are taken? The developers seem to have learnt their lessons!
Re: (Score:2)
Re:Statistics! (Score:5, Insightful)
Re: (Score:2)
Re: (Score:2, Insightful)
The only reason people are even considering Vista (given this level of failure) is that we have become so used to PCs crashing that we're blind to it.
Imagine if your TV switched itself off and took a minute or two to come back as often as your computer crashes. You'd send it back and demand a proper one - that worked.
I'm aware of all the arguments about 3rd party software/drivers etc. being the real cause. That's as maybe, but if the Vista architecture
Re:Statistics! (Score:4, Insightful)
The TV will do one thing, and one thing only. That's displaying an analog signal as moving images and sound. That's all. The day that's all Windows will ever have to do, that's the day you can demand a refund.
Re: (Score:2)
It also decrypts the signal from the receiver, carefully avoiding the dreaded analog hole. A modern TV uses more processing power to restrict my freedom than my PC did ten years ago to enhance it.
Come to think of it, that'd be an interesting prospect for a sci-fi story - a civilization where more c
Re: (Score:2)
so it's like my Tivo (Score:2)
p.s. my TIVO, a LINUX device, freezes now 2 or 3 times a month.. I have to switch it off (unplug it actually) and gosh yes, wait a minute (*5 or 6 actually) to resume my normal broadcast day.
Re: (Score:3, Informative)
Ignorant (Score:3, Informative)
It's more akin to you turning on your TV and finding out that your channels suck.
Re: (Score:2, Offtopic)
1. Long Term Power Failure.
2. Hardware failure.
That is all. Thank you come again.
I do. (Score:2)
Re: (Score:2)
You can do the same thing in Windows, but you need a third party application to do. http://www.sysinternals.com/ [sysinternals.com]
450,000 of 500,000 people report crashes ? (Score:5, Funny)
Just wanted to thank god for linux.
Re: (Score:2)
When you look at the possibilities, it's almost certain that EVERY user experienced some kind of crash, however minor. Whether that reflects on the state of Vista, or the state of modern operating systems in general, I don't know.
Re: (Score:2)
This surely is the solution to sift through it all (Score:4, Funny)
Surely Microsoft could use this to sift through such an vast quantity of code: http://www.google.com/codesearch [google.com]
Just please don't start hurling chairs at my Karma!
450,000? (Score:2)
What are the chances? Damn.
More info? (Score:2)
I for one would love to know more about the tools they use for automated testing.
In my company, we have a build & testing server running compiler and NUnit [nunit.org] tests for all data-layer tests (complete tests like "load all of everything" and more specific tests like "authorise user with known bad credentials - expect login-failure") alongside NUnitForms [sourceforge.net] tests for the application-layer (random, frantic clic
Phase 2 (Score:2)
Maybe they should try hooking WGA up to this thing.
Why try, and fail to, reinvent the wheel... (Score:3, Interesting)
Microsoft wanted a more reliable machine, improved memory management, a better filesystem, etc... Instead of throwing resources at doing these things from scratch, why didn't they just
From every point of view it seems to make more sense. They spend less money, get a more reliable product that can run very nicely on existing hardware, get some good press for a change, and benefit from the work of unpaid open-source programmers all over the world. But it isn't their way.
Re: (Score:2)
Microsoft wanted a more reliable machine, improved memory management, a better filesystem, etc... Instead of throwing resources at doing these things from scratch, why didn't they just
Because there is zero benefit to them in doing so.
Re: (Score:2)
Uh, there's a substantial benefit to them doing so -- a huge cost savings in their Windows development costs.
BSD maybe. IT is the OS of choifce ot the leeches. (Score:2)
Re:Why try, and fail to, reinvent the wheel... (Score:4, Insightful)
As for Apple, I wish that they had succeeded with Copeland, so there would still be at least one mainstream OS that wasn't Unix or NT based. Apple chose NexT (the BSD version (there was also an NT version)) out of desperation, not because they so loved BSD or Unix.
Re: (Score:2)
There is something to be said f
Re: (Score:2)
Apple's other choice before NeXT was BeOS, also based on UNIX.
BeOS was not UNIX-based. Heck, BeOS wasn't even multiuser.
Because Apple was desperate for a new OS doesn't mean the fact that they went with a UNIX-based solution was a mistake or something they settled on with reservations. On the contrary, I'm surprised anyone is arguing such given all the open source software that now happily runs on OS X and how open the system really is.
Because releasing Yet Another Unix is a) not particularly interest
Re: (Score:2)
It was most definitely based on and inspired by UNIX.
It's better to ship something that already works then something that some techies might find "interesting."
Steve Jobs was always progressive when it came to UNIX back in the 80s.
Again, this statement isn't backed up with anything.
Re: (Score:2)
It was most definitely based on and inspired by UNIX.
Rrrrright. So exactly which part of a microkernel-esque, pervasively multithreaded, single-user OS with a C++ API and GUI interface do you think was "based on and inspired by UNIX" ?
Indeed, apart from a certain level of POSIX compliance, what _similarities_ are there between UNIXes and BeOS ?
OS X supports things like SMP far better than NT does.
Rubbish. OS X's SMP support is basic, at best. NT has been running on - and being tuned for - multiproc
Re: (Score:2)
I don't think so. It is probably possible (though not easy) to get Wine working with any particular application you choose, though it may take more or less time depending on the development team you have. But to have something that works with *every* application? There are many millions of Windows apps out there, a large percentage of wh
Re: (Score:2)
So simple maths (Score:2, Funny)
What about compatibility? (Score:3, Interesting)
Something that I haven't heard much recently is about Vista compatibility. MS has said before that it will be compatible and for most software and hardware, it was true in previous versions. But there were enough exceptions. ME was supposed to be backwards compatible. But many specialized drivers had to be written for it. XP definitely required some driver updates. Since Vista is a architectural change, so one would except some compatibility issues especially when DRM and enhanced security is thrown into the mix.
Technically would MS classify incompatibility as not a a bug, especially if is does not cause a crash?
high crash rate (Score:3, Insightful)
High failure rate (Score:2, Insightful)
So the liklihood of a crash is near 100% ?
Big bang testing (Score:3, Interesting)
(Yes I am aware I used singular, it was a joke, OK?)
Has anyone considered... (Score:2)
At least they're trying to find their bugs, at least they are running a widespread beta.
Alternate wording (Score:2)
or, "Half a million installs of Vista, and almost half a million have crashed"
Haste makes waste. (Score:3, Insightful)
Bugs are the consequence of rushing the job in the first place. (Taking time, is of course, necessary but not sufficient).
If Microsoft knows a way to "rush" bug fixes without compromising quality, they would have been able to "rush" their development without creating the bugs in the first place.
Re: (Score:3, Funny)
Re: (Score:2, Informative)
Yes, you can use hardware to track down bugs... (Score:5, Interesting)
With XP, almost all of the crashes are due to bad (usually non-MS) device drivers. If you run a system with pure MS drivers and quality hardware you'll never see a BSOD. If you run the usual business suite of software (Office, Outlook, IE) you probably never see an application crash.
It's the crappy hardware and badly written drivers that cause the crashes. That's the difference with Apple.... since they control the hardware there's less crashes due to bad hardware and there are fewer third-party drivers for Mac hardware. The software is probably the same quality.
Re: (Score:3, Informative)
Huh ? This is called a serial console terminal, and I wouldn't call a terminal a 'hardware tracer'. Other OS just use a serial console to debug device drivers and scheduler, with most debugging done in software.
With XP, almost all of the crashes are due to bad (usually non-MS) device drivers. If you run a system with pure MS dr
Re: (Score:2)
Re:Yes, you can use hardware to track down bugs... (Score:4, Insightful)
Does Microsoft make any video drivers that can even run the Aero GUI? Or, by "quality hardware" do you mean really old generic hardware supported by MS's generic drivers? I've never seen any MS drivers for my scanner, printer, webcam... basically nothing in my "quality" system other than the MS keyboard. So, I don't know if what you propose is even possible.
While there are lots of entries in the MS KB that are totally due to 3rd party drivers, there are many that are not. So, even if what you propose is possible, it's unlikely to be correct.
Okay, you're high, aren't you?
Exploits come out every other day to crash IE And, there are hundreds if not thousands of MS KB articles regarding Outlook crashes. Office? I've seen it crash many times due to internal bugs. And, when it crashes, you can't shut down Windows because it tells you that you must exit all Office apps first - thanks to MS's wonderful OS integration.
Re: (Score:3, Informative)
I just had XP not merely BSOD, but destroy itself so it wouldn't boot any more, after I installed a set of Microsoft security updates.
It was using no 3rd party drivers, and it wasn't a hardware problem because I was running it under VMware and other Windows images continued to run just fine.
Re: (Score:2)
So basically, XP is perfectly reliable in some theoretical sense, but not if you actually use it?
Re: (Score:2)
One thing that Apple did when they changed to OS X was to revise the API. Overall they reduced the number of API calls from like 9,000 to 6,000. For legacy systems, Apple created an emulation environment. Apple did sacrific
Re: (Score:2)
The Win32 API unfortunately has only gotten larger with each new version as MS had decided to make backwards compatibility more of a priority than other goals. I think there are some 40,000 documented API calls. Then there are the undocumented ones. At some point MS has to clean up their API and deprecate some of them. At this point it will be very painful and legacy systems will not work. I'm not sure if Vista is headed in that direction.
What do you think .Net is ?
Re: (Score:2)
Re: (Score:2)
The efficiency of a group of people is not the sum of intelligence, it's the sum of stupidity.
Re: (Score:2)
We don't know there was "only one" lady in the room. We can only be sure that (a) there is at most one lady in the room and (b) that there is exactly one woman in the room. This is assuming the photographer is male.
Vista SP1 - should be stable by then (Score:2)
It's funny, though, that neither you nor I nor a lot of other
Re: (Score:2)
It's funny, though, that neither you nor I nor a lot of other /.'ers would even touch a brand new release of Windows. I just see it as moving from 500,000 beta-testers to 50,000,000 beta-testers.
On the contrary, I've used every version of Windows NT (except Vista[0]) as my full-time desktop OS since late in its beta cycle.
NT4's beta 2 was the OS that got me off OS/2, with better performance, stability and software support.
Windows 2000 never gave me any trouble from the RCs onwards.
XP was fine from the
Re: (Score:2)
Vista BitTorrent (Score:2)
Re: (Score:2)
You accuse MS of this but go on to praise Apple. Are you forgetting that Apple released what was essentially betaware as the RTM build of OSX 10.0?
And how about a more recent example: iTunes 7.0 was so buggy when it was released that Apple had to release iTunes 7.0.1 within a week. And it's still buggy as hell.