Are Betas Taking On Lives of Their Own?

Ant writes "CNET News.com's Paul Festa thinks the final stage of software development, beta versions, are taking on a life of their own, as companies tinker endlessly with their products in public according to a recent article. Google is one of the companies that keep using "beta" term for years for its products."

In my mind:

[the_unknown_soldier]

In my mind firefox will always be better software until it renders slashdot correctly!

God I hate that

[Proc6]

ICQ was like that (I dont know if it still is, I haven't used it for years.). They'd just be in permanent beta. What a cop out. Grow a set and put a "release" stamp on it, bugs and all. Works for Microsoft.

Re:This makes the term meaningless.

[kg4czo]

It's because they want people to know it's quite possibly unstable and could blow up at anytime. Kinda like CowboyNeal. :)

Perpetual beta sucks

[DNS-and-BIND]

The old style of perpetual beta was lazy, perfectionistic, or excessively cautious programmers simply going on and on towards v1.0 and never reaching it. Not enough work was done - typical of the lazy programmer. It's never "good enough" to call v1.0, typical of the perfectionist view, despite the fact that the program has been out in general use for years.

Now, we have the new perpetual beta. Any company can, with a wave of the magic wand, make itself blameless when its software doesn't work. "But it's in beta!" they gleefully shout when you tell them about something that doesn't work correctly. "Refer it to our testing team, who will ignore your report."

Fear of commitment

[Anonymous Coward]

It's a simple case of fear of commitment (or litigation). If a product is beta, you don't have to really support it, and if it breaks it's really no big deal. It is, after all, a beta version.

Once you make the jump to release versions then suddenly everything has to run (nearly) perfectly and any issues need to be properly dealt with. Perpetual beta has it's advantages in that you simple don't deal with these problems. Or you don't deal with them formally, but you do fix them.

Google News is stuck in beta because Google can and will be sued the instant they start trying to make money (via text ads or something) off other sites headlines and stories.

Would you rather they release it as final?

[Halcyon-X]

I agree that selling software actually labeled as beta is a bad idea, but don't we already pay for software that require constant patching, such as the latest release versions of Windows, Microsoft Office, and nearly all of the latest games? Does release software even live up to the quality expected?

Google is a bad example

[Doorjam]

Most of googles products, except for searching of course, deserve nothing more than "Beta" status. They are like me, they start great, impress people, but never finish the dang project and fail to realize potential. Froogle or Google News anyone?

Google's different

[Anonymous Coward]

Google's kind of more following the open source philosophy of "if it's 1.0, that means something". Just open source projects use 0.* version numbers and Google says "beta". None of this Microsoft crap of releasing something half finished as 1.0 and tinkering and maybe by version 3.1 it will be usable. No, Google is going with the idea that if you say it's done, [i]it's actually done.[/i] But in the meantime that isn't any reason to stop you from using it.

Re:This makes the term meaningless.

[prockcore]

Not really. Google, for example, uses the term "beta" to mean "unsupported". gmail, maps, froogle, etc.. they're
all neat tools but Google hasn't really decided whether or not any of those projects merit the full force of
Google behind them, but it costs Google next to nothing to provide them on their site.

Apple does the same thing. Quicktime Broadcaster is beta.. hell, Apple has called it "a technology example" not
a finished product.

The question becomes, would you rather companies not release their little pet projects at all?

Re:This makes the term meaningless.

[trollzor]

It clarifies between "working" and "rock solid".

There is a reason NASA doesn't send the latest "working" laptops up to the space station, it's because you can only say something is "rock solid" after very extensive testing.

My gmail account isn't any better or worse that it would have been, it's just I know not to run anything mission critical off it.

More things should be in beta, there are too many things that claim to be rock solid that aren't.

At the same time, I don't condone the abuse of "beta" to avoid offering proper support... but we haven't seen widespread abuse (yet) whereas we have seen widespread abuse of people claiming things are solid and secure when they are not.

If you want to use debian unstable or fedora vs debian woody or red hat enterprise it's better to be making an informed decision than one based on marketing.

Does anyone know what beta means anymore?

[DingerX]

From the article:

Once considered the final stage of software development, beta versions ...

and

The beta version, named for the second letter of the Greek alphabet, typically refers to the second stage of software testing. Traditionally distributed to a limited group of testers, it follows the alpha version, which is tested in the lab.

What little training I had seemed to involve code existing in four stages of development, and beta was the second:

Alpha: the phase in the development cycle where code first comes into being. Subsystems are being built, and testing takes place on the that (subsystem) level.
Beta: the phase in the cycle where all subsystems are nominally in place, and testing occurs on the system level; not everything works, and features may be added, but we're looking at the whole code.
Final: features are locked down, the system is tested in the form it intends to be released. I believe, under the influence of someone like Microsoft, this is now referred to as "Release Candidate" stage.
Released: The software has been distributed.

On the other hand, this article implies another notion of software development stages, one that I see applied rather frequently:

Alpha: Testing done in house.
Beta: Product released to a group of testers who aren't in-house QA specialists.

So does someone have the answer? What the hell do these terms mean, and are they useful any more?

Lower expectations

[Duncan3]

If you lower expectations enough, you don't have to spend any money do to the last 10% of development that takes 90% of the time.

It's so very modern :)

Re:GMail

[4Lancer.net]

So, basically what you're saying is I'm getting my transmission fixed. But you know what? My car's still a piece of crap. Sure, they're always working on the search engine, but I doubt they're doing much of anything with the actual GMail service itself. Which is what we're concerned with here.

Re:In my mind:

[aussie_a]

If slashdot would conform to standards it would render correctly. When slashdot conforms to IE, non-IE browsers may have difficulty. It's that simple.

Having said that, I haven't ever had slashdot render incorrectly in firefox.

There's a reason Google News is in Beta

[d2_m_viant]

One of Google's long-running beta services is the Google News site. It's been in Beta for years now, but there's a reason for it. Google can't switch to a subscription service, or even try to put advertisements up on the site. If they do, they face the legal hurdles that come from making money off other news organizations' work. There exist very few bugs still left in the service, but yet it still remains in beta. On a side note -- ironically, Google has been sending out cease-and-desist letters [yahoo.com] to people creating RSS feeds that scour and present the results of Google News.

In Germany, Google has already been found guilty of copyright infringement [linksandlaw.com] as a result of providing other websites' images in their Google image search. The potential legal obstacles could be multiplied exponentially if the American news services got a whiff of Google making money as a result of providing their hard work.

Re:GMail

[KinkifyTheNation]

Because with a limited number of invites, they have an idea of what kind of disk space can potentially could be filled up. I doubt that there is a specific Gigabyte of storage set aside for each account that has been created, (there's no way EVERYONE is going to use up an ENTIRE gig) But with invites they can control new account creation and prevent people from registering a million accounts.

So Long As It's Not Being Sold...

[segmond]

what's the big deal?! I have used programs that were very functional that never reached version 1. But I was happy, so what it's version 0.8, it met all my needs! Better than the version 5.5 that doesn't!

The answer.

[mboverload]

When you are in beta you are invincible. When someone claims that it is beta, they can tell you to shove it because it's "BETA SOFTWARE!" Even if you complain some troll will also point out that it's "BETA SOFTWARE!".

Beta prevents the need for support but allows you to sell/release your product. This is a dream as it prevents those damn leeches called "consumers" from harassing them.

Re:Microsoft has done the opposite

[Stephen Samuel]

I really don't think it's a joke. I've been telling people that for years. Stuff that the Open Source world would consider "public Beta", Microsoft has been shipping on a regular basis. XP Service Pack 2 is an example of what happens when you do that -- lots of broken software due to changes in a so-called "production" product.

Re:GMail

[melekcrescent]

On the contrary, I think they have opted to test with a larger sample size. I don't exactly see how more users equates to no new work, I think that its much harder work to compensate for a drastically larger userbase and face all of their collective reactions to the service, not to mention how it will affect the software. (Although its likely their wealth is enough to make sure that serverload is virtually negligible) I am, admittedly, -totally- unqualified to guess at the google master plan, or even speculate on things like markets or program testing ;) It just that I do believe strongly that gmail is still in beta, and that they are still doing a considerable amount of work on it. The more users is the more community feedback, in either the direct form of service requests/suggestions or in public articles/forums. The more feedback, the more they know what they need and need not do to ensure the widest possible audience.

Re:GMail

[drsquare]

gmail gave me 50 invites. pretty much everyone i know of has (or doesn't want) a gmail account.

Hmmm, if everyone who wants an account has one, why do they have the 'invite' system? Why not just let everyone sign up and take it out of 'beta'? I personally can't get an account, and by the sounds of things I don't want one, I don't like the idea of some corporation spying on my entire e-mail history. Also it doesn't really seem to offer anything over the other webmail systems.

it's probably some smart marketing strategy. i just haven't been able to figure it out yet

Apparently if they put 'beta' in the corner, if absolves them from taking responsibility for it, like when it all goes wrong they can say 'well it was only in a testing stage so fuck you all'. Or perhaps they just don't have the balls to say 'this system works'.

Re:Does anyone know what beta means anymore?

[Eric MB Lard MD]

This beta thing is just one aspect of version numbering nonesense.

I think the best you can hope for with version numbering is that you will see some consistency between products produced by the same organisation. One company's beta is another's version 6.3.2.

Open source projects generally care less about pushing up the version number (marketing droids tend to affect version numbers more than product features).

Unfortunately PHB's haven't yet figured out that what matters is not the version number, but how well the thing actually performs. I was quite shocked recently when a PHB said, 'now that firefox has reached 1.0, we have to consider it'.

Beta and version numbers 1.0 tend to indicate that API's might not have stabilised yet, although this might matter a lot less if the product is open source + a bit of research (eg looking at the project roadmap) ought to remove most fears you might have about the stability of the API.

Re:Google's different

[gl4ss]

no, google is NOT different in this regard.

they're EVEN WORSE.

pretending that it's invite only for example - when in reality _everyone_ can have an invite(and they want everyone to have, viral marketing).

Re:The answer.

[at_slashdot]

Usually beta software is for free, so if someone tries "beta" software for free they shouldn't expect support -- they should submit bugs though.

Re:Would you rather they release it as final?

[fm6]

You are quite correct. The truth is that the Google's "beta" software is really production quality. I've used (hell, I've helped release) "final" versions that had more kinks than Google Maps.

"Beta" is just a word, and Google is using it to play the "Underpromise and Overdeliver" game.

Re:agreed

[Columcille]

In a natural state it is quite common for humans to act in territorial ways and kill other humans for very minor reasons. History is full of examples. And yet we have decided it is better for society that we act against our nature and work together, organizing despite our differences and territories. And yet when we have animals as pets it's okay to let them fight each other because it's their nature, and because it entertains us?

Re:God I hate that

[R2.0]

"It's irresponsible to force all your customers to be testers"

"Force?" Last time I checked Google's "beta" services were
1. Voluntary
2. Free

Of course, they could be like other software companies and release what is essentially beta software as a final and then pretend that bug fixes are "upgrades."

Not many, apparently :-(

[Anonymous Brave Guy]

It used to be much simpler than that, with just three pretty clear phases for testing and QA.

Obviously you start with your in-house testing, hopefully a constant background activity as you write new code. This is just routine development activity, and might include unit testing, regression testing, and more. A lot of this will be done locally on specific areas of the software.

As you reach the end of the new feature development for your coming release, you bring everything together to build a complete version of the whole product. This is your first alpha release, and you run all the system tests, integration tests, etc. If there are serious failures identified here, they get passed back to the relevant dev teams, and we go back to the previous step until everyone brings their revised contributions together for the next alpha.

As an aside, obviously for smaller projects you might be working with complete builds from almost day one. In this case calling something an "alpha release" is giving it rather more significance than it really has: you're just identifying a "mental marker" where you switch focus from localised to global testing.

When we have what we believe is a solid alpha build, we might want to ship it to a select group of customers and prospects. This is a beta release. It's not supposed to be a marketing exercise; it's an opportunity to get feature-complete code tested in a wider variety of realistic contexts than you can ever create in-house, so that you have a better chance of finding any subtle bugs before release: hardware incompatibilities, interoperability problems with data from other applications, etc. As with alphas, if serious flaws are identified, we go right back to the dev teams at step 1 to get them fixed, and then go through the process of localised testing, global testing, and potentially (but this used to be a rare event) running a second beta test.

Note that further formal alpha tests should never be necessary at this stage. Once a project has passed an alpha test, no code changes should ever come through in the future that don't. If they do, they weren't properly regression tested. Hence you re-run all the system and integration tests as part of the next beta/final release testing just to be safe, but you don't expect to have another alpha cycle.

When you've run a beta test and are happy that you've got enough bugs for your software to be a product your customers want to buy from you, you make the release. The problem today is that marketing droids have taken over the beta release process; it's no longer about improving code quality in partnership with carefully chosen customers/prospects for everyone's benefit, it's about promoting your software before it's ready to manage customer expectations and get community support built up so you don't have to support it yourself. The additional testing and consequent quality improvement is often negligible.

For those who missed it, this implies that you ought to be feature-complete before going into alpha, though you might change something significant if your system tests identify a weakness you hadn't noticed before (e.g., the combination of features written in practice doesn't meet a requirement completely). Anything going out as a beta should be both feature-complete and very well tested internally. A lot of places would assume you'd get some significant flaws identified during the beta programme -- that's why you run it, after all -- but certainly anything beyond the first beta release should be a "release candidate".

I have no idea where this notion of several beta builds then several release candidates came from. Nor do I know when it became a Good Idea(TM) to make major functionality changes after you've entered the beta phase; doing so pretty much negates the point of the previous alpha and beta tests. It's certainly not a good approach to QA, and perhaps it's also why so many companies now seem determined to run year-long, 17-release beta programmes instead of shipping a finished product, and then a new release with the extra features customers requested six months later.