Please Stop Shipping Beta Software

I’d like to ask all of the technology companies out there to stop shipping beta software. You’re lowering the benchmark for software. You’re giving people something that doesn’t work quite right for free. You’re delivering two sets of expectations to your customers. “Beta” is starting to mean “I’m not smart enough to figure out a fix to this bug”.

A beta test of software used to mean something. Tests were put in place and the testers (or users) had an expectation to test for bugs and report those to the programmer. Beta software has been used been many companies as a way to “ship” but give them an out when things go wrong. Instead of saying “we make shitty software” they say “it’s a beta!”

As of this morning, 4 of the top 25 listings of Mac OS X software on VersionTracker were labeled “beta”.

If it’s good enough to send to the general public for “beta testing” then ship it.

11 thoughts on “Please Stop Shipping Beta Software

  1. I think you’re mindset on software development is indicative of another era. By labeling software as “beta”, you’re announcing that the software is not yet complete, not that it has bugs. Software will always have bugs. Getting an app out in the public forum so it can be used in a real-world setting allows it to evolve. You’re constantly getting feedback on what the users love and hate.

  2. If you release as “beta” you should release to a limited amount of people not throw it out there for everyone.

  3. Yes, that is a problem from Companies and Users… Companies like Beta products, because can allways say “Its beta” when data is lost, or problems apper. Users are also part of the problem, because they don’t seem to distingue betas from final releases, until they find a crash situation.

    From my point of view, Google is the main “contributer” to this idea… They offered mainstream products for long time in beta state.

  4. I am quite offended that you deleted my comment. It was not spiteful or rude. If you’re going to make your opinions public, you should not be afraid to have people disagree with you. I’m just wasting my breath, you’ll probably delete this, too.

  5. Beta is an important step for software. It tells the user that the software may or may not be production ready, it’s still being tested.

    No matter how good you are or how good your tests are, there is still a large potential for problems to arise when others start to use it.

    This is the purpose of Beta.

  6. JMC: I just got a chance to approve comments. It looks like from the time you left the first one and the time you left the second one was about 8 hours. Any chance you checked the site from a different computer? It would show up on the computer that submitted it.

    As to the topic of site comments in general:

    You’re entitled to an opinion but no one has any rights here but me. If I delete your comment then so be it. Not all letters to a newspaper are published but I’d be they are all viewed. Same here.

  7. JMC: Your comments show your lack of experience here. There was a time when software didn’t have bugs at the level that we see today. A bug that an average user saw was borderline shameful. I’m not naive enough to believe that today’s complex software won’t have bugs, but the release of beta software by anyone lowers the expectations of customers and therefore devalues good software programming in general. By repeating this here, I realize now that you may have missed the point of the post. How could I rewrite that to help you understand it better?

  8. Dave: I wasn’t saying “don’t do beta testing”. I am saying “do meaningful beta testing”. Shipping an app to the general public that’s buggy without guidelines for testing is a poor way to do software development. What have you done with your projects?

  9. Hi Steve,

    I agree that it’s annoying to download software and find it doesn’t work well, or at all. However, first impressions are lasting and the project will likely suffer, so releasing early is often a lesson learned. I tend to release quietly until I am satisfied with my testing. Then before going to a stable/production mode, I release large.

    I have released several open-source and commercial products, and in both cases even after thousands of downloads, you might not get any feedback. My experience is that feedback usually only comes when it’s
    negative, and usually in the form of a bug report. Because of the large number who are casually looking, or looking for the best fit, you can end up with a feedback to download ratio of 1 in several thousand.

    This makes releasing to a larger audience more palatable, as the more people looking, the more feedback.

    In the case of open source, announcing early also increases your chances of collaboration.

    But, in the end you are right! I think the actual problem is, when releasing, many projects should be labeled as alpha instead of beta.

  10. @Steve Kirks

    First of all, I apologize for my knee jerk comment regarding the “deletion”. This was, in fact, an issue on my end from using multiple browsers.

    As to my “lack of experience”, you are entitled to your (unfounded) opinion, yet I still think my point holds water.

    “There was a time when software didn’t have bugs at the level that we see today”

    I don’t see how you can say that with a straight face. The amount and severity of bugs MAY be higher than they were in this “time” that you speak of, but they are proportionate to the size and complexity of the software. It’s ridiculous to expect that something on the level of Gmail will be as bug-free as something like pine (you do remember pine, don’t you?).

    “release of beta software by anyone lowers the expectations of customers and therefore devalues good software programming in general”

    Release of beta software is just what it is… A BETA RELEASE. The only expectations attached to it should be that it’s a work in progress. If there are any expectations being lowered, it should be based on where the software is headed, not where it is.

  11. @Robert

    “If you release as “beta” you should release to a limited amount of people not throw it out there for everyone.”

    Says who? I find that having the most feedback possible accelerates the development process considerably. Stating clearly the purpose of a beta release is often warranted, especially considering the responses to this post.

Comments are closed.