OneStopTesting - Quality Testing Jobs, eBooks, Articles, FAQs, Training Institutes, Testing Software, Testing downloads, testing news, testing tools, learn testing, manual testing, automated testing, load runner, winrunner, test director, silk test, STLC

Forum| Contact Us| Testimonials| Sitemap| Employee Referrals| News| Articles| Feedback| Enquiry
Testing Resources
  • Testing Articles
  • Testing Books
  • Testing Certification
  • Testing FAQs
  • Testing Downloads
  • Testing Interview Questions
  • Career In Software Testing
  • Testing Jobs
  • Testing Job Consultants
  • Testing News
  • Testing Training Institutes
  • Introduction
  • Designing Test Cases
  • Developing Test Cases
  • Writing Test Cases
  • Test Case Templates
  • Purpose
  • What Is a Good Test Case?
  • Test Specifications
  • UML
  • Scenario Testing
  • Test Script
  • Test Summary Report
  • Test Data
  • Defect Tracking
    Software testing
  • Testing Forum
  • Introduction
  • Testing Start Process
  • Testing Stop Process
  • Testing Strategy
  • Risk Analysis
  • Software Listings
  • Test Metrics
  • Release Life Cycle
  • Interoperability Testing
  • Extreme Programming
  • Cyclomatic Complexity
  • Equivalence Partitioning
  • Error Guessing
  • Boundary Value Analysis
  • Traceability Matrix
    SDLC Models
  • Introduction
  • Waterfall Model
  • Iterative Model
  • V-Model
  • Spiral Model
  • Big Bang Model
  • RAD Model
  • Prototyping Model
    Software Testing Types
  • Static Testing
  • Dynamic Testing
  • Blackbox Testing
  • Whitebox Testing
  • Unit Testing
  • Requirements Testing
  • Regression Testing
  • Error Handling Testing
  • Manual support Testing
  • Intersystem Testing
  • Control Testing
  • Parallel Testing
  • Volume Testing
  • Stress Testing
  • Performance Testing
  • Agile Testing
  • Localization Testing
  • Globalization Testing
  • Internationalization Testing
    Test Plan
  • Introduction
  • Test Plan Development
  • Test Plan Template
  • Regional Differences
  • Criticism
  • Hardware Development
  • IEEE 829-1998
  • Testing Without a TestPlan
    Code Coverage
  • Introduction
  • Measures
  • Working
  • Statement Coverage
  • Branch Coverage
  • Path Coverage
  • Coverage criteria
  • Code coverage in practice
  • Tools
  • Features
    Quality Management
  • Introduction
  • Components
  • Capability Maturity Model
  • CMMI
  • Six Sigma
    Project Management
  • Introduction
  • PM Activities
  • Project Control Variables
  • PM Methodology
  • PM Phases
  • PM Templates
  • Agile PM
    Automated Testing Tools
  • Quick Test Professional
  • WinRunner
  • LoadRunner
  • Test Director
  • Silk Test
  • Test Partner
  • Rational Robot
    Performance Testing Tools
  • Apache JMeter
  • Rational Performance Tester
  • LoadRunner
  • NeoLoad
  • WAPT
  • WebLOAD
  • Loadster
  • OpenSTA
  • LoadUI
  • Appvance
  • Loadstorm
  • LoadImpact
  • QEngine
  • Httperf
  • CloudTest
  • Perl Testing
  • Python Testing
  • JUnit Testing
  • Unix Shell Scripting
    Automation Framework
  • Introduction
  • Keyword-driven Testing
  • Data-driven Testing
    Configuration Management
  • History
  • What is CM?
  • Meaning of CM
  • Graphically Representation
  • Traditional CM
  • CM Activities
  • Tools
  • What Is Software Testing?
  • Effective Defect Reports
  • Software Security
  • Tracking Defects
  • Bug Report
  • Web Testing
  • Exploratory Testing
  • Good Test Case
  • Write a Test
  • Code Coverage
  • WinRunner vs. QuickTest
  • Web Testing Tools
  • Automated Testing
  • Testing Estimation Process
  • Quality Assurance
  • The Interview Guide
  • Upgrade Path Testing
  • Priority and Severity of Bug
  • Three Questions About Bug
    Home » Testing Articles » Manual Testing Articles » What is beta testing?

    What is beta testing?

    A D V E R T I S E M E N T

    When you hear that a new or updated piece of software that you're interested in is 'in beta', your immediate reaction is usually a positive one: 'Great - it's nearly ready to be released!'

    Once beta testing has begun, it's easy to assume that the hard programming yards have been covered and now it's just a case of dotting the 'i's and crossing the 't's. Is this really the case for every company, though? Some products seem to be in beta for just a few weeks, while others remain in this state for months - maybe more! In some cases the beta process is shielded from users, with the public only hearing about the product when it's ready for release; but in others, the great unwashed are asked to test several different versions.

    Also, how do developers know when beta testing should stop? Can a product be declared 'finished' once it's out of beta, or does this just mean that it's stable enough to be sent to market?
    To beta or not to beta

    Let's try to define exactly what a company means when it says that its software is 'in beta'. Scott Fisher is Communications and Marketing Manager at Image-Line, developers of FL Studio. How would he define it?

    "I suppose we would contrast beta with alpha," Fisher says. "In alpha, the software is still under development, is likely to have structural changes and is tested by perhaps ten to 20 people. We go to beta when we have found most of the bugs we can with the alpha group and the software is basically in release form.

    "At this stage, it is far less likely to have any structural or design changes, but we are always open to that possibility. This phase is mainly about bug-hunting based on the wider testing and gathering feedback on workflow issues."

    Jonathan Hillman is Product Manager for PreSonus's Studio One. His view is that beta software "is a version that is ready to be evaluated beyond our internal group. At that point, to our knowledge, everything works as intended. The scope of the release is more or less proportional to the length of time we spend in beta; that is, a larger release requires more extensive beta testing."

    This is confirmed by Ohm Force, which is putting the finishing touches to its long-awaited, collaboration-friendly DAW Ohm Studio - the beta phase is currently in full effect.

    "Ohm Studio actually needs more beta testing than other sequencers because you need to also test the server loads, bandwidth, storage and features such as collaborative freeze that deal with non-shared plug-ins with a handy bounce. That means not only a beta, but a public one."

        "Public testing can begin when a product has reached a certain level of quality but now needs larger-scale feedback." Ohm Force

    Ah yes, the public beta testing programme - this has become increasingly common in recent years, particularly as the internet has made it easier for users to get hold of software and submit feedback on its performance. What are the benefits of public beta testing versus in-house/private evaluation, though?

    "Public beta testing involves a huge quantity and variety of systems, which is very desirable," says Hillman. "However, the quality of feedback tends to be inversely proportional to the size of the beta group. There's also the obvious disadvantage of showing your hand to the competition, which matters when you are innovating."

    There may be pros and cons to public beta testing, then, but Image-Line's Scott Fisher is of the view that the positives far outweigh the negatives - both for developers and users.

    "We adopted a strategy of public beta testing many years ago, simply because it means that when we do launch software formally, it's been pretty thoroughly tested by tens of thousands of people. We also have a core of customers who love to get their hands on our latest products as soon as possible. It's a win-win really."

    Is it the case, though, that customers are constantly being asked to use software that isn't finished? Scott Fisher doesn't think so.

    "Using the beta is entirely optional," he says. "We always direct customers to the latest official release for time- and mission-critical work. We do our best to keep the two separate in our customers' minds."

    Ohm Force makes the point that if you're going to send your beta out for public testing, you need to ensure you don't do it too early in the process.

    "If we had given access [to Ohm Studio] to anyone at the early stage of development, most people would have been disappointed, to the extent that frustration caused by technical failure would have been greater than frustration caused by staying on the pending list. Public testing can begin when a product has reached a certain level of quality but now needs larger-scale feedback."
    Knowing when to stop

    Just as importantly, how do you know when beta testing should end? "We set and track objectives for releases: address certain issues, expand or add features, rethink an interface element, and so on," says PreSonus' Jonathan Hillman.

    "When the objectives are met and stability is solid in our in-house tests, we enter beta. When stability is reported as solid within the beta group, and we're satisfied that no workflow issues have come up, we're finished."

    That sounds pretty clear, but 'finished' can be a tricky concept in software development, as Ohm Force confirm.

    "The truth is that Ohm Studio, like any DAW, will never be finished. Remember Ableton Live 1.0: it had a really innovative feature, but was also missing other big ones - it had no MIDI instruments at all! Many features that we consider essential were added in later versions.

    "Well, the same goes with Ohm Studio. As of today, no one offers a collaborative solution comparable to ours. While that and other aspects of Ohm Studio will be improved over the years, some people are very happy to use it already. That's one key marker � not for saying it's 'finished', but for the release. The other marker is having the package of features and bug fixes without which selling Ohm Studio would feel unacceptable."

    What should a company do if they've reached a set deadline for the end of beta testing and know that there are still problems to be fixed? Should the beta period be extended, or is it better to release first and iron out the kinks later?

    "Deadlines are man-made, so they are always open to being changed," says Scott Fisher. "If it's not ready, it's not shipped. On the other hand, software will always have bugs. Every developer knows there will be issues no matter how thorough the beta test was."

    In other words, you can beta test for as long as you like, but there will inevitably be some bugs that slip through the net. The important thing is that any issues that do remain aren't fundamental to the way the software works and performs.

    "If those issues have the potential to affect stability or functionality for a large number of users, no, [the software] should not be released," says Jonathan Hillman. "At the end of the day, users correctly value stability over everything." 

    More Manual Testing Articles
    1 2 3 4 5 6 7 8 9 10 11 Next

    discussionDiscussion Center


    Yahoo Groups
    Y! Group
    Sirfdosti Groups
    Contact Us

    Looking for Software Testing eBooks and Interview Questions? Join now and get it FREE!
    A D V E R T I S E M E N T

    Members Login

    Email ID:

    Forgot Password
    New User
    Testing Interview Questions
  • General Testing
  • Automation Testing
  • Manual Testing
  • Software Development Life Cycle
  • Software Testing Life Cycle
  • Testing Models
  • Automated Testing Tools
  • Silk Test
  • Win Runner
    Testing Highlights

  • Software Testing Ebooks
  • Testing Jobs
  • Testing Frequently Asked Questions
  • Testing News
  • Testing Interview Questions
  • Testing Jobs
  • Testing Companies
  • Testing Job Consultants
  • ISTQB Certification Questions
    Interview Questions

  • WinRunner
  • LoadRunner
  • SilkTest
  • TestDirector
  • General Testing Questions

  • Testing Forum
  • Downloads
  • E-Books
  • Testing Jobs
  • Testing Interview Questions
  • Testing Tools Questions
  • Testing Jobs
  • A-Z Knowledge
    Study ABROAD ?

    Study Abroad

    Vyom Network : Free SMS, GRE, GMAT, MBA | Online Exams | Freshers Jobs | Software Downloads | Programming & Source Codes | Free eBooks | Job Interview Questions | Free Tutorials | Jokes, Songs, Fun | Free Classifieds | Free Recipes | Bangalore Info | GATE Preparation | MBA Preparation | Free SAP Training
    Privacy Policy | Terms and Conditions
    Sitemap | Sitemap (XML)
    Job Interview Questions | Placement Papers | SMS Jokes | C++ Interview Questions | C Interview Questions | Web Hosting
    German | French | Portugese | Italian