OneStopTesting - Quality Testing Jobs, eBooks, Articles, FAQs, Training Institutes, Testing Software, Testing downloads, testing news, testing tools, learn testing, manual testing, automated testing, load runner, winrunner, test director, silk test, STLC

Forum| Contact Us| Testimonials| Sitemap| Employee Referrals| News| Articles| Feedback| Enquiry
Testing Resources
  • Testing Articles
  • Testing Books
  • Testing Certification
  • Testing FAQs
  • Testing Downloads
  • Testing Interview Questions
  • Career In Software Testing
  • Testing Jobs
  • Testing Job Consultants
  • Testing News
  • Testing Training Institutes
  • Introduction
  • Designing Test Cases
  • Developing Test Cases
  • Writing Test Cases
  • Test Case Templates
  • Purpose
  • What Is a Good Test Case?
  • Test Specifications
  • UML
  • Scenario Testing
  • Test Script
  • Test Summary Report
  • Test Data
  • Defect Tracking
    Software testing
  • Testing Forum
  • Introduction
  • Testing Start Process
  • Testing Stop Process
  • Testing Strategy
  • Risk Analysis
  • Software Listings
  • Test Metrics
  • Release Life Cycle
  • Interoperability Testing
  • Extreme Programming
  • Cyclomatic Complexity
  • Equivalence Partitioning
  • Error Guessing
  • Boundary Value Analysis
  • Traceability Matrix
    SDLC Models
  • Introduction
  • Waterfall Model
  • Iterative Model
  • V-Model
  • Spiral Model
  • Big Bang Model
  • RAD Model
  • Prototyping Model
    Software Testing Types
  • Static Testing
  • Dynamic Testing
  • Blackbox Testing
  • Whitebox Testing
  • Unit Testing
  • Requirements Testing
  • Regression Testing
  • Error Handling Testing
  • Manual support Testing
  • Intersystem Testing
  • Control Testing
  • Parallel Testing
  • Volume Testing
  • Stress Testing
  • Performance Testing
  • Agile Testing
  • Localization Testing
  • Globalization Testing
  • Internationalization Testing
    Test Plan
  • Introduction
  • Test Plan Development
  • Test Plan Template
  • Regional Differences
  • Criticism
  • Hardware Development
  • IEEE 829-1998
  • Testing Without a TestPlan
    Code Coverage
  • Introduction
  • Measures
  • Working
  • Statement Coverage
  • Branch Coverage
  • Path Coverage
  • Coverage criteria
  • Code coverage in practice
  • Tools
  • Features
    Quality Management
  • Introduction
  • Components
  • Capability Maturity Model
  • CMMI
  • Six Sigma
    Project Management
  • Introduction
  • PM Activities
  • Project Control Variables
  • PM Methodology
  • PM Phases
  • PM Templates
  • Agile PM
    Automated Testing Tools
  • Quick Test Professional
  • WinRunner
  • LoadRunner
  • Test Director
  • Silk Test
  • Test Partner
  • Rational Robot
    Performance Testing Tools
  • Apache JMeter
  • Rational Performance Tester
  • LoadRunner
  • NeoLoad
  • WAPT
  • WebLOAD
  • Loadster
  • OpenSTA
  • LoadUI
  • Appvance
  • Loadstorm
  • LoadImpact
  • QEngine
  • Httperf
  • CloudTest
  • Perl Testing
  • Python Testing
  • JUnit Testing
  • Unix Shell Scripting
    Automation Framework
  • Introduction
  • Keyword-driven Testing
  • Data-driven Testing
    Configuration Management
  • History
  • What is CM?
  • Meaning of CM
  • Graphically Representation
  • Traditional CM
  • CM Activities
  • Tools
  • What Is Software Testing?
  • Effective Defect Reports
  • Software Security
  • Tracking Defects
  • Bug Report
  • Web Testing
  • Exploratory Testing
  • Good Test Case
  • Write a Test
  • Code Coverage
  • WinRunner vs. QuickTest
  • Web Testing Tools
  • Automated Testing
  • Testing Estimation Process
  • Quality Assurance
  • The Interview Guide
  • Upgrade Path Testing
  • Priority and Severity of Bug
  • Three Questions About Bug
    Home » Testing Articles » QTP Articles » Mercury WinRunner Evaluation

    Mercury WinRunner Evaluation

    A D V E R T I S E M E N T

    "The organization has created a suite of manual test cases using a text editor but is finding it difficult to maintain, use, and execute these test cases efficiently as the test organization's role grows. The test cases have proven effective in detecting defects before they reach production but the time required to manage and execute these test cases is now impacting the return on investment. Solution - invest in a test automation tool or suite of tools."


    The first thing an organization must accomplish is to catalogue what needs or requirements the Testing Software is expected satisfy. There are three categories or "points-of-view" that must be addressed: Management / Organization, Test Architecture, and End-User.

    Management / Organization Perspective

    Needs Analysis

    Management clearly stated the objective for purchasing the Test Automation solution was:

    "The selected Test Automaton tool shall enable end-users to author, maintain, and execute automated test cases in a web-enabled, shareable environment. Furthermore the test automation tool shall support test case design, automation, and execution .best practices. as defined by the Test Organization. Minimum acceptable ROI is 5 hours saved for every hour currently invested."


    General: Mercury WinRunner enabled the end-users to organize, author, and maintain a set of automated test cases and supporting software libraries in a shareable environment. Mercury WinRunner did support test case design, automation, and execution "best practices" as defined by the Test Organization but the framework to perform these tasks was certainly not "out-of-the-box". Mercury WinRunner provides the toolbox but it is the responsibility of the users to build and support the testing framework or purchase a testing framework that is compatible with Mercury WinRunner.

    Development: Mercury WinRunner provides a software development environment designed to meet the needs of Test Automation Engineers. The programming language, Test Script Language (TSL), has the look and feel of a simplified version of C. Experienced Automation Engineers or developers should have little difficulty in becoming proficient TSL developers. The actual organization of the TSL code is left up to the Engineers - the test organization must take responsibility for organizing and maintaining this code base.

    Mapping: Mercury WinRunner provides an effective GUI mapping utility that enables a skilled test automation engineer to map and maintain information on the application interface.

    Wizards: Mercury WinRunner is a fully mature Tier 1 test automation development tool. It comes with a large catalogue of automation wizards - most of these wizards are aimed at the play and record test automation paradigm.

    Maintenance: Mercury WinRunner is a test automation development studio, which allows for both development and maintenance of TSL code. Management and control of the software is left to the testing organization - therefore maintenance can become burdensome if the testing organization does not implement adequate configuration management practices.

    Summary: Mercury WinRunner met the needs of the testing organization but additional investment in a Testing Framework and a clear development paradigm must be implemented with the automation tool. If the testing organization does not implement a clear development paradigm and testing framework any value derived from test automation will be lost to the long-term maintenance burden.

    Test Architecture

    Needs Analysis

    Management has defined the immediate organizational goal but the long-term architectural necessities must be defined by the testing organization. The Testing Organization clearly stated the Architectural requirements for the Test Automation tool were:

    "The selected Test Automation tool shall have a history of operational success in the appropriate environments with a well established end-user community to draw upon. The tool must support enterprise wide collaboration over several simultaneous engagements / projects and smooth software upgrade path. The minimum acceptable ROI of 5 hours saved for every hour currently invested must be maintained across the enterprise and during any integration or upgrade activities."


    Summary: Mercury WinRunner enabled the end-users to organize, author, and maintain a set of automated test scripts across the enterprise. WinRunner is used and supported by large geographically dispersed user community - Mercury supplies an open forum where this community can share solutions. Integration is available for configuration management tools, test management tools, requirement management tools, and commercial testing frameworks - including Mercury's own Mercury Quality Center. Mercury's WinRunner is a well established toolset which several vendors from inside and outside the testing tool space has chosen to integrate with.

    End-User "Automation Engineer"

    Needs Analysis

    The End-User needs analysis detailed product capabilities as they apply to the testing process - this list of requirements extended for several pages and included several test automation challenges. In brief the End-User's stated that the Test Automation solution shall:

    1. Support the creation, implementation, and execution of Automated Test Cases.
    2. Support enterprise wide, controlled access to Test Automation (Web enabled preferred).
    3. Support Data Driven Automated Test Cases.
    4. Support Keyword enabled Test Automation.
    5. Enable Test Automation and verification of Web, GUI, .NET, and Java applications.
    6. Support the integration of other toolsets via a published API or equivalent capacity.


    Summary: The Test Automation Engineers found that WinRunner supplied the basic framework required to meet all the organization's automation needs. An enterprise wide test automation solution was implemented once WinRunner was combined with a disciplined approach to Test Case Design and software control.

    Lessons Learned

    From their automation work with WinRunner the Test Automation engineers learned three (3) valuable lessons:

    1. WinRunner is very dependent on the accuracy and robustness of the GUI map. The most effective way to construct and test a GUI map was to map against the most recent version and then test the map against earlier versions of the application. This helped the engineers identify object characteristics that would remain stable between releases - the engineers then used these object characteristics to map all objects of the same type. These mapping characteristics could be captured in the general startup script for all engineers.

    2. WinRunner comes with the capacity to execute a standard startup script when it is invoked. This script could be used to standardize several operational aspects of WinRunner: GUI map configuration, standard functions, and shared libraries or scripts. This startup script could also be manipulated to allow each engineer to load shared libraries, GUI maps, and functions and to load their own versions of the same elements. In other words the shared libraries were loaded first and then (when desired) the engineer could load their versions of these libraries or functions allowing them to develop against a common code base without impacting other engineers.

    3. WinRunner allows a group of engineers to work together by creating a common GUI map and script libraries. This enables the creation and maintenance of a toolkit that can be used and enhanced by the entire team. Once the automation solution has matured this shared resource can be managed using most configuration management tools - protecting the automation investment.

    Evaluation Summary

    Mercury WinRunner is Tier 1 Test Automation solution that met the need of our Testing Organization. Once the organization had moved into the Test Automation space the architectural limitations of WinRunner. were recognized and compensated for by applying a supporting test framework. It should be noted that there are several competing testing frameworks that integrate smoothly with WinRunner. It should also be noted that there are several Test Automation tools - including Mercury's own QuickTestPro (QTP) - that provide a more comprehensive testing framework. The testing organization was able to maintain a return-on-investment of eight hours saved for every hour invested for each test cycle.

    More QTP Articles
    1 2 3 4 Next

    discussionDiscussion Center


    Yahoo Groups
    Y! Group
    Sirfdosti Groups
    Contact Us

    Looking for Software Testing eBooks and Interview Questions? Join now and get it FREE!
    A D V E R T I S E M E N T

    Members Login

    Email ID:

    Forgot Password
    New User
    Testing Interview Questions
  • General Testing
  • Automation Testing
  • Manual Testing
  • Software Development Life Cycle
  • Software Testing Life Cycle
  • Testing Models
  • Automated Testing Tools
  • Silk Test
  • Win Runner
    Testing Highlights

  • Software Testing Ebooks
  • Testing Jobs
  • Testing Frequently Asked Questions
  • Testing News
  • Testing Interview Questions
  • Testing Jobs
  • Testing Companies
  • Testing Job Consultants
  • ISTQB Certification Questions
    Interview Questions

  • WinRunner
  • LoadRunner
  • SilkTest
  • TestDirector
  • General Testing Questions

  • Testing Forum
  • Downloads
  • E-Books
  • Testing Jobs
  • Testing Interview Questions
  • Testing Tools Questions
  • Testing Jobs
  • A-Z Knowledge
    Study ABROAD ?

    Study Abroad

    Vyom Network : Free SMS, GRE, GMAT, MBA | Online Exams | Freshers Jobs | Software Downloads | Programming & Source Codes | Free eBooks | Job Interview Questions | Free Tutorials | Jokes, Songs, Fun | Free Classifieds | Free Recipes | Bangalore Info | GATE Preparation | MBA Preparation | Free SAP Training
    Privacy Policy | Terms and Conditions
    Sitemap | Sitemap (XML)
    Job Interview Questions | Placement Papers | SMS Jokes | C++ Interview Questions | C Interview Questions | Web Hosting
    German | French | Portugese | Italian