OneStopTesting - Quality Testing Jobs, eBooks, Articles, FAQs, Training Institutes, Testing Software, Testing downloads, testing news, testing tools, learn testing, manual testing, automated testing, load runner, winrunner, test director, silk test, STLC

Forum| Contact Us| Testimonials| Sitemap| Employee Referrals| News| Articles| Feedback| Enquiry
Testing Resources
  • Testing Articles
  • Testing Books
  • Testing Certification
  • Testing FAQs
  • Testing Downloads
  • Testing Interview Questions
  • Career In Software Testing
  • Testing Jobs
  • Testing Job Consultants
  • Testing News
  • Testing Training Institutes
  • Introduction
  • Designing Test Cases
  • Developing Test Cases
  • Writing Test Cases
  • Test Case Templates
  • Purpose
  • What Is a Good Test Case?
  • Test Specifications
  • UML
  • Scenario Testing
  • Test Script
  • Test Summary Report
  • Test Data
  • Defect Tracking
    Software testing
  • Testing Forum
  • Introduction
  • Testing Start Process
  • Testing Stop Process
  • Testing Strategy
  • Risk Analysis
  • Software Listings
  • Test Metrics
  • Release Life Cycle
  • Interoperability Testing
  • Extreme Programming
  • Cyclomatic Complexity
  • Equivalence Partitioning
  • Error Guessing
  • Boundary Value Analysis
  • Traceability Matrix
    SDLC Models
  • Introduction
  • Waterfall Model
  • Iterative Model
  • V-Model
  • Spiral Model
  • Big Bang Model
  • RAD Model
  • Prototyping Model
    Software Testing Types
  • Static Testing
  • Dynamic Testing
  • Blackbox Testing
  • Whitebox Testing
  • Unit Testing
  • Requirements Testing
  • Regression Testing
  • Error Handling Testing
  • Manual support Testing
  • Intersystem Testing
  • Control Testing
  • Parallel Testing
  • Volume Testing
  • Stress Testing
  • Performance Testing
  • Agile Testing
  • Localization Testing
  • Globalization Testing
  • Internationalization Testing
    Test Plan
  • Introduction
  • Test Plan Development
  • Test Plan Template
  • Regional Differences
  • Criticism
  • Hardware Development
  • IEEE 829-1998
  • Testing Without a TestPlan
    Code Coverage
  • Introduction
  • Measures
  • Working
  • Statement Coverage
  • Branch Coverage
  • Path Coverage
  • Coverage criteria
  • Code coverage in practice
  • Tools
  • Features
    Quality Management
  • Introduction
  • Components
  • Capability Maturity Model
  • CMMI
  • Six Sigma
    Project Management
  • Introduction
  • PM Activities
  • Project Control Variables
  • PM Methodology
  • PM Phases
  • PM Templates
  • Agile PM
    Automated Testing Tools
  • Quick Test Professional
  • WinRunner
  • LoadRunner
  • Test Director
  • Silk Test
  • Test Partner
  • Rational Robot
    Performance Testing Tools
  • Apache JMeter
  • Rational Performance Tester
  • LoadRunner
  • NeoLoad
  • WAPT
  • WebLOAD
  • Loadster
  • OpenSTA
  • LoadUI
  • Appvance
  • Loadstorm
  • LoadImpact
  • QEngine
  • Httperf
  • CloudTest
  • Perl Testing
  • Python Testing
  • JUnit Testing
  • Unix Shell Scripting
    Automation Framework
  • Introduction
  • Keyword-driven Testing
  • Data-driven Testing
    Configuration Management
  • History
  • What is CM?
  • Meaning of CM
  • Graphically Representation
  • Traditional CM
  • CM Activities
  • Tools
  • What Is Software Testing?
  • Effective Defect Reports
  • Software Security
  • Tracking Defects
  • Bug Report
  • Web Testing
  • Exploratory Testing
  • Good Test Case
  • Write a Test
  • Code Coverage
  • WinRunner vs. QuickTest
  • Web Testing Tools
  • Automated Testing
  • Testing Estimation Process
  • Quality Assurance
  • The Interview Guide
  • Upgrade Path Testing
  • Priority and Severity of Bug
  • Three Questions About Bug
    Home » Testing Articles » Automated Testing Articles » A Good Test Automation Framework

    A Good Test Automation Framework

    A D V E R T I S E M E N T

    If you're looking at any kind of job posting (company career site,, and etc) for a software test engineer now, you will see almost all the posting mention some level of experiences in test automation framework as a requirement. And actually I do use test framework for my testing and it is very useful tool that resolves several limitations of manual testing.

    So I thought it is worthwhile to put my thought about what a good test automation framework is in my blog. First, let me scope the area of discussion since test automation framework is a pretty big topic. I'm going to set aside the discussion around pros and cons of test automation in software testing since that's not my main point of this article. And I'm talking about your work related test automation framework not the generic programming language specific test framework. For example, my test automation framework is built on top of JUnit or TestNG. JUnit and TestNG are also test automation frameworks, but those frameworks do not have the specific business logic of my testing.

    OK. Let's start. What is test automation frameworks?  Wikipedia mentions "A test automation framework is a set of assumptions, concepts and tools that provide support for automated software testing." My simple definition is "a test automation framework is a program that helps testers execute and validate test cases(test scenarios). So it is a program that helps software testing. The meaning of "help" will be saving time, being more efficient in testing process, gathering useful data easily after test execution and etc compared to the manual human test execution. But it is a program, which means someone has to design, implement and maintain. This brings a very important ground rule of test automation. Benefit of using test automation should always be greater than the effort of designing, implementing and maintaining the test automation. Otherwise there is no point using test automation beside you being cool. :)

    I believe having a good test automation framework is to maximize the benefits of test automation and minimize the effort of implementing and maintaining it. So let me address the characteristics of a good test automation framework.

    1. Have a clear strategy and everyone should be on the same page. This is the most important and most effective way to maximize the benefit of using the test automation framework. There are many different ways to design and implement test automation frameworks. And it is REALLY hard to come up with one solution that will satisfy every test engineers, which means when a test architect or experienced SDET come up with a framework, he/she should explain the strategy of the test automation and help everyone in the team understand how it works and how to use it. Once everyone is on the same page (know how it works and how to use), implementing test cases (scenarios) is much faster and maintaining the framework is much easier. By following a certain rule or instruction, test writers are less distracted and he/she can understand others code very easily. And when it comes to a situation where the current approach does not fit, it is much easier to notice and notify test automation designer (architect or SDET) to resolve the issue with proper solution.
    2. Favor object composition over class hierarchy. This is one advice from Gang of four design pattern. One of the common mistake with test framework is that some framework base class provide too many functionality and become giant class (a few thousand lines of code). This approach works fine at the beginning but it does not scale very well and becomes very complex code base as number of tests grows to handle various test conditions, error scenarios and exceptional cases. One key characteristics of good test framework is to provide flexibility to test writers. Test writers should be able to choose classes or modules they need to execute one test case. Just Mix and Match for their tests.
    3. Separation between test execution and validation. This is more about validation code being independent from test execution. It also means validation code by itself does not contain any context about test cases. What this allows to the framework is that you can package one logical chunk of  validation into one class. It will takes some parameters as source of validation. You call validate() method and it will simply return true or false. No decision making process here. For example, if error expecting flag is set do this. if some other case do that. NO. JUST DO THE VALIDATION. What does this allow test writers? You gather source of validation during test execution and pick and construct validation instances you need from pool of validation classes. DO NOT let the validation code to be workflow. LET IT BE collection of appropriate validations for each test case. This promotes great re-usability of the code and less complex code.
    4. Build the test framework to support the way test execution steps are defined. The ideal framework, in my opinion, is that defined (written or thought) test execution steps are transformed to test automation code with 1:1 mapping. This seems pretty easy to do but this requires thoughtful design to expose natural language like method names and encapsulate well structured implementation details. One way to think about this is that you're providing framework for test writers who are not as technical as you. This promotes great readability of the code as well. 
    5. Be cautious about re-factoring codes. Re-factoring is useful work. It's a good process that turns spaghetti codes to well-structured code. But often times, test automation writers get confused by this concept where he/she does not understand the difference between test automation code and development code. If you look closely on test automation code, there are two big parts: one is test execution steps related part and the other is test automation framework part that supports test execution step code. Re-factoring should largely applied in the latter part of automation code. Here is the common problem that I've seen. At first, test cases that a tester is implementing is very similar beside some inputs, config, or environment, so he re-factor the common piece in one method. Looks good. On the next iteration or release, he adds more test cases. He notices a little variation of the execution step or input, so he add some conditionals (if/switch statement) to the common method. As more iteration or releases come by, the common method becomes more and more complex. Complexity is not the only problem. His thought process of executing test steps is restricted by existing code base and he is trying to fit his thought on that code path. Also it is very hard for other testers to read and follow the logic. Refactoring test execution steps code can be very dangerous because it restricts testers thought process, makes test automation code less flexible, and makes readability of the code very bad and it hurts other test writers to take over. I admit that it is hard process because sometime it is hard to draw a line between test step code and supporting code. The key is to have test execution steps to be flexible and to have supporting code to be well structured.
    6. Pinpointing the failure is more important than handling error gracefully. Do not be scared of null pointer exception. Automation code is not production code. I don't think this needs a lot of explanations
    7. Minimize bootstrap code. Setup methods like @BeforeClass in JUnit or some sort of prep code runs before the test run is useful. But it also prevent your framework from being flexible. And it also make the framework very heavy. Sometimes you just want to check a simple validation that was used in test automation for other purpose. Well modularized code does not need much bootstrap code. You should be able to run your test in main method if your automation framework is well modularized.

    More Automated Testing Articles
    1 2 3 4 5 6 7 8 9 10 11 Next

    discussionDiscussion Center


    Yahoo Groups
    Y! Group
    Sirfdosti Groups
    Contact Us

    Looking for Software Testing eBooks and Interview Questions? Join now and get it FREE!
    A D V E R T I S E M E N T

    Members Login

    Email ID:

    Forgot Password
    New User
    Testing Interview Questions
  • General Testing
  • Automation Testing
  • Manual Testing
  • Software Development Life Cycle
  • Software Testing Life Cycle
  • Testing Models
  • Automated Testing Tools
  • Silk Test
  • Win Runner
    Testing Highlights

  • Software Testing Ebooks
  • Testing Jobs
  • Testing Frequently Asked Questions
  • Testing News
  • Testing Interview Questions
  • Testing Jobs
  • Testing Companies
  • Testing Job Consultants
  • ISTQB Certification Questions
    Interview Questions

  • WinRunner
  • LoadRunner
  • SilkTest
  • TestDirector
  • General Testing Questions

  • Testing Forum
  • Downloads
  • E-Books
  • Testing Jobs
  • Testing Interview Questions
  • Testing Tools Questions
  • Testing Jobs
  • A-Z Knowledge
    Study ABROAD ?

    Study Abroad

    Vyom Network : Free SMS, GRE, GMAT, MBA | Online Exams | Freshers Jobs | Software Downloads | Programming & Source Codes | Free eBooks | Job Interview Questions | Free Tutorials | Jokes, Songs, Fun | Free Classifieds | Free Recipes | Bangalore Info | GATE Preparation | MBA Preparation | Free SAP Training
    Privacy Policy | Terms and Conditions
    Sitemap | Sitemap (XML)
    Job Interview Questions | Placement Papers | SMS Jokes | C++ Interview Questions | C Interview Questions | Web Hosting
    German | French | Portugese | Italian