3 Misconceptions about Test Automation

— Adam Satterfield, Vice President of Testing Services for KMS Technology, discusses how you can avoid common test automation mistakes.

The move towards a continuous delivery model of software development has contributed to the growing popularity of automated testing. DevOps has also encouraged the breakdown of barriers between traditional roles. Testers with coding skills (which is what automation testers really are) fit nicely into this new paradigm.

Test automation offers many potential advantages, including increasing efficiency and providing an effective way to solve complex testing issues. Automating tests also introduces more consistency, allowing for more efficient use of resources during off-peak hours. However, test automation is not a panacea. To realise the potential of test automation, it’s important to be aware of three common test automation mistakes.

3 Misconceptions Test Automation

3 Misconceptions of Test Automation

1) Automation will replace testers

The idea that you can take a tester with critical thinking skills, classically trained testing ability, and investigatory powers and get rid of them by putting code in their place is a fallacy. Scripts are not going to think critically or perform deep-dive investigations into your software. The scripts you employ are only as good as the tester who writes them.

We no longer see the industry as made up of manual and automated testers.  We have testers with coding skills and those without – all contribute in different ways. Trained testers will always have a place because of their ability to think critically, beyond a script.

Until we have powerful and affordable artificial intelligence, where the scripts are writing themselves, automated testing is just another tool in the box. It’s also worth noting that we fundamentally write software for humans to use, so a tester’s perspective will always be valuable.

2) You should automate UI tests first

This is a common misconception. In today’s DevOps and continuous delivery world, automating through the UI first is a recipe for disaster. The UI is brittle. It is in a constant state of flux as it evolves based on the needs of the client and the market. If you start test automation with the UI, then be prepared for your automation team to spend the bulk of their time rewriting UI tests.

UI tests typically take a very long time to run. What you want to do is focus on lower level or component-level tests first.  Not only are these the tests that are going to run more quickly and with every build, but lower level component, unit and integration tests tend not to change as frequently. Your scripts can validate that your base functionality is working and then you can employ manual exploratory testing to validate the UI.

UI tests are easier to write, but if you end up with a lot of UI tests and very few unit and integration tests, then you have an ice cream cone instead of a test pyramid. You’ve built a massive test repository that’s unmanageable and your automation team is going to be bogged down in rewriting test cases.

3) Automation testers just script manual test cases

The term ‘automation tester’ is restrictive. The last thing you want to do is confine a tester with coding skills to scripting manual test cases. Let them think outside the box and use their coding skills creatively.

For example, let’s say you have a tester working on RegEx functionality and they need to create 500,000 documents in a user’s Google drive to validate that functionality.  Instead of spending days creating the documents, if you give them the freedom, they might just spend a day or so using Google APIs to create a reusable script that generates the docs on demand.

Mocking or stubbing test data is an important part of testing. You don’t want testers waiting for developers to create mock environments/data with the right dependencies before they test. If they have coding skills and they are empowered to solve problems for themselves, then they can easily be self-reliant. If they spot an opportunity to boost efficiency through the creation of a simple tool, then they should be free to build it.

The same logic that is driving the DevOps movement applies to testers with coding skills. Blurring of roles can lead to new insights that harness your employee’s skills. This can improve the quality of the software currently under development, but also improve your entire set of development processes.

Test automation is an important part of modern software development. Just make sure that you take the time to formulate a solid strategy to get the best out of it.

About the Author: Jordan Platt

Source: Software Testing News

Best Automation Testing Tools for 2018 (Top 10 reviews)

Best Automation Testing Tools for 2018 (Top 10 reviews)

Software development practices change over time, so do the tools and technologies. Such changes aim to improve productivity, quality, customer satisfaction, to tackle ever-shorten delivery time, and to deliver successful products and services. Software testing obviously plays an important role in achieving these objectives.

The recently released World Quality Report 2017–2018 by Capgemini, Sogeti, and Marco Focus points out several interesting trends in software quality and testing. Two of three key trends are increasing test automation and widespread adoption of agile and DevOps methodologies. As the report shows, organizations need intelligent automation and smart analytics to speed up decision making and validation and to better address the challenges of testing smarter devices and products that are highly integrated and continuously changing. The report also suggests the need of smart test platforms that are self-aware and self-adaptive to support the complete application lifecycle.

In the test automation landscape, automation tools certainly take a center stage. This post summarizes the top test automation tools and frameworks that have the potential to help organizations to best position themselves to keep pace with the trends in software testing. The list includes both open-source and commercial test automation solutions.

Top 5 Automation Testing Tools for 2018


1. Selenium

Selenium is possibly the most popular open-source test automation framework for Web applications. Being originated in the 2000s and evolved over a decade, Selenium has been an automation framework of choice for Web automation testers, especially for those who possess advanced programming and scripting skills. Selenium has become a core framework for other open-source test automation tools such as Katalon Studio, Watir, Protractor, and Robot Framework.

Selenium supports multiple system environments (Windows, Mac, Linux) and browsers (Chrome, Firefox, IE, and Headless browsers). Its scripts can be written in various programming languages such as Java, Groovy, Python, C#, PHP, Ruby, and Perl.

While testers have flexibility with Selenium and they can write complex and advanced test scripts to meet various levels of complexity, it requires advanced programming skills and effort to build automation frameworks and libraries for specific testing needs.

Website: http://www.seleniumhq.org/

License: Open-source

2. Katalon Studio

Katalon Studio is a powerful test automation solution for web application, mobile, and web services. Being built on top of the Selenium and Appium frameworks, Katalon Studio takes advantage of these solutions for integrated software automation.

The tool supports different levels of testing skill set. Non-programmers can find it easy to start an automation testing project (like using Object Spy to record test scripts) while programmers and advanced automation testers can save time from building new libraries and maintaining their scripts.

Katalon Studio can be integrated into CI/CD processes and works well with popular tools in the QA process including qTest, JIRA, Jenkins, and Git. It offers a nice feature called Katalon Analytics which provides users comprehensive views of test execution reports via dashboard including metrics, charts, and graphs.

Website: https://www.katalon.com/

License: Free

3. UFT

Unified Functional Testing (UFT) is a well-known commercial testing tool for functional testing. It provides a comprehensive feature set for API, web services, and GUI testing of desktop, web, and mobile applications across platforms. The tool has advanced image-based object recognition feature, reusable test components, and automated documentation.

UFT uses Visual Basic Scripting Edition to register testing processes and object control. UFT is integrated with Mercury Business Process Testing and Mercury Quality Center. The tool supports CI via integration with CI tools such as Jenkins.

Website: https://software.microfocus.com/fr-ca/software/uft

License: Commercial

4. Watir

Watir is an open-source testing tool for web automation testing based on Ruby libraries. Watir supports cross browser testing including Firefox, Opera, headless browser, and IE. It also supports data-driven testing and integrates with BBD tools like RSpec, Cucumber, and Test/Unit.

Website: http://watir.com/

License: Open-source

5. IBM Rational Functional Tester

IBM RFT is a data-driven testing platform for functional and regression testing. It supports a wide range of application such as .Net, Java, SAP, Flex, and Ajax. RFT uses Visual Basic .Net and Java as scripting languages. RFT has a unique feature called Storyboard testing in which users’ actions on AUT are recorded and visualized in a storyboard format through application screenshots.

Another interesting feature of RFT is its integration with IBM Jazz application lifecycle management systems such as IBM Rational Team Concert and Rational Quality Manager.

Website: https://www.ibm.com/

License: Commercial

6. TestComplete

TestComplete by SmartBear is a powerful commercial testing tool for web, mobile, and desktop testing. TestComplete supports various scripting languages such as JavaScript, VBScript, Python, and C++Script. Like Katalon Studio, testers can perform keyword-driven and data-driven testing with TestComplete. The tool also offers an easy-to-use record and playback feature.

Like UTF, TestComplete’s GUI object recognition capability can automatically detect and update UI objects which helps reduce the effort to maintain test scripts when the AUT is changed. It also integrates with Jenkins in a CI process.

Website: https://smartbear.com/

License: Commercial

7. TestPlant eggPlant

An image-based automated functional testing tool that enables testers to interact with AUT the same way end users do. TestPlant eggPlant is completely different from traditional testing tools in its approach: modeling user’s point of view rather instead of the test scripts view often seen by testers. This allows testers with less programming skills to learn and apply test automation intuitively. The tool supports various platforms like Web, mobile, and POS systems. It offers lab management and CI integration as well.

Website: https://www.testplant.com/

License: Commercial

8. Tricentis Tosca

Tricentis Tosca is a model-based test automation tool that provides quite a broad feature set for continuous testing including dashboards, analytics, and integrations to support agile and DevOps methodologies.

Tricentis Tosca helps users to optimize the reusability of test assets. Like many other test automation tools, it supports a wide range of technologies and applications such as web, mobile, and API. Tricentis Tosca also has features for integration management, risk analysis, and distributed execution.

Website: https://www.tricentis.com/

License: Commercial

9. Ranorex

Ranorex is a quite comprehensive commercial automation tool for web, mobile, and desktop testing. The tool features advanced capabilities for GUI recognition, reusable test scripts, and record/playback. Codeless test creation is also a very useful feature that allows new automation testers to learn and apply test automation to their projects.

The tool supports Selenium integration for web application testing. Testers can distribute the execution of their tests across platforms and browsers using Selenium grid. Ranorex offers a low-pricing model for businesses.

Website: https://www.ranorex.com/

License: Commercial

10. Robot framework

Robot Framework is an open-source automation framework that implements the keyword-driven approach for acceptance testing and acceptance test-driven development (ATDD). Robot Framework provides frameworks for different test automation needs. But its test capability can be further extended by implementing additional test libraries using Python and Java. Selenium WebDriver is a popular external library used in Robot Framework.

Test engineers can leverage Robot Framework as an automation framework for not only web testing but also for Android and iOS test automation. Robot Framework can be easy to learn for testers who are familiar with keyword-driven testing.

Website: http://www.robotframework.org/

License: Open-source

— — — — —

As we can see, each of these automation tools has unique features to offer in addressing the growing challenges of software automation in the years ahead. Most provide capabilities for continuous testing and integration, test managementing, and reporting. They all support increasing automation needs for Web and Mobile testing. However, intelligent testing and smart analytics for adaptive and heterogeneous environments are still something to be desired for automation tools.

For further information on automation tool selection, refer to article A Comparison of Automated Testing Tools on Dzone.


Quick Start: How to set up your first automation test with Katalon Studio

A quick guide to set up and start your first automation test with Katalon Studio, a complete test automation tool, built on the top of Selenium and Appium. You now can start automation testing on web & mobile with the least efforts and no costs. Free download at www.katalon.com

Read full guideline https://docs.katalon.com/x/l4Ei

  • Introduction

This Getting Started Guide gives you a quick introduction to Katalon Studio, from downloading to activation and running your first automation test.

  • Download Katalon Studio

Katalon Studio is available for free, supporting both Windows and Mac. Visit Katalon Studio website to download the latest version.

  • Environment Configuration

First you need to verify if your computer meets the System Requirements of Katalon Studio.

If you do Web UI testing, then no additional settings are needed except making sure the required browsers are installed. Check this list for all supported browsers.

In case of Mobile testing, you need to install Node.js, Appium and enable USB Debugging mode on your device first. Please refer to Installation Guide for more details.

  • Start Katalon Studio

To start Katalon Studio, double-click on the katalon.exe (Microsoft Windows) or katalon (Mac) file in the directory where you unpack Katalon Studio.

Once started, the application should display a splash screen.

  • Activate Katalon Build

After launching Katalon Studio, provide your registered username and password to activate your Katalon Studio Instance. The username and password are the ones used to login to https://www.katalon.com/

  • Feature Introduction

When your build is activated, the Quick Guide screen is displayed to guide you through all major features. You can skip this and view the Quick Guide later from the Help menu.

At the end of the introduction, Katalon Studio suggests to create a new project. Click OK.


  • Create a Project

Enter the name of your new project and the location to store the project data on your machine. Click OK.

A new project will be generated at specified location.

  • Create your first Test Case
  1. Select option to create New Test Case from the main Toolbar.
  2. Provide the name for your test case then click OK.
  3. A new test case will be created accordingly. The test case editor is also displayed for editing.
  4. You can compose the test case by recording, scripting or manually entering test steps. For now, let’s quickly generate the test script using Record feature. Click on the Record icon from the main Toolbar.
  5. Click Record to launch the browser. During recording, actions performed on the browser will be captured and be generated into test steps.

Note: You can specify the target browser to be launched by selecting from the Record dropdownlist.

  1. Let’s create a simple script by going to Google Search and lookup for some keywords as illustrated on the screen above. Click OK of Record dialog when you finish recording.
  2. You are prompted to confirm saving all captured objects to Katalon Studio. Click OK.
  3. Captured actions and objects are generated in Katalon Studio.
  4. Now you can click on Run command from the main Toolbar to execute the script.

Note: You can specify the target browser to run the script by selecting it in the Run dropdown list.

Congratulations! You have just successfully created and executed your first test case.

For further instructions and help, please refer to Katalon User Guide or Katalon Forum

Source: Quick Start – Guide to set up and start your first automation test with Katalon Studio

1 2 3 4 46