Why should you apply test automation?

Why should you apply test automation

The complexity of advanced technologies these days makes it a real challenge to deliver high-quality applications quickly, especially when your products need to go through intensive manual testing in the release process.

Taken into account different factors such as multi-platform support, cross-browser, responsive design and more, the manual testing of a website could face unexpected complications and extra effort. More time than not, these impediments prove to be considerable burdens and drag down delivery time as a consequence.

Automation testing, while initially seems to take more time, especially during the first few months of developing test scripts, will prove to be a more effective approach in the long run, both in regards of time and resources since by the time the automation solution is completed, it’s a matter of execution and maintenance with limited effort. The benefit is greatly multiplied in the project that requires intensive regression testing. The ability to reuse the test framework later by other teammates is also a big plus. Mature automation solutions typically do not require many staff for daily operations so that could save a bit budget for our project.

One of the key criteria for automation is that certain testing activities would prove to be impractical when carried out by human. Things such as validating the exact size & color of all controls on the application under test, processing a large amount of data or calculating the performance of the website are better executed by machine for consistency purpose. Besides, having such cases automated improves test coverage for many areas across projects.

What kind of tests should be considered for automation?

Ideally, one would prefer to automate as much test cases as possible to improve test coverage and take advantage of test automation. However, a typical real project will have limited time and resources. In that situation, the following tests should be given higher priority than others:

  • Repetitive Tests or regression testing: These are the most common cases for automation. The tests here are usually basic features of the product that need to be checked whenever a new build is available. These tests require much time to execute manually while they are typically easy to automate.
  • Detailed GUI Validation: These tests include cases such as checking for the exact size/color of a certain control or comparing two lengthy paragraphs.
  • Data Processing and Validation: Data-driven is a common practice in automation testing which helps users to evaluate whether the system processes data correctly or not. The ability to iterate a test multiple times thru a set of data or commit massive data transaction will help testers save a lot of regression time.
  • Cross Platforms Tests: Automation projects that cover these kinds of tests will improve test coverage and reduce manual test effort significantly. Automation tools nowadays usually support cross-platform execution where test scripts are written once and can be executed on different platforms later.
  • Impractical Manual Tests: These extremely-hard-to-execute test cases are also good candidates for automation. An example would be a test routine where the tester needs to interact with a certain control either in split seconds or after an exact timeout.

The benefit of test automation for these types of testing is greatly affected by which automation tools are used. While test automation often requires investment upfront in terms of time, budget and expertise, some tools require considerably much less than others. For example, Selenium and Katalon Studio are free test automation tools while TestComplete and HP UFT are comprehensive automation solutions but they are very costly. For a more detailed discussion of these tools, refer to A Comparison of Automated Testing Tools.

Conclusion:

It’s understandable that most of the time, the testing team cannot convert every single case of their manual test into automation. However, an optimal rate as much as 70% test coverage would remarkably improve the general quality of the product.

There are a lot of automation tools that do not require the user to be an expert in order to make use of them. So, next time, when you have a new test project with many test cases that fall into the mentioned situations above, take sometimes to consider an automation plan to make your release cycle faster and boost the overall productivity of the team.

About the Author

Abhishek Kumar, Product Manager

Abhishek Kumar is the Product Manager at KMS Technology. He combines strategic and tactical management expertise with strong qualifications in software development, software testing, test automation, product management, new venture initiation, business development, project management, and general operations. He earned a Master of Computer Applications (Software Engineering) from Guru Gobind Singh Indraprastha University Delhi.

3 Misconceptions about Test Automation

— Adam Satterfield, Vice President of Testing Services for KMS Technology, discusses how you can avoid common test automation mistakes.

The move towards a continuous delivery model of software development has contributed to the growing popularity of automated testing. DevOps has also encouraged the breakdown of barriers between traditional roles. Testers with coding skills (which is what automation testers really are) fit nicely into this new paradigm.

Test automation offers many potential advantages, including increasing efficiency and providing an effective way to solve complex testing issues. Automating tests also introduces more consistency, allowing for more efficient use of resources during off-peak hours. However, test automation is not a panacea. To realise the potential of test automation, it’s important to be aware of three common test automation mistakes.

3 Misconceptions Test Automation

3 Misconceptions of Test Automation

1) Automation will replace testers

The idea that you can take a tester with critical thinking skills, classically trained testing ability, and investigatory powers and get rid of them by putting code in their place is a fallacy. Scripts are not going to think critically or perform deep-dive investigations into your software. The scripts you employ are only as good as the tester who writes them.

We no longer see the industry as made up of manual and automated testers.  We have testers with coding skills and those without – all contribute in different ways. Trained testers will always have a place because of their ability to think critically, beyond a script.

Until we have powerful and affordable artificial intelligence, where the scripts are writing themselves, automated testing is just another tool in the box. It’s also worth noting that we fundamentally write software for humans to use, so a tester’s perspective will always be valuable.

2) You should automate UI tests first

This is a common misconception. In today’s DevOps and continuous delivery world, automating through the UI first is a recipe for disaster. The UI is brittle. It is in a constant state of flux as it evolves based on the needs of the client and the market. If you start test automation with the UI, then be prepared for your automation team to spend the bulk of their time rewriting UI tests.

UI tests typically take a very long time to run. What you want to do is focus on lower level or component-level tests first.  Not only are these the tests that are going to run more quickly and with every build, but lower level component, unit and integration tests tend not to change as frequently. Your scripts can validate that your base functionality is working and then you can employ manual exploratory testing to validate the UI.

UI tests are easier to write, but if you end up with a lot of UI tests and very few unit and integration tests, then you have an ice cream cone instead of a test pyramid. You’ve built a massive test repository that’s unmanageable and your automation team is going to be bogged down in rewriting test cases.

3) Automation testers just script manual test cases

The term ‘automation tester’ is restrictive. The last thing you want to do is confine a tester with coding skills to scripting manual test cases. Let them think outside the box and use their coding skills creatively.

For example, let’s say you have a tester working on RegEx functionality and they need to create 500,000 documents in a user’s Google drive to validate that functionality.  Instead of spending days creating the documents, if you give them the freedom, they might just spend a day or so using Google APIs to create a reusable script that generates the docs on demand.

Mocking or stubbing test data is an important part of testing. You don’t want testers waiting for developers to create mock environments/data with the right dependencies before they test. If they have coding skills and they are empowered to solve problems for themselves, then they can easily be self-reliant. If they spot an opportunity to boost efficiency through the creation of a simple tool, then they should be free to build it.

The same logic that is driving the DevOps movement applies to testers with coding skills. Blurring of roles can lead to new insights that harness your employee’s skills. This can improve the quality of the software currently under development, but also improve your entire set of development processes.

Test automation is an important part of modern software development. Just make sure that you take the time to formulate a solid strategy to get the best out of it.

About the Author: Jordan Platt

Source: Software Testing News

Best Automation Testing Tools for 2018 (Top 10 reviews)

Best Automation Testing Tools for 2018 (Top 10 reviews)

Software development practices change over time, so do the tools and technologies. Such changes aim to improve productivity, quality, customer satisfaction, to tackle ever-shorten delivery time, and to deliver successful products and services. Software testing obviously plays an important role in achieving these objectives.

The recently released World Quality Report 2017–2018 by Capgemini, Sogeti, and Marco Focus points out several interesting trends in software quality and testing. Two of three key trends are increasing test automation and widespread adoption of agile and DevOps methodologies. As the report shows, organizations need intelligent automation and smart analytics to speed up decision making and validation and to better address the challenges of testing smarter devices and products that are highly integrated and continuously changing. The report also suggests the need of smart test platforms that are self-aware and self-adaptive to support the complete application lifecycle.

In the test automation landscape, automation tools certainly take a center stage. This post summarizes the top test automation tools and frameworks that have the potential to help organizations to best position themselves to keep pace with the trends in software testing. The list includes both open-source and commercial test automation solutions.

Top 5 Automation Testing Tools for 2018

 

1. Selenium

Selenium is possibly the most popular open-source test automation framework for Web applications. Being originated in the 2000s and evolved over a decade, Selenium has been an automation framework of choice for Web automation testers, especially for those who possess advanced programming and scripting skills. Selenium has become a core framework for other open-source test automation tools such as Katalon Studio, Watir, Protractor, and Robot Framework.

Selenium supports multiple system environments (Windows, Mac, Linux) and browsers (Chrome, Firefox, IE, and Headless browsers). Its scripts can be written in various programming languages such as Java, Groovy, Python, C#, PHP, Ruby, and Perl.

While testers have flexibility with Selenium and they can write complex and advanced test scripts to meet various levels of complexity, it requires advanced programming skills and effort to build automation frameworks and libraries for specific testing needs.

Website: http://www.seleniumhq.org/

License: Open-source

2. Katalon Studio

Katalon Studio is a powerful test automation solution for web application, mobile, and web services. Being built on top of the Selenium and Appium frameworks, Katalon Studio takes advantage of these solutions for integrated software automation.

The tool supports different levels of testing skill set. Non-programmers can find it easy to start an automation testing project (like using Object Spy to record test scripts) while programmers and advanced automation testers can save time from building new libraries and maintaining their scripts.

Katalon Studio can be integrated into CI/CD processes and works well with popular tools in the QA process including qTest, JIRA, Jenkins, and Git. It offers a nice feature called Katalon Analytics which provides users comprehensive views of test execution reports via dashboard including metrics, charts, and graphs.

Website: https://www.katalon.com/

License: Free

3. UFT

Unified Functional Testing (UFT) is a well-known commercial testing tool for functional testing. It provides a comprehensive feature set for API, web services, and GUI testing of desktop, web, and mobile applications across platforms. The tool has advanced image-based object recognition feature, reusable test components, and automated documentation.

UFT uses Visual Basic Scripting Edition to register testing processes and object control. UFT is integrated with Mercury Business Process Testing and Mercury Quality Center. The tool supports CI via integration with CI tools such as Jenkins.

Website: https://software.microfocus.com/fr-ca/software/uft

License: Commercial

4. Watir

Watir is an open-source testing tool for web automation testing based on Ruby libraries. Watir supports cross browser testing including Firefox, Opera, headless browser, and IE. It also supports data-driven testing and integrates with BBD tools like RSpec, Cucumber, and Test/Unit.

Website: http://watir.com/

License: Open-source

5. IBM Rational Functional Tester

IBM RFT is a data-driven testing platform for functional and regression testing. It supports a wide range of application such as .Net, Java, SAP, Flex, and Ajax. RFT uses Visual Basic .Net and Java as scripting languages. RFT has a unique feature called Storyboard testing in which users’ actions on AUT are recorded and visualized in a storyboard format through application screenshots.

Another interesting feature of RFT is its integration with IBM Jazz application lifecycle management systems such as IBM Rational Team Concert and Rational Quality Manager.

Website: https://www.ibm.com/

License: Commercial

6. TestComplete

TestComplete by SmartBear is a powerful commercial testing tool for web, mobile, and desktop testing. TestComplete supports various scripting languages such as JavaScript, VBScript, Python, and C++Script. Like Katalon Studio, testers can perform keyword-driven and data-driven testing with TestComplete. The tool also offers an easy-to-use record and playback feature.

Like UTF, TestComplete’s GUI object recognition capability can automatically detect and update UI objects which helps reduce the effort to maintain test scripts when the AUT is changed. It also integrates with Jenkins in a CI process.

Website: https://smartbear.com/

License: Commercial

7. TestPlant eggPlant

An image-based automated functional testing tool that enables testers to interact with AUT the same way end users do. TestPlant eggPlant is completely different from traditional testing tools in its approach: modeling user’s point of view rather instead of the test scripts view often seen by testers. This allows testers with less programming skills to learn and apply test automation intuitively. The tool supports various platforms like Web, mobile, and POS systems. It offers lab management and CI integration as well.

Website: https://www.testplant.com/

License: Commercial

8. Tricentis Tosca

Tricentis Tosca is a model-based test automation tool that provides quite a broad feature set for continuous testing including dashboards, analytics, and integrations to support agile and DevOps methodologies.

Tricentis Tosca helps users to optimize the reusability of test assets. Like many other test automation tools, it supports a wide range of technologies and applications such as web, mobile, and API. Tricentis Tosca also has features for integration management, risk analysis, and distributed execution.

Website: https://www.tricentis.com/

License: Commercial

9. Ranorex

Ranorex is a quite comprehensive commercial automation tool for web, mobile, and desktop testing. The tool features advanced capabilities for GUI recognition, reusable test scripts, and record/playback. Codeless test creation is also a very useful feature that allows new automation testers to learn and apply test automation to their projects.

The tool supports Selenium integration for web application testing. Testers can distribute the execution of their tests across platforms and browsers using Selenium grid. Ranorex offers a low-pricing model for businesses.

Website: https://www.ranorex.com/

License: Commercial

10. Robot framework

Robot Framework is an open-source automation framework that implements the keyword-driven approach for acceptance testing and acceptance test-driven development (ATDD). Robot Framework provides frameworks for different test automation needs. But its test capability can be further extended by implementing additional test libraries using Python and Java. Selenium WebDriver is a popular external library used in Robot Framework.

Test engineers can leverage Robot Framework as an automation framework for not only web testing but also for Android and iOS test automation. Robot Framework can be easy to learn for testers who are familiar with keyword-driven testing.

Website: http://www.robotframework.org/

License: Open-source

— — — — —

As we can see, each of these automation tools has unique features to offer in addressing the growing challenges of software automation in the years ahead. Most provide capabilities for continuous testing and integration, test managementing, and reporting. They all support increasing automation needs for Web and Mobile testing. However, intelligent testing and smart analytics for adaptive and heterogeneous environments are still something to be desired for automation tools.

For further information on automation tool selection, refer to article A Comparison of Automated Testing Tools on Dzone.

References:

1 2 3 45