Automated Testing Tool 152U1A0550

1. INTRODUCTION
Katalon Studio is a complete and thorough toolset for web and mobile app automation
testing. It comes with a spectrum of powerful features, including pop-up, iFrame, wait-time
and more, to overcome common challenges in Web UI test automation. This user-friendly
solution helps testers test better, work faster and launch high quality software, thanks to the
intelligence it adds to the whole test automation process.

Katalon Studio is a free test automation tool developed by KMS technology. It can be used to
automate websites, mobile applications as well as APIs. Learning Katalon doesn't require a
steep learning curve because of that it has gained wide popularity among the manual QAs
having limited programming knowledge. Our Katalon Studio tutorial is designed for
beginners with little or no knowledge of programming. Hence, it can be used by manual
testers as well, who want to move to automation.

It has record & playback and manual mode that help non-programmers to effectively create
automation test cases. For users with programming expertise, it has script mode as well using
which users can write test scripts in Groovy.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 1

Automated Testing Tool 152U1A0550

2.AUTOMATED VS MANUAL TESTING

Manual testing is testing of the software where tests are executed manually by a QA Analysts. It
is performed to discover bugs in software under development.

In Manual testing, the tester checks all the essential features of the given application or
software. In this process, the software testers execute the test cases and generate the test reports
without the help of any automation software testing tools.

It is a classical method of all testing types and helps find bugs in software systems. It is
generally conducted by an experienced tester to accomplish the software testing process.

In Automated Software Testing, testers write code/test scripts to automate test execution. Testers
use appropriate automation tools to develop the test scripts and validate the software. The goal
is to complete test execution in a less amount of time.

Automated testing entirely relies on the pre-scripted test which runs automatically to compare
actual result with the expected results. This helps the tester to determine whether or not an
application performs as expected.

Automated testing allows you to execute repetitive task and regression test without the
intervention of manual tester. Even though all processes are performed automatically,
automation requires some manual effort to create initial testing scripts.

FIG.2.1:AUTOMATED TESTING VS MANUAL TESTING

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 2

Automated Testing Tool 152U1A0550

3. BEST AUTOMATED TESTING TOOLS

The demand for delivering quality software faster — or “Quality at Speed” — requires

organizations to search for solutions in Agile, continuous integration (CI) and DevOps
methodologies. Test automation is an essential part of these aspects. The latest World Quality
Report 2018–2019 suggests that test automation is the biggest bottleneck to deliver “Quality at
Speed,” as it is an enabler of successful Agile and DevOps adoption.
Test automation cannot be realized without good tools; as they determine how automation is
performed and whether the benefits of automation can be delivered. Test automation tools is a
crucial component in the DevOps tool chain. The current test automation trends have increased
in applying artificial intelligence and machine learning (AI/ML) to offer advanced capabilities
for test optimization, intelligent test generation, execution, and reporting. It will be worthwhile to
understand which tools are best poised to take advantage of these trends.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 3

Automated Testing Tool 152U1A0550

FIG:3.1 BEST AUTOMATED TESING TOOLS

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 4

Automated Testing Tool 152U1A0550

4. CHALLENGES OVERCOMED USING KATALON STUDIO

Test automation is an essential subset of software testing. By using automated testing, we can
expedite the process of software validation and increase testing coverage. However, there are a
lot of challenges in applying test automation for applications under test (AUT).

Without overcoming these challenges, testers might face countless nightmares that can cause
software automated testing failure. This topic aims to outline the top five challenges that have
the highest impact on the overall automation test effort and project success. Hopefully, the
earlier these challenges are understood, the better solutions are prepared to deal with them.

4.1. Effective Communicating and Collaborating in Team

This is perhaps a challenge not just in test automation but also in manual testing teams.
However, it is more complicated in test automation than in manual testing because it requires
more communication and collaboration in the automation team. Indeed, test automation is an
investment. Therefore, like any other investments, to get the whole team members to involve
in identifying test automation objectives and setting targets, we need to spend significant efforts
on communication and provide huge evidence, historical data, and we even do a proof of
concept. In addition, to have clear purposes and goals, we necessarily keep the entire team on
the same page.

4.2. Selecting a Right Tool

Nowadays, there are a variety of testing tools, ranging from free and open-source tools
like katalon and selenium to commercial tools like and supporting different testing types and
technologies. Each tool tends to support particular situations. Vendors of testing products have a
tendency to exaggerate the ability of their products. Vendors often assume that they have a
“secret sauce” for all automation tastes. This causes misconceptions and confusions for us to
select an appropriate testing tool satisfying our needs. Plus, many of us do not do enough
research before making a decision about tool selection, and we tend to buy popular commercial
tools quickly based on an inadequate evaluation. Remember that a sufficient assessment
includes defining a set of tool requirements criteria based on the AUT and the experience of
experts who have already used the tools considerably.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 5

Automated Testing Tool 152U1A0550

4.3. Demanding Skilled Resources

Some people claim that test automation can be just handled by manual testers or any technical
testers because many test tools already support recording test scripts and playing back them so
easily and quickly. This is a huge myth. In fact, test automation requires necessary technical
skills to accurately design and maintain test automation framework and test scripts, build
solutions, and resolve technical issues. Automated testing resources need to have strong
knowledge of framework’s design and implementation. To fulfill these job requirements,
obviously, these resources need to have both strong programming skills and solid test
automation tools.

4.4. Selecting a Proper Testing Approach

Automation tests not simply require a right tool to create scripts but also need a correct testing
approach. This is one of the biggest challenges for test automation engineers. Technically,
it’s vital for testers to find an appropriate test automation approach. In order to do so, they have
to answer several important questions: How to reduce effort in both implementation and
maintenance of test script and test suite? Will automation test suites be having a long lifetime?
How to generate useful test reports and metrics? With adopting the Agile development in recent
years, the application under test often changes through development cycles.

4.5. High Upfront Investment Cost

Talking about test automation, most of us agree that automated regression testing is crucial and
useful in most Agile contexts. But when turning into the cost, we have many concerns. As a
matter of fact, the initial phase of test automation is usually expensive. It’s necessary to analyze,
design and build a test automation framework, libraries or reusable functions, etc. In some
cases, it is required to take into account licensing costs, facilitating and operating costs such as
hardware and software costs. Moreover, even though we can use free open-source tools to
reduce the licensing costs, we might spend significant efforts on learning, training and
maintaining them.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 6

Automated Testing Tool 152U1A0550

5. SETUP KATALON STUDIO

1. Go to Katalon Studio official website -Katalon studio

2. Click on "Download Now" button. New users will be asked to Sign up.

3. Enter the required fields and click on 'Download' button (save the credentials as the same
will be used to activate the tool later on).

4. In the "Download Katalon Studio" window, select the operating system and click on
"Accept and Download".

5. Now you just need to extract the package and clicking on Katalon.exe(in case of windows)
will launch the tool.

6. In order to activate the tool, it will ask for the email Id and password. Enter the credentials
used while creating account on Katalon (step #3).

This concludes the setup and installation of Katalon studio, next we will learn how to create
test scripts for web application using record and playback feature.

FIG:5.1 SETUP AND INSTALLATION OF KATALON

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 7

Automated Testing Tool 152U1A0550

6. EXECUTION OF KATALON
 You can configure general execution preferences by accessing from main menu: Project
> Project Settings > Execution > Default.The default environment that Katalon Studio
will use to execute automation test. The default timeout period (in seconds) that Katalon
Studio will wait for the application under test to be loaded when executing automation
test. These options decide the actions that Katalon Studio will perform after finishing
automation test. Open report Specify whether the report generated after your test suite
finishes its execution will also be open immediately. Terminate drivers specify when any
driver remains after execution will be terminated. All the above preferences are
savedinto .prefs filesunderthe“config\.metadata\.plugins\org.eclipse.core.runtime\.settin
gs” location in your Katalon Studio build folder. You can manually modify the values in
files to change these preference settings.

FIG 6.1:DEFAULT EXECUTION

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 8

Automated Testing Tool 152U1A0550

7. FEATURES OF KATALON STUDIO

Simple deployment: A single, well organized deployment package that contains everything a
tester needs for deploying a powerful test automation tool.
Quick & easy set-up: Beyond the simple installation, Katalon Studio makes it easy to set up
the environment. Using Katalon’s pre-built templates and test scripts, it is very easy for
testers to run their first test script.
Faster & weightier outcomes: Built-in example copies, with clear steps on field-specific
training, help testers create and run automation test rough writings expeditiously. Every step
ranging from undertaking organization, test engenderment, execution, report generation and
support, can be executed using Katalon.
Flexible modes: New testers can use recordings and keywords to create automation tests,
while expert testers can employ a full IDE to create increased rough writings.
Easy usage: Manual testers with minimum programming skills can get the full benefits with
very easy steps.
Cross-browser application: Katalon supports the following platforms – Windows 32 and 64
(7, 8, and 10) and OS 4 X 10.5+.

FIG :7.1KATALON FEATURES

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 9

Automated Testing Tool 152U1A0550

8.KATALON PRODUCTS

As test automation becomes more crucial in the technology landscape, it can be challenging to
find an automation solution that is affordable, simple to set up, and comprehensive to meet a
variety of automation needs.

To address these concerns, we initially created Katalon Studio as a viable alternative to both
open-source and commercial testing tools. Now we have expanded our range to a variety of
products that will assist your automation testing needs.

 Katalon Studio

Katalon Studio is a free and robust automation solution for API, Web, and Mobile
testing. It integrates all necessary components with built-in keywords and project
templates into a complete automation framework.\ Katalon Studio is easy to use for
beginners but still offers advanced capabilities for experienced users. This solution is
trusted by an active community of over 150K users from 150+ countries around the
world.

 Katalon Analytics

A cloud-based application that provides in-depth views of test execution reports through

through powerful visualization including charts, graphs, and metrics.

 Katalon Recorder

A Selenium’s IDE-compatible replacement on latest Chrome and Firefox versions.

Katalon Recorder records, plays, debugs, manages automated tests, and exports to C#,
Java, Ruby, Python, Groovy, or Robot Framework.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 10

Automated Testing Tool 152U1A0550

9.Contribution Guidelines
We strongly encourage you to help improve Katalon Knowledge Base by providing feedback,
suggestions, or even issues regarding our products. Below are several ways for you to
contribute to Katalon documentation, including editing, requesting docs change, rating, or
raising issues. Here are some suggestions:

 Basic

o Requests for docs change due to bugs, e.g. missed item, incorrect item, or
confusing content

o Typos or grammatical errors

 Advanced

o Reorganizing or rewriting docs content

o New docs about Katalon Studio features

Please notice that although we very much appreciate all of your feedback, pull requests should
be related to the documentation in this repository only.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 11

Automated Testing Tool 152U1A0550

10. WHAT MAKES KATALON STUDIO MORE POWERFUL?

 Git for version control system and team collaboration.
 Jira BBD plugins for BDD practices and bug submission.
 Slack/email integration for notification and reports.
 qTest integration for Test management.[11]
 Kobiton [12]/Browser stack/Sauce Labs integration for cloud test environments.

FIG 10.1: FACTORS THAT MAKE KATALON MORE POWERFUL

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 12

Automated Testing Tool 152U1A0550

11. KATALON VS SELENIUM

Selenium is an automation API available in various programming languages like Java, JS,
Python, Ruby, C#. You need to download the library and create the automation framework of
your choice. You should be strong enough in coding to solve the automation scenario-based
problem.

Katalon studio is a software tool, has been built with readily usable features. You just need to
configure the software and use it for automation. Programming knowledge not required.

Selenium is open for integration with other tools and frameworks to enhance its capability, but
Testing teams need to have good background in programming. Selenium is an automation API
available in various programming languages like Java, JS, Python, Ruby, C#.

The Katalon Studio is integrating necessary frameworks and features for fast test cases
execution and creation. It’s also good for testers with limited technical knowledge.
Programming knowledge not required.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 13

Automated Testing Tool 152U1A0550

FIG:11.1 KATALON VS SELENIUM

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 14

Automated Testing Tool 152U1A0550

12. PROS OF KATALON

 Its setup and installation is very simple, you just need to download the respective
package based on your OS from katalon studio official website, extract it and then
activate it using your credentials.

 Katalon caters to the need of non-programmers by providing record & playback feature
and scripting in manual mode for test case creation. Along with that it has a relatively
simpler learning curve which helps manual tester to easily create automation tests.

 For those who have good programming expertise, Katalon provides test case creation in
scripting mode also.

 The in-built templates, frameworks, integration to Jira, Git, Jenkins etc features help in
speeding up the test case creation time. User only has to focus on automation of the test
scripts.

 The script recorded in one browser can be made to run in any of the supported browsers,
making cross browser testing fast and easy.

 Its test result reports are graphically intutive and can be exported to pdf and CSV
formats.

 It provides inherent logging mechanism and screenshot on failure feature.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 15

Automated Testing Tool 152U1A0550

13. CONS OF KATALON

– Lack of scripting languages

Unlike Selenium and TestComplete, the only scripting language supported by Katalon is
Groovy. This scripting language is in the Java family so anyone who knows Java can use it.
But, users would like to see more languages supported.

– Small community

Since Katalon was developed in 2015, the community is small, and you may experience a
lack of support from peer testers. Users must rely on the paid customer support team or
Katalon forum replies. To put this in perspective, Ranorex has 104 Capterra reviews, while
Katalon Studio has only 11.

– Not open source

The tool has a closed source code, resulting in a smaller number of developers in the
community. Selenium, one of the main competitors to Katalon, is an open-source tool that
allows engineers to customize it or use community-built packages. However, Katalon Studio
has some open components and is planning to introduce an open source platform.

– Documentation update lags

As much as the users like the number and variety of training materials for Katalon, they hate
the documentation. It’s too long, complicated, and causes confusion as it’s not updated along
with new version releases. So, anticipate that something won’t work as expected.

– Performance issues

Users report some bugs that hamper testing, slowing it down. For example, sometimes the
tool freezes or may start lagging, the text and object verification within iframes is
problematic. Mobile testing takes more time due to the necessity to capture and write code.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 16

Automated Testing Tool 152U1A0550

15. LIMITATIONS OF KATALON

 Unlike tools like Selenium the script creation is limited to Java and Groovy only.

 As of now there is no support for distributed testing.

 It cannot automate desktop applications unlike some licensed tools like UFT and
TestComplete.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 17

Automated Testing Tool 152U1A0550

16. CONCLUSION

Katalon Studio is a good choice for small and medium businesses. It’s an evolving solution
with many integrations that allow you to cover a variety of testing types with a single tool. It
comes with all the necessary installations out of the box to run various types of tests,
including API testing. It’s free and easy to use. Katalon can be used by specialists with
different QA engineering roles and varied programming skills, making it an attractive solution
for teams with testers of different levels. Katalon Studio is mostly used to automate functional
testing to reduce time spent on scripting. That said, because it has downsides like minor bugs
and support for only one scripting language, it’s not the best enterprise solution.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 18

Automated Testing Tool 152U1A0550

17. ACRONYMS

ACRONYM ABBREVATION

QA Quality Assurance

BDD Behaviour Driven Development

SOAP Simple Object Access Protocol

UFT Unified Functional Testing

ALM Application Lifecycle Management

UI User Interface

API Application Programming Interface

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 19

Automated Testing Tool 152U1A0550

18. REFERENCES
1. "KMS Technology Releases Katalon Studio, Free Intelligent Test Automation

Toolset". Retrieved 2018-05-16.

2. https://www.gartner.com/reviews/customers-choice/software-test-automation/Mar-

2019

3. Jump up to:a b c Adams, Ajamo (2018). Stop Coding - Learn to test automate without

coding and get that automation testing job.

4. "Importing an External Library Into Your Automation Project in Katalon Studio -

DZone DevOps". dzone.com. Retrieved 2018-05-16.

5. "A Structured Evaluation for Selecting a Right Automated Testing Tool". HCMC

Software Testing Club. Retrieved 2018-05-16.

6. "Creating Test Cases Using Script Mode With Katalon Studio - DZone

DevOps". dzone.com. Retrieved 2018-05-16.

7. "Best Automation Testing Tools for 2018 - Medium". Medium.com. Retrieved 2017-

10-26.

8. "Roadmap and Voting for your features". Katalon Q&A. Retrieved 2018-05-16.

9. "uTest". www.utest.com. Retrieved 2018-05-16.

10. Jump up to:a b "Test Integrations - Katalon Studio - Katalon

Documentation". docs.katalon.com. Retrieved 2018-05-16.

11. "KMS Technology Releases Katalon Studio v5.0, Featuring Katalon Analytics

(Beta)". Benzinga. Retrieved 2018-05-16.

12. "Integrating Katalon Studio, a free automated testing solution, with qTest Manager -

QASymphony". QASymphony. 2017-08-10. Retrieved 2018-05-16.

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING Page 20