Running Automated UI Tests
Technology
Mar 02, 2021
Overview
User interface (UI) testing is a process that simulates user interactions with a website in order to test if the application is functioning correctly. UI testing can be performed manually by a human tester, or it can be performed automatically using software.
Automated UI testing is the automation of manual test tasks. Because manual testing can be time-consuming and error-prone, we implement automated UI tests as a more accurate, efficient, and reliable method. Over an extended period of time, automated UI testing becomes a cost-effective replacement for manual testing. However, it should be implemented after some portion of the functionality is finished (e.g. result of a sprint). Developing tests in parallel with mainstream implementation can result in headaches while maintaining new functionality.
Furthermore, automated UI tests can run simultaneously on different platforms and on different browsers. However, they will not detect cases where the UI gets distorted. As soon as all DOM elements are in the document, the test will succeed. Automated UI tests become much more important when there are multiple people working on the same piece of code.
One of the biggest benefits of automated UI testing is finding regression errors in order to ensure that new code doesn’t break or change the existing functionality.
Technologies
Selenium
Selenium is a portable framework for testing web applications. Selenium provides a playback tool for authoring functional tests without the need to learn a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including C#, Groovy, Java, Perl, PHP, Python, Ruby, and Scala. The tests can then run against most modern web browsers. Selenium runs on Windows, Linux, and macOS. It is open-source software released under the Apache License 2.0.
NUnit
NUnit is an open-source unit testing framework for the .NET Framework and Mono. It serves the same purpose as JUnit does in the Java world and is one of many programs in the xUnit family.
Features
- Tests can be run from a console runner, within Visual Studio through a Test Adapter, or through third-party runners.
- Tests can be run in parallel.
- Strong support for data-driven tests.
- Supports multiple platforms including .NET Core, Xamarin Mobile, Compact Framework, and Silverlight.
- Every test case can be added to one or more categories to allow for selective running.
NUnit provides a console runner (nunit3-console.exe), which is used for batch execution of tests. The console runner works through the NUnit Test Engine, which provides it with the ability to load, explore, and execute tests. When tests are to be run in a separate process, the engine makes use of the nunit-agent program to run them.
The NUnitLite runner may be used in situations where a simpler runner is more suitable. It allows developers to create self-executing tests.
Jenkins
Jenkins is a free and open source automation server. Jenkins helps to automate the non-human part of the software development process, with continuous integration and facilitating technical aspects of continuous delivery. It is a server-based system that runs in servlet containers such as Apache Tomcat. It supports version control tools, including AccuRev, CVS, Subversion, Git, Mercurial, Perforce, TD/OMS, ClearCase, and RTC and can execute Apache Ant, Apache Maven, and sbt-based projects as well as arbitrary shell scripts and Windows batch commands. The creator of Jenkins is Kohsuke Kawaguchi. Released under the MIT License, Jenkins is free software.
Selenium Grid
Selenium Grid is a server that allows tests to use web browser instances running on remote machines. With Selenium Grid, one server acts as the hub. Tests contact the hub to obtain access to browser instances. The hub has a list of servers that provide access to browser instances (WebDriver nodes), and lets tests use these instances. Selenium Grid allows running tests in parallel on multiple machines and to manage different browser versions and browser configurations centrally (instead of in each individual test).
The ability to run tests on remote browser instances is useful to spread the load of testing across several machines and to run tests in browsers running on different platforms or operating systems. The latter is particularly useful in cases where not all browsers to be used for testing can run on the same platform.
Workflow
Test Plan
Setting up
The first step in the workflow of the automated tests is the definition of the test. We need to define what we are going to test. The most common elements that we check are:
- Check all the GUI elements for size, position, width, length, and acceptance of characters or numbers. For instance, you must be able to provide inputs to the input fields.
- Check you can execute the intended functionality of the application using the GUI.
- Check Error Messages are displayed correctly.
- Check for Clear demarcation of different sections on screen.
- Check Font used in an application is readable.
- Check for proper test alignment.
- Check the color of the font and warning messages are aesthetically pleasing.
- Check that the images have good clarity.
- Check that the images are properly aligned.
- Check the positioning of GUI elements for different screen resolution.
We can create an individual test definition based in each use case, or create a general test plan with related tests. We can do it manually, through a document, Confluence, or using a tool.
Azure Test Plan is a highly recommended way to define or Test Plans, Test Suits or Test cases.
After we document our tests, it’s time to set up our test environment. On oshyn.com we use Selenium Driver for C# . Selenium is an open-source, web automation testing tool that supports multiple browsers and multiple operating systems. It allows testers to use multiple programming languages such as Java, C#, Python, .Net, Ruby, PHP, and Perl for coding automated tests. We can easily install Selenium Web Driver on a Visual Studio using the Nuget Package Manager and searching for Selenium.WebDriver by Selenium Committers.
Using Selenium official documentation we can set up the Web Driver for our project. In our case C#, which you can see in the following image:
Test definition
Once we have our Selenium Web Driver successfully installed, we can start to define our test based on our test specifications. The most recommended way is to integrate our UI Test with NUnit. NUnit is a unit-testing framework for all .Net languages. Initially ported from JUnit, the current production release, version 3, has been completely rewritten with many new features and support for a wide range of .NET platforms. It’s open source software released under the MIT license and is part of xUnit family.
In the image bellow we can check two tests defined using NUnit for our home page:
And, as you can see, we have a Base Page class for each PageTests class. This way it’s easier to maintain our tests and separate the logic of the tests from the additional page related logic that we need to use to accomplish our test. (See next image)
Control definition
A very important step in the UI tests is the controls definitions that we use for our tests. The controls are the elements that the Web Driver uses to interact with our site. In Selenium automation, if the elements are not found by the general locators like id, class, name, etc. then XPath is used to find an element on the web page. Defining these elements in a separate place from our test logic is very good practice. Doing this avoids affecting the test logic when we change the element or the element location.
Running Test with Visual Studio Test Explorer
The last step in our UI test flow is to run the tests. With Visual Studio this is really simple. You need to open the Test Explorer, which can be accessed under the Test Menu (Ctrl + E, T). After a quick scan, you are going to see all the tests under your project. You can run them all, or select a single one or a group. You are going to see a green circle icon if the test successfully passed or a red circle icon if the test fails. If it fails, you will also see a description of the cause of the failure.
Deploying and running tests with Jenkins against the grid
If you work within a continuous integration or continuous delivery environment, the integration of the UI test process becomes really powerful. Jenkins is an excellent bet to set up your integration environment. It offers a simple way to set up for almost any combination of languages and source code repositories using pipelines. It also can automate other routine development tasks, like in this case, integration tests. While Jenkins doesn’t eliminate the need to create scripts for individual steps, it does give you a faster and more robust way to integrate your entire chain of build, test, and deployment tools than you can easily build yourself.
You can add nunit3-console after finishing any environment build using your tests-library.dll as a source and running your tests against a local or remote environment. A good call is to run your tests pointing to a Selenium Grid. Selenium Grid uses a hub-node concept where you only run the test on a single machine called a hub, but the execution will be done by different machines called nodes. So in this case you can set up different configurations for your tests, like browsers, operating systems, and browsers versions.
Conclusion
The UI testing process is of great importance in the software development process. It validates how your website interacts with the final user. It tests the application by considering usability, making sure that all the functionality of the site is easy to understand and use (e.g. navigate between pages). A good UI test reflects a complete user interaction with the site, and should be simple and clean. A UI test also checks for errors. It automatically tests and validates all possible scenarios according to the project requirements. Finally, you have a complete validation of the look-and-feel of the application. (This can only be accomplished with UI Tests. Unit tests can also check the first two.) For example, you can validate that all the images on your site are visible, that the color combinations match the specs, or if the required fields are correctly validated and can be seen by the user, etc.
Ultimately the client or user is the person who is actually going to use the application. It is critical to test all aspects of the UI, so they will not have any problems using your site or application.
References
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.