How UI testing can fit into the CI/CD pipeline

Testing plays a vital role in the software delivery cycle. UI testing in the continuous integration and continuous delivery (CI/CD) pipeline balances delivery speed and test coverage. So, does UI testing really fit into the CI/CD? And what tests should developers implement without slowing down their pipelines?

Below are ways for developers to be more efficient while scaling their testing operations without compromising project quality.

UI testing in the pipeline

Ramiro Millan

Ramiro is an automation testing strategist with over 20 years of experience in software development, quality assurance, and management. He has worked with small startups and large multinational companies. Ramiro has spent over a decade at Telerik and Progress in roles including product management, sales engineering, product testing leadership and, as recently as 2017, professional services leadership for DevTools and for Sitefinity. Ramiro currently leads the automated testing solutions division of Progress.

CI/CD is not a one-size-fits-all solution. Using CI/CD to run automated UI tests as part of the build process requires careful selection of tools, strategies, and tests to run in the pipeline. This requires a cultural shift in every organization, especially those that still practice completely manual UI testing. The main obstacles preventing teams from successfully integrating UI testing into CI/CD include:

  • Lack of skills in the team to automate tests and integrate them
  • Lack of capacity and infrastructure to support testing and maintain test automation
  • The tests are not stable enough and are mostly manual

Managing them is key to successfully transitioning from manual to automated in-process testing. While lack of skills and capabilities is a matter of investment in training and infrastructure, unstable and manual testing is rooted in the selection of appropriate tools to support the introduction of automated testing, even for QA less technical.

Why CI/CD?

The goal is to speed up the CI/CD pipeline and have a healthy application when deployed. Automation ensures a level playing field for all team members, including testing. The automation environment integrates source control, bug tracking, build integrations, test results, and remote test execution. Developers aren’t the only ones involved in the process as an agile environment requires collaboration across multiple departments, including testing.

Tooling and technology ease the learning curve for DevOps, QA, IT, and engineering teams. Success relies on the agile collaborative mindset rather than the traditional waterfall review process that can hamper application development and slow down the pipeline. As soon as an app is released, decay is imminent as browsers need updates, especially for security. Developers then start offering updates immediately, and because organizations want to deliver quality standards to their customers, critical updates need to be done quickly.

How to make UI testing efficient enough to fit CI/CD

The reliance on manual testing is the reason why organizations cannot successfully implement CI/CD. If CI/CD involves manual processes that cannot be maintained as they slow down the entire delivery cycle. Testing is no longer the sole responsibility of developers or testers and requires infrastructure investment and integration. Developer teams should focus on building essential coverage. They should focus on testing workflows and not features to be more efficient. Additionally, manual testers who aren’t developers can still be part of the process, as long as they use a testing tool that gives them the automation capabilities needed in a low-code environment. For example, with Telerik Test Studio, a manual tester can create an automated test by interacting with the application’s user interface in a browser. This test can be presented without code and therefore they can learn how the code behaves.

Another best practice for making UI testing effective is to be selective with what gets included in the pipeline. Automated UI tests can be very complex and take up to three hours and there can be too much room for error. This is not ideal as it will slow CI/CD cycles.

When managed effectively, some tests can be automated to run in 3-5 minutes and performed on a weekly basis. This includes basic smoke testing, regression testing, and acceptance testing, checking the quality of critical app features such as login, checkout, payment, etc.

Such testing will not cause additional delivery delays and at the same time ensure the quality and experience that end users are looking for. Developers can then focus on reliability if they don’t constantly rebuild and maintain testing. CI/CD enables automation flexibility and containerized sprint testing.

How to design robust, non-fragile automated tests and speed up test execution

In-sprint automation means collaboration with the development team. Ideally, testing and development should happen at the same time, and automation makes the test sprint easier. However, not all UI tests are eligible for CI/CD, and pipeline build tests are different from a full end-to-end visual test cycle. They must be fast, stable, strategic and headless. Headless tests save time because there is no user interface, they are faster and can be easily multiplied and scaled.

Accessibility is also an important aspect in the design testing process and should be a priority for designers, developers, and testers. For example, applications should be able to have color contrast and text zoom in and out capabilities. Only basic functionality and critical aspects should be tested in the pipeline, which may include functional cosmetic testing. Long cosmetic tests with many visual checks such as logos, videos, fonts, and images are difficult to maintain for ongoing scheduled tests.

Implementation of tests in the CI/CD

Cracking automated tests is difficult, even for the best testing experts. These tips will help teams set up CI/CD environments to run tests, prioritize testing efforts in their build process, and design timely testing strategies for their regression suites. Test stability and reliability, less duplication, speed and reduced maintenance are the critical criteria that define the success of continuous testing.

Band Created with Sketch.