This blog post is the first in a two-part series on website testing automation that can help your organization better understand how to maximize the effectiveness of your tests and find the right tools to meet your needs. Below we offer insights that can help your organization improve its testing automation process. Our follow-on blog post will help your organization understand the different software tools available to begin automating your tests.
At AEM, we are continuing to increase our use of test and task automation to create data-driven websites that allow us to focus on delivering new value for clients. Making the business case to pursue test automation is crucial to becoming a data-first organization, but the abundance of information and choices available regarding test automation can make it challenging to make the case for your organization.
We have identified five key insights so you can confidently move toward automation.
1. Consider Websites That are Prime Targets for Automation for a Pilot Program
When selecting a website to pilot automated testing, consider choosing a smaller site with features that represent what is typically available on your main site. Any kind of task automation is most helpful when you are manually performing the same steps repetitively. Conducting regression, cross-browser, cross-platform, and load testing (see Figure 1) are suitable for automation. Your first test should build off a current scenario instead of a completely new one to make it easier to set up, run, and understand.
Figure 1. Website Test Types
| Test Type | Test Features |
| Regression Tests | Tests for stable features include clicking throughout the front end of a website, entering data, checking contents and results to confirm the site and features are operating as expected. |
| Cross-Browser and Platform Tests | Runs the same tests created for regression but on different browsers and devices to ensure sites function as intended. |
| Load Tests | Simulates a predetermined number of users accessing the website or a specific feature simultaneously. |
2. Identify Technical Obstacles and Involve Technical Resources
As you are developing your business case for testing automation, it is essential to define boundaries for automation. A few items to discuss with your team are your organization's established preferences and policies for open source vs. COTS tools, programming languages, budget, staffing, and collaboration tools.
It will be vital to consult with your site hosting, security, and IT teams to understand how current technology policies could impact automated testing. Test developers will need administrative access to their machines to install and configure the software and, in some cases, for COTS products to create and run automated tests. Considerations for sites with two-factor authentication and tiered access include whether storing credentials in a test script for the login process is permitted, setting the length of the website session duration to run longer tests, and removing restrictions for simultaneous logins.
Over time, it is essential to continually discuss technical roadblocks with your internal teams. For example, engage with your developers to determine if website elements are blurred or masked after a site deployment because they may not be identified consistently during tests. Cross-browser testing can involve installing browser drivers and browser extensions that could require changes to your IT policy. Early discussions with your internal teams save time and money during the product selection process and can help make the evaluation process more manageable.
3. Determine the Team's Capabilities to Support Test Development
Utilizing new automated testing tools requires a few specific roles to set up, run, and analyze test results (see Figure 2).
Figure 2. Testing Roles
| Role | Responsibilities |
| Test Analyst | A Test Analyst understands the testing conditions and creates the test scenario. Test Analysts can be trained to edit automated tests, including identifying website elements and dynamic content and how to store and retrieve data used in testing. |
| Test Architect or Senior Test Engineer | A Test Architect or Senior Test Engineer has experience designing the test infrastructure using various tools and techniques. Depending on the organization's size, these individuals may also manage test teams, determine test strategies, and develop tests using a programming language. |
| Test Engineer | A Test Engineer understands and applies programming in the test steps. Test Engineers have experience with two or three code languages, including data programming. The Test Engineer creates test scripts using commands available within the software, and commands that use supported code. The Test Engineer prepares data for testing that may involve accessing data from warehouses or stores and data manipulation. Test Engineers develop tests to validate site elements and data that include coding to perform math, randomly select data, generate strings and dates, and other functions required for the test. |
4. Perform Your own Review of Products
Running an effective research process to review automated testing products is crucial to making the right choice for your organization. We have prepared a blog post dedicated to understanding the differences between testing automation software here. Before seriously jumping into sales conversations, put together a list of the 'must haves' and 'nice to haves' using a product matrix to organize your thinking. Every organization and team will have unique considerations and criteria that drive the best fit to meet those needs.
Depending on your team's capabilities and workload capacity for product review, white papers, like Gartner's Magic Quadrant or Peer Insights, can be consulted to build knowledge of available platforms. Engaging directly with product sales teams can be a time sink. We highly recommend assigning or contracting a technical lead to help recommend solutions. This individual can liaise with product reps, interpret technical information, break down costs and benefits, and coordinate the right discussions with the right teams.
When it comes time to evaluate products, set benchmarks for product selection by conducting some automated tests. You may use open-source software like Selenium IDE or opt to trial a product. You can use the results, including what you learn from the process, to shape your criteria.
5. Share Automated Tests With the Entire Product Team to Build Capacity
The benefits from learning how to use automated tests can apply to product managers, developers, and key users on the helpdesk or site support teams. Development teams can integrate automated testing into their continuous improvement framework to help reduce defects and improve overall site quality.
One way to ensure multiple teams can access and update tests over time is to establish a central and secure site to store tests. The test repository can also feature training resources, typically short, impactful videos covering the basic concepts of automated testing and short checklists for creating or editing tests. Live and recorded training and listening sessions also help build team capabilities.
Conducting internal training can help your organization take full advantage of automated testing. Starting with short one-on-one sessions with testers to understand their experiences with testing and how testing automation could apply to their responsibilities will help right-size training. Over time, shift beginning testers from using tests to editing tests through coaching and collaboration to advance skills and knowledge across the team.
AEM can help your organization implement the test and task automation and other best practices to keep your websites optimized. If you are interested in learning more, read our blog post on evaluating automation software products to explore the space further. Please reach out to our team to learn more about our service offerings.
