Perils of Record and Playback Test Automation

One prevalent practice since the early days of test automation has been the concept of record and playback. This process is marketed by automation tool vendors (against controlled demo environments) as an effective means to achieve “quick & magical” results for automating the application-under-test. But this approach is not scalable to any real-life, non-trivial applications and ultimately leads to failure of automation as a whole – lack of reliability, questionable ROI, maintenance complexity, etc.

Record/playback gets you started on a wrong foot by thinking about test script development before you had a chance to properly design the automation suite. Imagine your application development team develops code first and then takes up design as a post-processing step. It is easy to see how impractical that would be. Test Automation is no different!

Lack of modularity and reusability

Recorded scripts are structured as a linear collection of statements. No contextual division or modularity is applied since the statements are purely dependent on the user’s keyboard and mouse interactions during the recording. Parameterizing and modularity of test assets are a basic necessity for long term maintainability. Any post-processing effort is not only time consuming, but also largely ineffective since the user has to track back his actions to understand the context boundaries and apply.

Recorded scripts are brittle

Scripts generated by a recording tool are very sensitive to user actions during the recording session. Recording of unrelated and non-repeatable actions that occurred during the recording process, like focusing or closing an external application, clicking on unrelated elements, etc. create a lot of noise in the output.

Recorded scripts capture the current state of UI elements, such as the value of input fields, selected values in drop-down, radio option selection, etc. This strong coupling of element state to the scripts could lead to failures during execution.

Element Identification is unstable

During the recording process, UI elements interact in the application are automatically added to the generated scripts and the element repository. Depending on the application UI framework, the element identification criterion may be ill-chosen and limited to a set of predefined attributes. For current-day application UI, conventional ID/name (or other attribute-based) mode of element definition is far from adequate. Record/Playback approach denies the opportunity to thoroughly analyze the element before being exercised.

Logging and error recovery is not embedded

Recorded scripts only contain linear navigation and data entry steps. No error state validations, logging or graceful recovery is recorded. Scripts may fail with cryptic messages and debugging the error is extremely difficult. This is one of the major reasons for the lack of reliability of automated scripts.

Advanced interactions and business rule validation are hard to achieve

Business validations and advanced interactions tend to be complex in nature. Recorded scripts ether ignore these validations or duplicate this logic in several places. This leads to code redundancy and maintenance overheads.

A fresh perspective

Given the issues, even the short-term gains of record/play-back is misleading and fails to provide any viable ROI. This approach may seem compelling when you are challenged with resource availability and lack the bandwidth to create and maintain well-designed automation suite. But contrary to this belief, creating a modular automation suite need not be complex or effort-intensive – provided proper tools and practices are applied.

accelQ takes a unique ‘design-first’ approach for a sustainable test asset development. The universe presents a cohesive picture of basic building blocks of test assets and establishes natural traceability. Predictive Scenario Designer promotes modular thinking. Natural language-based test asset development makes it easy and quick to build verification logic. Peripheral concerns such as logging, recovery etc. are baked into the overall test development process without any user effort.

Our customers have gained over 3x acceleration compared to conventional tools and approaches in initial test development as well as ongoing maintenance. When you can get both, you don’t have to choose between the speed and stability !