Before we dive deep into architecture of Self healing automation lets first understand current state of manual maintenance
Current state of manual maintenance
Unfortunately, “fragile” is a word all-too-commonly associated with test automation scripts. When a script breaks, manual object identification maintenance can take up to 15 minutes per occurrence. A script breaks when object properties change, and an automation engineer must stop developing new scripts to troubleshoot and fix the broken one. The team manually inspects or spies the object to see the new property value or find new properties to use, then updates the script or object repository accordingly and reruns the script.
The math is daunting: One application deployment per week could encounter around 35 object changes (which varies greatly based on application maturity, development
methodology, size of project, etc.). At 15 minutes per manual fix, the result is more than one person’s full workday — 8.75 hours — spent per week on basic automation maintenance.
Self-healing automation
But it doesn’t have to be that way. Self-healing automation is a solution that addresses the No. 1 cause of test automation script maintenance: object changes. The “object,” in this context, is an item in a script — such as a button or text box on a webpage — that the script (or the user) would interact with to perform tasks. Scripts must be able to unique identify which object it needs to perform an action on — which text box should it put your
username into? Just as a person can be identified by physical attributes such as size, hair color or eye color — or by other relative means (“that person we saw at the store yesterday”) — objects must also be uniquely identified in some way. And, just as people’s appearances can change to the point that they aren’t recognizable to others, objects that no longer fit their original “description” can confuse traditional automation scripts. When
that happens, scripts break and downtime accumulates.
Self-healing employs data analytics to identify objects in a script even after they have changed. The result is a system that goes far beyond the “Band-Aid” approach often written
into scripts, such as the use of wildcards or regular expressions to handle variation in object names or identifiers. Rather than relying on those methods — and allowing productivity to grind to a halt anytime they fail — the self-healing approach introduces a higher level of intelligence and analysis.
When your script fails due to being unable to find the object it expected, the self-healing mechanism provides a fuller understanding and analysis of options. Rather than shutting down the process, it examines objects holistically, evaluates attributes and properties of all available objects and uses a weighted scoring system to select the one most similar to the one previously used. Self-healing can scrape, evaluate and choose among 10 objects in less than 0.05 seconds. Stopand-go syndrome is effectively cured.
The self-healing difference can be fully realized as it:
• Changes the mindset regarding automation approaches
• Allows automation efforts to start earlier as fears of maintenance subside
• Automates the maintenance process itself in real time
• Improves or preserves the return on investment
Lets dive deep now into architecture part of Self Healing Automation Framework developed by Next Generation Automation Academy experts for US Clients.
How self-healing works?
With self-healing automation, the same number of object changes per week mentioned earlier (35) — which could take more than a full workday to handle manually — can
be remediated in a fraction of a second. When an object’s properties change, the self-healing tool springs into action, scraping similar objects on the page and comparing them to the previously stored historical data on that object. Data analytics and custom property
weighting are deployed, enabling the tool to predict the most probable object being sought. The automation script continues running, using new object properties and an automatically updated object repository to head off future errors. The default weights assigned to each
property name can be modified.
Ultimately, empowered automation engineers will turn their attention to use cases that seemed too aspirational in the past. They can:
• Create generic scripts regardless of application or objects, resulting in much higher reusability
• Reduce the effort to update automation scripts to work after application upgrades
• Reuse more automation scripts based on out-of-the-box (OOTB) functionality
• Use the same scripts between environments
• Use the same scripts between clients/projects (important for consulting firms and our clients to accelerate testing by reducing initial ramp-up time for automation efforts!)
The self-healing process
The process is straightforward. When an object cannot be found (due to a property name or value change), the failed object is fetched from a historical object repository file, along with all its property names and values. All similar objects (such as all other text boxes) that do exist on the page are scraped, including all their properties and values and saved into an “Object Capture” table. Self-healing will use various similarity scoring algorithms to evaluate how similar each property is between the missing historical object and the available objects on the page. Each property is given a similarity score (while also considering customizable weightage so you could say that “name” is more useful to identify a match than “color,” for example) and ultimately each possible match is given a total score.
Self-healing will return the object and its properties with the highest score for use. The script can then attempt to identify the new object, continue on in execution and update framework repositories with the new object information, as long as everything works.
Some common questions which clients ask to Next Generation Automation related to Self Healing Automation?
How does it handle objects that are extremely similar, such as Address 1 and Address 2 fields?
The first thing to try is adding more properties to compare that may help differentiate these objects. In this example, they are both text boxes, have similar names and sizes and locations — but what is different about them? Perhaps considering which is mandatory would help correctly identify them, or more heavily weighting location.
How configurable can be Self Healing Automaiton Framework?
It is entirely configurable, either by using config files or by directly editing the code. In the configuration files, you can configure properties to scrape and determine their weight. In the code, you can change the similarity scoring mechanism, the input/output formats (JSON by default) and other mechanisms
Other tools claim to have self-healing capabilities. How is this different?
Other tools are proprietary and siloed — they must be purchased from the vendor, with everything done the vendor’s way. To use their “self-healing” mechanisms, you must totally transition to their tool, which is a significant effort for large organizations.
Many of these tools, while more modern in their approach and capabilities, are lacking the robustness needed by most large organizations. Next Generation Automation self-healing
utility can be integrated with existing tools, reused and customized. It provides a more direct, scalable solution that has the flexibility to work differently with other tools or applications.
Are there any known limitations or issues?
The primary application is for web-based applications, so further adaptation would need to be done to apply to other platforms (native mobile apps, desktop, etc.). In concept, this applies to those as well as robotic process automation (RPA), since they identify objects in similar ways for the most part (some tools support image cognition-based identification that would not be covered here). However, most RPA tools are proprietary and closed off and do not allow for integration of external utilities such as this.
Even if they allow custom code to be run, that code cannot manipulate their internal object repository information.
Why not use machine learning or other concepts for self-healing?
While it could be accomplished using machine learning, it introduces a greater level of complexity to the process. Machine learning (ML) requires large amounts of training data to be accurate and then is only accurate based on that training data. Different applications, projects and automation frameworks will have different quirks — or “features” as they are called in the data science world — that the ML model may not adapt well to. If a new property needs to be assigned to an object, machine learning requires a change to its model, new training data and a comparatively cumbersome procedure. The selfhealing
automation solution, by contrast, provides a lighter-weight, more direct approach that is eminently more controllable and flexible.
What if it doesn’t work?
While we’ve found that our self-healing mechanism is accurate more than 80% of the time, much depends on the application, the scripts and other factors. It’s flexible enough to add more properties to aid in comparison accuracy, as well as weight the importance of properties (such as determining that a similar name is more important than a similar color).
In a worst-case scenario, if self-healing identifies the wrong object, you are no worse off than before — you’ll have to perform maintenance manually for that one instance. So why not try it? We have not encountered this, but even if it worked only 25% of the time, that’s
25% less manual maintenance.
Building better QA for tomorrow