Testing AI Enabled Platforms


“Testing AI systems presents a completely new set of challenges. While traditional application testing is deterministic, with a finite number of scenarios that can be defined in advance, AI systems require a limitless approach to testing,” said Ankur Chaudhry, Founder Next Generation Automation. “

There is huge need to create new capabilities for evaluating data and learning models, choosing algorithms, and monitoring for bias and ethical and regulatory compliance.

Experts in nearly every field are in a race to discover how to replicate brain functions – wholly or partially. In fact, by 2025, the value of the artificial intelligence (AI) market will surpass US $100 billion. For corporate organizations, investments in AI are made with the goal of amplifying the human potential, improving efficiency and optimizing processes. However, it is important to be aware that AI too is prone to error owing to its complexity. Let us first understand what makes AI systems different from traditional software systems:

Software systems

a) Features – Software is deterministic, i.e., it is pre-pro- grammed to provide a specific output based on a given set of inputs

b) Accuracy – Accurate software depends on the skill of the programmer and is deemed successful if it produces an output in accordance with its design

c) Programming – All software functions are designed based on if-then and for loops to convert input data to output data

d) Errors – When software encounters an error, remediation depends on human intelligence or a coded exit function

AI systems

a) Features – Artificial intelligence/machine learning (AI/ML) is non-deterministic, i.e., the algorithm can behave differently for different runs

b) Accuracy – Accuracy of AI algorithms depends on the training set and data inputs

c) Programming – Different input and output combinations are fed to the machine based on which it learns and defines the function

d) Errors – AI systems have self-healing capabilities whereby they resume operations after handling exceptions/errors

The above figure shows the sequential stages of AI algorithms. While each stage is necessary for successful AI programs, there are some typical failure points that exist within each stage. These must be carefully identified using the right testing technique as mentioned below:

Stage 1:

Learning Process from Data Sources

Points of Failures:

• Issues of correctness, completeness and appropriateness of source data quality and formatting

• Variety and velocity of dynamic data resulting in errors

• Heterogeneous data sources

How Testing Can be performed:

• Automated data quality checks

• Ability to handle heterogeneous data during comparison

• Data transformation testing

• Sampling and aggregate strategies

Stage 2:

Input data condition- ing – Big data stores and data lakes