The basics are pretty straightforward, click this, enter data into that… There is more often than not a reliable strategy to find a UI element and interact with it.
So… being a little controversial here… do we really need AI to find a search Icon and click it? I’m not sure we do, it really isn’t that difficult to achieve with what we have already. Baked in Testability eliminates the need for most of the current advancements in AI. Unique ids anyone?
Moreover, most websites and apps “tag” particular fields with a type identifier so that the browser can assist us in entering common data like user names, passwords, credit cards numbers etc. You know when your browser says, “let me enter that password for you”, It’s just a tagged field being recognized by a browser and it offers you that option.
Why not just work backwards on the problem to the source, get Apps to tell us a bit more about themselves?! “Hey I’m a password field”, “Hey I’m a shopping cart button”. Remove the uncertainty, it’s not rocket science. Maybe advances in a universal set of semantic tags, rather than training ML systems to recognize imagery might be a better path. IMHO.
There are however some areas in test automation that are difficult and really could benefit from a little AI uplift. So, what are the 2 main culprits…
Public Enemy No.1 – Uncertain Data Sets
The Automation Engineers worst nightmare, the unknown. We’re going to set our automated test script off against an application backed up by an uncertain set of baseline data. We need to make sure that script reacts appropriately and achieves its outcome regardless.
It’s like we’ve asked the autopilot to get us to LAX and check the text on the welcome sign, but there may be storms, wind-shear, drones, sick passengers, volcanoes! who knows? Well there are 7 million lines of code in the autopilot on a humble 787, no AI, just algorithms designed to deal with eventualities to get us there, and MOST IMPORTANTLY we still have 2 human pilots to back it up.
So… a little “AI” sifting of data sets would be awesome, to help our automated test understand the landscape it has to deal with. Questions like, “What account has a balance with more than $100”, “which user booked a flight and cancelled it less than 24hrs before today”. A kind of data mining challenge.. But of course what if there isn’t a matching data item? What then? Could it create one for us?
If AI can turn a question into a SQL query, return us a value and help us stage test data, both for manual and automated tests, it will really be of value. Again IMHO.
Complex Data Presentation
Actually pretty easy to deal with folks, with a fixed set of Test Data, If I search for X I get Y. It’s always number 1 on my list of results. Hey! no need to write complex code to pick Y out of a list of stuff. I can reliable say it’s always going to be in position 1.
But the bulk of projects work with uncertain data sets. So, based on this, wouldn’t it be great if we could get AI to pick the item we want from a list of items. Select the item in the list that matches our criteria? “Search for Sheds”, Click “Add to Cart” for the Shed named “ShedXL” or select “Payment Plan” for the “Medium shed deluxe”.
The challenge isn’t about identifying a Shed Icon, it’s about finding the one we want that meets our test criterion.
So… AI is here, but it needs to augment, not try and replace what we can currently do. I love it but am underwhelmed by recent progress purely because it solves a problem we don’t currently face, or at least one which has a simpler answer. But, keep on trucking guys, hit the data problem not the navigation problem and then we’ll be cooking on gas!!