This RPA was built differently. Typical RPAs use the “selector” technique to identify GUI elements and operate on them. Ours rather employed Computer Vision to understand the GUI elements. This resulted in a better developer experience because some of the GUI elements in back office software weren’t operable with the selector technique. But with Computer Vision we were able to operate just like a human agent would.
To interpret the language of the user, we used a mix of Google Dialogflow and our own proprietary NLP AI. Google Dialogflow’s ready-made AI helped us prototype faster, and our proprietary NLP AI helped us interpret some curve-ball natural language inputs where