UI automation and selectors

What it does

UI elements are the saved references a desktop flow uses to find buttons, fields and windows at runtime. Each element holds one or more selectors — an ordered chain of attributes describing where the thing sits in the application's accessibility tree. Get the selectors right and the flow survives release cycles; get them wrong and it breaks the first time a developer renames a control.

Key facts

Elements come in two flavours. Desktop UI elements work with UI automation actions; web UI elements are captured from webpages and work only with browser automation actions. The pickers don't mix them.
Capture happens with Ctrl + left click in the UI element picker, or automatically during recording. Everything captured lands in the UI elements pane, where you can rename, find usages, and strip out unused elements.
Three capturing modes for desktop elements: UIA (default, and the right answer for WPF, WinForms and UWP), MSAA (legacy Win32 and VB6 apps that expose nothing else), and UIA3 Raw (the unfiltered tree, for Electron apps and custom-rendered controls where the refined view hides what you need). The pane shows which mode produced each element.
An element can carry several selectors. If the first fails, Power Automate tries the next in order. You can disable individual selectors while testing.
Text-based selectors match on the element's Name attribute for desktop and Text for web, with a configurable operator and support for variables. They're only available via the picker, not during recording, and only for elements that hold text themselves. SAP is the exception — it uses Text, though SAP generally behaves better on the default ID-based selectors.
UI elements collections are a premium feature: shared, reusable groups of elements across multiple flows.
Image-based automation and OCR (Tesseract) are the fallback when an application exposes no accessibility API at all. Image recording is a toggle in the recorder.

When to use / skip

Use UIA selectors by default and only drop to MSAA or UIA3 Raw when UIA genuinely can't see the control — each step down costs you reliability and speed. Save image-based automation for the applications that leave you no choice: Citrix-published apps, custom-rendered canvases, remote desktops. If a colleague is reaching for image recognition on a normal WinForms app, they've captured the wrong element rather than found a real limitation.

Configuration decisions

Capturing mode per application, decided once during discovery rather than argued about per element.
Whether to hand-edit selectors down to a stable subset of attributes, or accept the generated chain and add fallbacks.
Which elements belong in a shared UI elements collection versus staying local to a flow.
Naming convention for elements, because the auto-generated names are unusable once you have forty of them.
Tolerance values on image-based actions, which is the knob that decides between missed clicks and wrong clicks.

Gotchas

Capturing a web control through the UI automation picker gives you a desktop element with desktop selectors. It'll work in a demo and break when the browser updates. Use browser automation actions for anything inside a page.
Generated selectors often pin dynamic attributes — window titles containing a document name, indexes that shift when a list grows. Trim them by hand.
Screen resolution differences between the authoring machine and the unattended session change which elements are rendered and visible, so a selector that resolves in dev finds nothing in production.
Drag-and-drop on web elements is unreliable; the documented workaround is capturing a desktop element on the page and using the UI automation drag action, which is less reliable still.
Text-based selectors quietly break when the application is used in a different display language.

Consultant notes

Selector hygiene is the difference between an RPA estate that survives a year and one that needs a full-time babysitter. Make it a code-review item, not an afterthought.
Ask the client's application owners for their release calendar for every automated system. Selector breakage is predictable if you know when releases land.
Push back on automating anything that only works via image recognition unless the business case really carries the maintenance cost. Say so in writing.
Multiple selectors per element is the cheapest resilience you can buy. Add a text-based fallback to the elements that matter.

Worth revisiting after any major version bump of the automated applications, or when UIA3 Raw capture behaviour changes