Skip to main content

Locators

A locator is an immutable, lazy description of how to find a control. It never talks to the backend when created or chained; every operation resolves it afresh against the live UI. Every locator starts from a window of the application: app.window(), its main window, and the dialogs that window owns, or another window (see Windows). This page is the guide; the exact signatures are in api.md.

Creating locators​

const main = app.window();
main.getByRole("button", { name: "Save" });        // semantic role + accessible name  ← preferred
main.getByRole("checkbox", { checked: true });      // ... and by state: checked, expanded, disabled
main.getByLabel("First name");                       // input-like controls by their label
main.getByText("Welcome");                           // any control whose name or value contains the text; a field by its value
main.getByAutomationId("btnSave");                   // exact AutomationId
main.getByClassName("OrderRow");                     // exact native or framework class name
main.getByTitle("Customer");                         // windows, dialogs, panes by title
main.getByOcr("Save");                               // text read off the screen: a ScreenLocator
main.read().getByLabel("Name");                      // the steps after read() may use what the screen shows
main.getByImage("save.png");                          // wherever this picture appears on screen: a ScreenLocator
main.getByVision("blue floppy disk save icon");      // experimental, needs a grounding provider; a ScreenLocator
main.locator('role=dialog[name="Customer"] >> label=First name'); // string grammar
main.locator({ type: "controlId", id });             // an id from snapshot()/controls(); used by agents

Text matching is a case-insensitive, whitespace-normalised substring by default; { exact: true } compares the whole string case-sensitively; a RegExp is tested as-is. { fuzzy: true } also accepts text a few characters off, with characters readers confuse (E/F/P, O/0, …) counting half, for text read off the screen: getByOcr("Plugins", { fuzzy: true }) (or getByText under read()) finds Elugins. See resolution.md. Roles are ARIA-like (button, textbox, checkbox, menuitem, listitem, dialog, …; the full list is DESKTOP_ROLES). dialog also matches plain windows because UI Automation does not distinguish them, and table also matches a grid, since the two hold the same rows and cells.

Chaining and scoping​

const main = app.window();
const dialog = main.getByRole("dialog", { name: "Customer" });
await dialog.getByLabel("First name").fill("Alex");
await dialog.getByRole("button", { name: "Save" }).click();

Each step resolves within the controls matched by the previous step. Chaining never changes the parent locator.

Two locators also combine. and keeps the controls matched so far that another locator matches too; or matches either, in document order, each control once:

const main = app.window();
await main.getByRole("button").and(main.getByText("Save")).click();        // a role and a text at once
const outcome = main.getByRole("dialog", { name: "Saved" }).or(main.getByRole("dialog", { name: "Error" }));
await outcome.waitFor();                                                  // whichever opens

The other locator is one of the same application. Its whole chain is matched over the same observation, from the window it starts from, as a relative anchor is, never inside the controls matched so far. Both narrow and never descend, like filter; chain a getBy* afterwards to go inside. An or of two things stays strict: when both are there, an operation that needs one control fails with StrictModeViolationError.

Windows​

Every locator starts from a window. An application is its windows; a window is the scope its locators look in, and the window its own operations act on. A window is a scope you hold, never a mode the application is switched into.

// The main window: the one no window of the application owns
const main = app.window();
await main.getByRole("button", { name: "Open…" }).click();

// A dialog the main window owns is part of it: no window of its own to name
await main.getByRole("dialog", { name: "Open report" }).getByRole("button", { name: "OK" }).click();

// A window of its own (a second document, a tool another process shows): by its title, found again whenever it is used
const findCustomer = app.window({ title: "Find customer" });
await findCustomer.getByLabel("City").fill("Delft");

// By its class: Windows' own dialog whatever its title's language, a window without a title
await app.window({ className: "#32770" }).getByRole("button", { name: "OK" }).click();
const splash = await app.waitForWindow({ className: /^WindowsForms10\.Window\./ });

// Which windows are open: each a Window held by its handle, with its title, bounds, ...
for (const window of await app.windows()) console.log(window.handle, window.title);

// Three windows alike: each is a window of its own, no position needed
const [first, second, third] = await app.windows();
await second.getByRole("button", { name: "Save" }).click();
await app.window({ title: "Order" }).nth(1).getByRole("button", { name: "Save" }).click(); // the second titled "Order"

// The window an action opens
const report = await app.waitForWindow(() => main.getByRole("button", { name: "Print preview" }).click());
await report.bringToFront();
await report.close();
  • What a window holds: the window, every window it owns however deep (its dialogs and message boxes, a credential prompt another process shows for it), and the menus, drop-downs and tooltips open over them: those are windows of their own without a title, or reported as a menu, lying over the window.
  • The main window is the one no window of the application owns. A window another process shows of its own (an updater's) is a window of its own, named by its title, while the application's own process shows one. When several are open (a second document window, a second instance), app.window() is an ambiguity: a wait settles it, as a dialog replaced by its successor does, and then fails with StrictModeViolationError, naming each window and how to name it. A forms application whose forms are owned by a window Windows does not show (Delphi before MainFormOnTaskbar) has those forms as one scope. An application that shows no window a user names (only menus, or windows without a title) has every window as its scope.
  • Held or found again: app.windows() and app.waitForWindow() give a Window held by its handle: it keeps meaning that one window, whatever its title becomes, and once it closed, what is asked of it fails rather than finding another window. app.window() and app.window({ title }) find their window again each time, through a splash screen handing over to the main window and a window opened again.
  • A narrowing step right after a window picks among the windows: app.window({ title: "Order" }).nth(1), .filter({ has }). The steps after it look into the window kept.
  • The window itself: a Window is a locator of the window: close(), minimize(), maximize(), restore() and bringToFront() act on it; isOpen() says whether it is open; screenshot() pictures it alone; currentTitle() reads its title now, and expectDesk(window).toHaveTitle(/modified/) waits for one.
  • Real input: window.keyboard brings the window to the front first, unless it or a dialog it owns is there already. window.mouse takes points relative to the window's top-left corner, the pixels of its screenshot.
  • Combined locators: an and, or or relative anchor from a window is matched over every window of the application, then kept to its own window's; filter({ has }) looks inside each control, whatever window the has locator was made from.
  • Recorded tests name each window once, the main one too: const main = app.window();, const findCustomer = app.window({ title: "Find customer" });.
  • The application gives one Window per open window, so the same window is the same object: === and Map keys work, and its title, bounds and the rest are those of the latest listing.

Windows that open by themselves​

An error box, a confirmation or a tip of the day opens when it likes. A handler deals with it where the test waits anyway:

const main = app.window();
const off = await app.onWindow({ title: /^Error/ }, async (dialog) => {
  await dialog.getByRole("button", { name: "OK" }).click();
});
await main.getByRole("button", { name: "Import" }).click(); // an error box that opened is answered first
off();

At the start of every attempt of an action, a wait or an assertion, each handler runs once for every window that opened since it was added and whose title matches. Nothing runs in the background, so a handler never interleaves with the test's steps. A window without a title (a menu, a drop-down) is never handled. While handlers are added, each attempt lists the application's windows once.

Title bars​

What a window's title bar holds — Minimize, Maximize/Restore, Close and the System menu — belongs to the window, not to the application, and is matched only by a locator that scopes into the title bar. So a dialog's own "Close" button is never confused with its window's:

const main = app.window();
await main.getByRole("dialog", { name: "Tip of the Day" }).getByRole("button", { name: "Close" }).click(); // the dialog's button
await main.getByTitle("Tip of the Day").close();                                                        // the window itself
await main.getByRole("titlebar").getByRole("button", { name: "Close" }).click();                         // the title bar, explicitly

Operate windows with close(), minimize(), maximize() and restore() on a window locator (see actions.md); app.window().currentTitle() reads the main window's title. A locator that finds nothing but would have matched a title-bar control says so in its error.

Several matches: strict mode and positions​

An operation that needs one control fails with StrictModeViolationError when several controls match. Transient duplicates are waited out: a dialog replaced by its successor overlaps it for most of a second, a list refreshes. Once the same controls have matched for about a second (three attempts in a row spanning at least a second), the ambiguity is reported, about two seconds after the first sighting with the default intervals rather than at the timeout. The failure lists every candidate, and says what may narrow the locator:

getByRole("button", { name: "OK" }) resolved to 2 controls.
  1) button "OK" [automationId=okBasic] hidden
  2) button "OK" [automationId=okAdvanced] (615,591 113x33)
Hint: 1 of them is hidden: .filter({ visible: true }) keeps the one shown
Hint: to pick 1): getByAutomationId("okBasic")
Hint: to pick 2): getByAutomationId("okAdvanced")

Make the locator more specific first: exact, a role, a scope, or filter({ visible: true }) for a control a hidden tab page repeats. For deliberately repeated controls pick a position:

const main = app.window();
await main.getByRole("listitem").first().click();
await main.getByRole("tab").nth(2).click();          // 0-based; negative counts from the end
await main.getByRole("row").last().getByRole("button", { name: "Delete" }).click();

nth/first/last are evaluated at resolution time in document order, like every other step, and later steps scope into the chosen control.

Refining matches: filter and states​

Repeated controls usually differ by what they contain, not by position. filter keeps the matches whose subtree has (or lacks) a text, or a match of another locator, and getByRole can ask for a state:

const main = app.window();
const order = main.getByRole("listitem").filter({ hasText: "Order 42" });
await order.getByRole("button", { name: "Delete" }).click();

await main.getByRole("row").filter({ has: main.getByRole("checkbox", { checked: true }) }).count();  // selected rows
await main.getByRole("row").filter({ hasNotText: "Shipped" }).allTextContents();
await main.getByRole("combobox", { expanded: false }).first().expand();
await main.getByRole("button", { name: "Save", disabled: false }).click();
await main.getByRole("button", { name: "OK" }).filter({ visible: true }).click();                  // not the hidden tab page's copy

A grid cell can also be asked for by its column, the text of the header it sits under:

const main = app.window();
const orders = main.getByRole("grid", { name: "Orders" });
await orders.getByRole("row").filter({ hasText: "Customer 42" }).getByRole("cell", { column: "Email" }).textContent();

hasText/hasNotText take a string (case-insensitive substring; exact: true for a whole-string comparison) or a RegExp and look at every name and value in the subtree, the control's own included. has/hasNot take a locator that is resolved inside each control. Like nth, filter narrows the current matches and never descends; chain a getBy* afterwards to go inside. checked, expanded and disabled only match controls that report that state, so getByRole("button", { checked: false }) matches nothing. visible: true keeps the controls the backend reports visible, visible: false the hidden ones. A hidden tab page keeps its controls in the tree, and getByRole keeps matching them, so that an action can scroll a virtualised list's off-screen row into view; which copy a test means is its own to say.

Tables: rows by their cells, cells by their column​

The same, in the words a test uses:

const main = app.window();
const row = await main.table("Orders").row().where("Customer").equals("Customer 42").and("Status").equals("Open").find();
await expectDesk(row.getByColumn("Email")).toHaveText("customer42@example.com");
await row.getByColumn("Actions").getByRole("button", { name: "Edit" }).click();

app.window().table(name) is the grid or table with that name (app.window().table({ root, rows, cells, columns }) lays one out from locators where no grid is reported); row().where(column) narrows its rows by what the cell under a column holds (equals, contains, matches); find() waits for exactly one row, paging a virtualised grid to it, and returns it as a locator; all() and count() observe once. getByColumn(column) is the cell under the header that reads as the whole column name, on a row, a grid or any locator. It all compiles to the locators above, so nothing resolves differently. app.window().list(name) does the same for a list's items: app.window().list("Templates").item().named("Template 12").find(). See tables.md.

Web pages​

A page an application shows (an Electron, CEF or WebView2 page, or one in Internet Explorer's WebBrowser control) is found by its address and its title, and what is inside it is found as anything else is:

const main = app.window();
const orders = main.webPage({ url: /orders\.html$/ });
await orders.getByLabel("Customer").fill("Ada");
await orders.getByRole("button", { name: "Save" }).click();
await expectDesk(orders).toHaveUrl(/saved\.html$/);
console.log(await orders.currentTitle());
  • When to use it. A page's controls are found from the window too. Scope to the page where a name repeats in the application around it (a host's own "Save" beside the page's), or where a window shows several pages.
  • Which page. webPage() takes the outermost page inside the locator that matches. A page's frames (<iframe>, a frameset's <frame>, a frame's frames), and the webviews it shows, are pages inside it: page.webPage() finds them, and so does an address only they have. Several pages that match are an ambiguity, as while an application's page shows in a dialog of its own: webPage({ url: /modal\.html$/ }) names one.
  • Frames are inside their page. What a frame shows is found from the page that frames it as from the window: a "Save" in the page and one in a frame make page.getByRole("button", { name: "Save" }) strict, and page.webPage({ url: /orders\.html$/ }) is the frame's alone.
  • Addresses. Each engine writes them its own way: file:///C:/app/orders.html from Chromium, file://C:\app\orders.html from Internet Explorer, about:blank for a page written from a string. Match on what each says, the file name, as above.
  • What a page adds. currentUrl(), currentTitle(), and expectDesk(page).toHaveUrl(…), which also waits for a navigation. What is found inside it is a plain locator.
  • Headings and landmarks. A page's headings are heading, found by their level too, and its landmarks have their ARIA roles: banner, navigation, main, complementary, contentinfo, form, region, search. getByRole("text") still finds a heading, and getByRole("group") a landmark. Chromium reports them all; Internet Explorer only what an ARIA role names, with no level.
const page = main.webPage();
await page.getByRole("navigation", { name: "Main" }).getByRole("link", { name: "Orders" }).click();
await expectDesk(page.getByRole("heading", { level: 1 })).toHaveText("Orders");

Trees​

A tree item by its path, from the tree down:

const main = app.window();
const active = main.tree("Folders").node("Customers").node("Active");
await active.click();                       // opens "Customers" first when it is collapsed
console.log(active.describe());             // tree("Folders").node("Customers").node("Active")
const folders = main.tree({ root: main.getByAutomationId("folderTree") });
  • A node is a locator: every action, assertion and node() one level down. Its name is the whole text, case-sensitive ({ exact: false } for a substring), and it is strict at each level.
  • The items of one depth are getByRole("treeitem", { level: 1 }) (role=treeitem[level=1]), from 1 at the top, as the tree reports it; a tree grid's rows take a level too.
  • Each level is a tree item with that exact name that no other tree item holds below where the step starts ([outermost=true] in the locator grammar; getByRole takes no such option). A level then holds its parent's own children, not theirs, whether the tree nests its items directly (WPF, Win32) or in groups (the web), and two items of one name under two parents are told apart.
  • An action on a node opens a collapsed item along the path first, once per wait: through its expand pattern, or, for an item that reports no state (Avalonia), its expander button. The record says expanded tree item "Customers". Resolving alone (count(), all()) opens nothing.
  • A tree that marks its levels only by aria-level, with every item a sibling of the others, is not nested for a path: write such items by name alone.
  • Studio and the agent offer the path as a candidate, and a recorded test writes it as app.window().tree(…).node(…).

Locating by position​

Some controls have no name, no id and no text of their own — a form field is identified only by the label beside it, a button only by the icon on it. A relative step narrows the matches to those sitting in a given direction from an anchor, nearest first:

const main = app.window();
await main.getByRole("textbox").rightOf(main.getByText("Customer number")).first().fill("42");
await main.getByRole("button").below(main.getByText("Details")).first().click();
await main.getByRole("checkbox").near(main.getByText("Include archived"), { maxDistance: 30 }).first().check();
  • rightOf, leftOf, above, below require the two to share a row or a column, which is what makes "the field to the right of this label" mean that and not "anything further right on the screen".
  • near reaches in every direction, within maxDistance pixels (default 50).
  • Matches come back ordered by distance from the anchor, so .first() is the nearest one. Like nth and filter, a relative step narrows what came before and never descends into it.
  • The anchor may match several controls; a control counts if it sits in the right place relative to any of them. The anchor itself is never a match.

The anchor and the target need not have been seen by the same thing: a label read off the screen (getByOcr("Name")) or a picture (getByImage("key.png")) can position a field that only the accessibility tree knows about. A chain may name one way of looking at the screen: an anchor found by a picture cannot position a grounded description, nor can read text position a picture, because each of those resolvers sees only what it found itself; such a chain is refused as it is written.

Finding a control by a picture of it​

const main = app.window();
await main.getByImage("save.png").click();                    // a file in the configured image folder
await main.getByImage(bytes).click();                         // or the bytes of one
await main.getByImage("save.png", { threshold: 0.95 }).click();
await main.getByImage("theme-icon.png", { colour: false }).click(); // its accent follows the theme: match in grey alone

// Scoped to a parent: only the copy inside that control matches.
await main.getByRole("dialog", { name: "Export" }).getByImage("save.png").click();

A picture is matched in grey, which keeps it found across themes and is blind to hue. A picture that is itself coloured is also compared in colour where it matched: a status lamp drawn green is told from one drawn red, and an indicator from another of the same shape. A grey picture matches in grey; { colour: false } turns the colour check off.

Pictures are kept by name in the image folder, one folder per application and screen: getByImage("crm/customer-dialog/save") finds tests/images/crm/customer-dialog/save.png; .png may be left off. Studio's Pictures tool takes, checks and saves them, and writes a typed index beside them, so a test names a picture without spelling its path (studio.md):

import { images } from "../images/index.mjs";   // written by Studio, or by `npx spart-desk images`
const main = app.window();

await main.getByImage(images.crm.main.save).click();

// A page object takes its screen's branch once.
const pics = images.crm.customerDialog;
await dialog.getByImage(pics.ok).click();

On its own the locator looks at every window of the application, each in a picture of its own, front first: a picture of something in a dialog is found in the dialog, and a match that a window in front covers is not taken. That is what makes it useful for a window the tree cannot describe. Give it a parent and the picture must lie inside that control: the engine observes the tree, places each detection in the deepest control whose bounds contain it, and matches from there. The same holds for getByVision. A scoped locator is the way to tell two identical icons apart when only their surroundings differ. A picture holds nothing: getByImage returns a ScreenLocator (api.md), on which no step locates inside (getByImage("x").getByText("y") does not compile); a control beside it is found by measuring from it, with rightOf, below or near.

The picture is looked for at the size each window is drawn at: a template captured at 100% is rescaled for a 150% display, using images.scale and the window's own DPI. Sizes are never swept blindly — pass scales to look at particular multiples. Matching is correlation on brightness, so a control that lights up on hover still matches a template captured when it was cold, and two icons differ only if their shape does.

When the accessibility tree knows about something at the place the picture was found, the action goes through it: an icon-only toolbar button is usually a real button with no name, and invoking it beats clicking its coordinates.

Counting and iterating​

Every action and assertion targets one control. To ask about several, count(), all() and allTextContents() observe once (no waiting, never strict) and expectDesk(locator).toHaveCount(n) waits for a number:

const main = app.window();
await expectDesk(main.getByRole("listitem")).toHaveCount(3);
for (const row of await main.getByRole("listitem").all()) {
  console.log(await row.textContent());
}
const names = await main.getByRole("listitem").allTextContents(); // value, else name, from one observation

Reading a state​

isEnabled(), isChecked() and isFocused() wait for the control to exist, then answer from that observation; to wait for a state, assert with expectDesk(locator).toBeEnabled(), toBeChecked() or toBeFocused(). A control that reports no such state (no toggle state; a backend that does not read the focus with the tree) is refused with BackendError not_supported rather than answered false. isVisible() does not wait.

String grammar​

role=button                       role=button[name=Save]     role=button[name="Save"]  (quoted = exact)
role=checkbox[name=Advanced][checked=true]      role=combobox[expanded=false]      role=button[disabled=true]
role=heading[level=2]             role=navigation[name=Main]  (a web page's headings and landmarks)
role=cell[column=Email]           role=cell[column="Email"][name="ada@example.com"]  (name and column are quoted alike)
text=Welcome  label=First name  title=Customer  automationId=btnSave  id=btnSave  className=TStringGrid
ocr=Save                          ocr="Save"                 (text read off the screen; quoted = exact)
read >> label=First name          (the steps after read may use what the screen shows)
vision=blue floppy disk save icon
image=save.png
controlId=3935002:103
role=listitem >> nth=-1
role=listitem >> hasText=Order 42 >> role=button[name=Delete]     (hasText / hasNotText narrow like nth)
role=dialog[name="Customer"] >> label=First name
role=tree[name=Folders] >> role=treeitem[name="Customers"][outermost=true] >> role=treeitem[name="Active"][outermost=true]
role=button[name=">> next"]       text=Don't save      (a quoted value may hold >>; a bare value keeps its apostrophe)

>> splits steps outside quoted values only. A quote opens a value right after =, so role=button[name=">> next"] is one step and text=Don't save keeps its apostrophe; an unterminated quote is a LocatorSyntaxError. has, hasNot, and, or and the relative locators have no string form; use the methods on a locator.

Finding the right locator​

  • app.snapshot() returns the window and control tree the backend sees, as roots and as the flat list controls. The MCP tool desktop_snapshot renders the same information one line per control.
  • Prefer, in this order: role + name, label, AutomationId, title, text. Names are what users see and what UI Automation exposes most reliably; text matches anything and is the most ambiguous.
  • candidatesFor(snapshot.roots, control.id, …) derives that choice for you: every locator that addresses the control, ranked, with recommended the best unique one (role + name, exact: true when the substring form is ambiguous, AutomationId, text for nameless controls with a value, the role alone), scoped under the nearest uniquely addressable ancestor when needed and positioned with nth as a last resort. recommendLocators(roots) does it for a whole tree at once; the agent layer uses it to annotate snapshots and to export tests (api.md).
  • Electron, WPF, WinForms, Win32 and WinUI 3 applications expose UI Automation (WPF and Windows Forms specifics: dotnet-apps.md; which kinds of application are covered how well: app-coverage.md). Custom-drawn controls (canvases, game UIs, some Java/Qt apps) may not — that is what read() and getByOcr are for.
  • A virtualised list or grid (WPF) holds only the rows on screen. Scope a locator into it (grid.getByRole("row").filter(...)) and an action pages it to the row; scrollIntoView() does the same on request.
  • A resolution reads each window of the application whole, so a chained locator costs no more than a root one. Scope with a parent locator to cut ambiguity, not time.

Reading the screen on request​

Every locator above is answered by the accessibility tree alone. Where the tree reports nothing, a locator says so: getByOcr("Save") finds text read off the screen, which on Windows needs nothing installed, and read() lets the steps after it use what reading contributes — a button named by the text drawn inside it, a field named by the label painted beside it (main.read().getByLabel("Name")), a grid laid out from what was read (main.read().getByRole("cell", { name: "C2" })). A table given a root is read when its root is: main.table({ root: pane.read() }). getByImage finds a control from a picture of it, and getByVision from a description of how it looks, through a grounding provider you supply. Details: resolution.md.

Which to write, in this order:

  • The tree reports the control (Studio's Inspector shows what it knows): a plain getBy*. Nothing is captured or read.
  • The tree reports the control without its name, or not at all, and you want it as a control: a field to fill, a check box to check, a combo box to selectOption, a cell or a list item to pick, a value to assert with toHaveValue. Put read() on the smallest container that holds it and write the same getBy* you would for an application that reports everything. Reading names a field after the label painted beside it, recognises painted widgets and their states, and lays out painted grids. A Delphi VCL form, a Tk window, a Krypton toolbar and a status bar that only paints its message are found this way; the result is a DesktopLocator.
  • You only need text that is painted somewhere: to click where it is, to read a value (textContent of a pattern gives the whole line), to wait for it or for it to go. getByOcr is lighter than read() (the line itself, no widgets recognised) and gives a ScreenLocator: no role, no state; an action at its place goes through the control the tree has there, if any.
  • Nothing is painted as text: getByImage from a picture, getByVision from a description.

read() and getByOcr each capture the window and read it on every look, so scope them to the container that holds what you want (dialog.read().getByLabel("Name") reads the dialog's area only).

Inspecting a locator​

locator.describe() / String(locator) render it as written (getByRole("dialog") >> getByRole("button", { name: "Save" }).first()); a combined locator renders the other one inside its step (getByRole("button").and(getByText("Save"))). locator.query is the declarative step list and locator.target the application it belongs to. Both appear in records, traces and errors.