Java applications

SPART-Desk tests Java desktop applications with the same locators, actions and assertions as any other Windows application. This page says how a Java application is recognised and launched with the Java Access Bridge, how its controls read, and what acting on them needs.

Toolkit	How SPART-Desk sees it
Swing, AWT	Through the Java Access Bridge, next to UI Automation. UI Automation alone sees only the window frame.
JavaFX	Through UI Automation, which JavaFX implements itself.
SWT (Eclipse)	Through UI Automation: SWT uses native Windows controls.

import { Desktop, expectDesk } from "@spartaficial/spart-desk";

await using desktop = await Desktop.create();
const app = await desktop.launch({ executable: "C:/Apps/Orders/orders.exe" });
const main = app.window();

await main.getByLabel("Customer").fill("Ada");
await main.getByRole("checkbox", { name: "Express" }).check();
await main.getByRole("button", { name: "Save" }).click();
await expectDesk(main.getByText("Order saved")).toBeVisible();

Nothing Java-specific is needed in the common case: SPART-Desk recognises the application and turns the bridge on for it.

Launching

A Java Virtual Machine exposes its Swing/AWT controls only when told to load the Java Access Bridge. SPART-Desk tells it by adding -Djavax.accessibility.assistive_technologies=com.sun.java.accessibility.AccessBridge to JAVA_TOOL_OPTIONS in the environment of the process it starts — never in a user or system file, and on top of whatever JAVA_TOOL_OPTIONS already held.

It does so when it can see that the Java runtime that will run the application has the bridge (the jdk.accessibility module; every full JDK and JRE has it):

You launch	The runtime checked
java.exe / javaw.exe	That runtime (links such as Oracle's javapath are followed).
app.jar	JAVA_HOME's javaw.exe, else the one on PATH; the jar runs as javaw -jar app.jar.
A launcher with a bundled runtime (jpackage runtime\, launch4j/install4j jre\)	The bundled one.
A recognised launcher stub (launch4j, exe4j, install4j, jpackage), or any executable with target: { processName: "javaw" }	JAVA_HOME's, else PATH's.

The check matters: a runtime without the bridge refuses to start AWT when told to load it (AWTError: Assistive Technology not found). Such a runtime is left alone, and its windows show only their frame.

Say what the application is when SPART-Desk cannot tell:

// A Java application behind a launcher SPART-Desk does not recognise.
const app = await desktop.launch({ executable: "C:/Apps/Legacy/start.exe", java: true });

// Not a Java application, or a Java one you want treated as a plain window.
const tool = await desktop.launch({ executable: "C:/Tools/tool.exe", java: false });

With java: true the bridge option is added whatever the runtime, and the launch fails with LaunchError unless the application's Java window can be read. With java: false nothing is changed.

Attaching (desktop.attach(...)) to a Java application SPART-Desk did not start works only if its JVM was started with the bridge: jabswitch -enable in that Java runtime turns it on for every Java application of the user (and makes runtimes without the bridge refuse to start AWT). Otherwise the window shows only its frame, and the snapshot says why.

In Studio (studio.md) the same holds: an application started from the Inspector or the Recorder (--launch, Start application…) has the bridge on and shows its inside; one started elsewhere without the bridge raises a banner that says so, with the two ways out. The Recorder records a click on the Java control the bridge finds under the press, and keys on the control the bridge says has the focus.

Finding controls

Swing components read like their Windows counterparts:

Swing	getByRole
JButton, JToggleButton	button
JCheckBox / JRadioButton	checkbox / radio
JTextField, JPasswordField	textbox
JTextArea, JEditorPane	document
JComboBox	combobox: chosen in through its own selection by selectOption() and fill(), and by a click on an item of its open popup; its value is the selected item
JMenuBar / JMenu, JMenuItem / open JPopupMenu	menubar / menuitem / menu
JTabbedPane / its tabs	tablist / tab
JList / items, JTree / nodes	list / listitem, tree / treeitem
JTable / cells	table / cell
JLabel	text
JToolBar, status bars	toolbar, status

• Labels. A field whose label was attached with JLabel.setLabelFor is named by it: getByLabel("Customer"). A field with no label tied to it takes the text beside it or above it, with nothing between, as in a Windows window.
• States. A tab, list item or tree node says whether it is selected (isSelected()), from the bridge's selected and selectable states; a control that can take the focus says whether it has it (toBeFocused()). A button's, check box's or radio button's text is its name.
• Icon-only buttons are named by their tooltip: getByRole("button", { name: "Save" }) finds a tool bar button whose tooltip reads "Save (Ctrl+S)".
• Only what is showing is in the tree: the items of a closed menu or the content of another tab appear once they are shown.
• Owned dialogs (a Save dialog, a JOptionPane) appear inside the window that owns them, with their Swing content: app.window().getByRole("window", { name: "Save" }).getByLabel("File name"). A dialog's own "Close" button is dialog.getByRole("button", { name: "Close" }); the window itself closes with app.window().getByTitle("Save").close().

Acting

Operation	How
click()	A real mouse click at the control's centre when the window is in the foreground, like a user. When it is not, the control's own action through the bridge (strategy java.action).
click({ button: "right" }), dblclick(), click({ modifiers }), click({ position })	Real input only: the bridge has no right or double click, and holds no keys. The window is brought to the front for it; with foreground: "require" it fails with foreground_required, and a control with no place on the screen fails with not_supported.
click() on a tab, list item, tree node or table cell	Selects it (java.select); no foreground needed.
expand() / collapse()	The node's own action (a tree node toggles).
check() / uncheck()	A click, as above, until the state is the wanted one.
fill()	Sets the text through the bridge (java.set-text); no foreground needed. A slider is moved by its keys instead, which the bridge cannot set: focused through the bridge, then Home or End for an end of its range, else the arrow keys a step at a time, its value read back between rounds (real input, so the window comes forward). A JSpinner, which never commits the text set in its field, is stepped to a number by its own increment and decrement actions through the bridge, its value read back after each step (java.action, no foreground); anything but a number is refused not_supported.
focus()	Through the bridge (java.focus).
pressSequentially(), press()	Focus through the bridge, then real keys (needs the foreground).
textContent()	The whole text of a text component, else its value, else its name.
hover()	The real mouse brought over the control's middle (input.mouse.hover; needs the foreground and the helper executable). A table's row or cell has no place on the screen: hover the table.
scroll(...)	{ wheel }: the real wheel turned over the control's middle (input.mouse.wheel); for a table's row or cell, over the table. { direction } and { page }: the wheel turned three notches that way. { to } an end and a position ({ percent }, what the pager sets): the thumb of the scroll pane's vertical scroll bar dragged with the real mouse to 0 %, 100 % or the position (input.mouse.drag). scrollIntoView() drags the thumb to where the item stands among its list's items, so that it shows in the view's middle.

What lies in a scroll pane is placed where it shows: the bridge reports a long list or text area at its whole length, so SPART-Desk cuts it to the scroll pane's view, and an item wholly out of view is hidden where it is. A click, a hover or the wheel at a control's middle so lands on what the user sees.

Modal dialogs. A bridge action that opens a modal dialog (a Save dialog, a confirmation) does not return until the dialog closes, and meanwhile the JVM answers nothing through the bridge. That is why clicks use real input whenever they can. If an action in a background window opens one anyway, SPART-Desk reports it (has not returned: it most likely opened a modal dialog) and fails later operations on that application quickly with java_busy rather than hanging. Keep the application's window in the foreground — which a freshly launched application usually is — when a click opens a dialog.

Diagnostics

• app.snapshot() carries notes when a Java window could not be read, and a locator that finds nothing repeats the note in its error.
• npx spart-desk doctor lists the Java runtimes JAVA_HOME and PATH lead to, whether each has the bridge, and whether SPART-Desk's helper executable (spart-desk-host.exe, which talks to the bridge) runs.
• desktop.capabilities.java says whether this desktop reads Java windows.

Notes

• Windows Sandbox (experimental): an application that brings its runtime beside its launcher (jpackage's runtime\, a jre\) runs in the guest and is read through the bridge there; a .jar, a JDK's javaw.exe and a 32-bit JVM are refused, since the guest has no Java of its own.
• Custom components that report no accessible text stay unreadable through the bridge: an editor that paints its own lines is one. Reading the screen (resolution.md) still applies to them: getByText("Hello from the editor") finds such an editor's text, and a click there followed by keyboard.type() types into it.
• Long lists report only the rows they show. A locator scoped to the list, getByRole("list", { name: "Files" }).getByRole("listitem", { name: "file-150.txt" }), reaches an item the list is in order by, by dragging the thumb of the scroll pane's scroll bar with the real mouse to where the item should be and adjusting by what shows; a list in no order by what is asked for is paged with the real wheel until the item shows. One that names the item alone does not know which list to page.
• Table cells have no position: click() selects them, and toBeVisible() cannot tell a scrolled-out row from a visible one.
• 32-bit JVMs and Java 8 are read through their own runtime's bridge, WindowsAccessBridge-32.dll, which only the helper's 32-bit build, spart-desk-host-x86.exe, can load: the engine asks it about a JVM the 64-bit helper finds to be 32-bit.
• Application control: the bridge client runs in SPART-Desk's helper executable; a machine whose WDAC or AppLocker policy blocks it cannot read Java windows until the signed spart-desk-host.exe (and, for a 32-bit runtime, spart-desk-host-x86.exe) is allowed.

Agents

spart-desk-cli launch app.jar and desktop_launch work the same way; add --java / --no-java (MCP: java: true | false) when the application cannot be recognised. Snapshot notes are printed as note: lines under the header.