Skip to main content

Security considerations

SPART-Desk drives real applications with the privileges of the user running it. This page states what the engine does with processes, input, files and network, which controls exist, and what remains the operator's responsibility.

Processes and privileges​

  • The engine spawns these kinds of external process, all with shell: false and explicit argument arrays; no string is ever interpolated into a shell: SPART-Desk's helper executable (native/spart-desk-host.exe, shipped inside the package and signed in releases), which is the Windows backend itself (backend-windows.md): kept running to read every window through UI Automation and act on what it read, to post keys and send real input, to take pictures, to read the process table, to tell which Store app each ApplicationFrameHost frame holds (read-only), to start a Store app by its app ID through Windows' application activation (as the Start menu does), and, as long-lived hosts started on first use, to read screen text and Java windows; tasklist.exe to tie a launcher-stub launch to the processes it created; and wsb.exe for the Windows Sandbox runtime (experimental). npx spart-desk doctor starts the helper once to check that it runs. The engine never starts PowerShell or any other script host, and no third-party automation binary, of its own accord.
  • desktop.run starts the program a test names, where the applications run, as the user running the tests (the guest's user in the sandbox), with its arguments each passed as one and no shell; the action log keeps the program's name and how many arguments, never the arguments.
  • Java applications. The helper's Java Access Bridge host loads windowsaccessbridge-64.dll from the Java runtime of the application being automated (then System32, then JAVA_HOME): a DLL belonging to the application under test runs inside the host, which runs as the same user as that application. SPART-Desk ships no copy of it. To make a JVM expose its controls, a launch adds the bridge option to JAVA_TOOL_OPTIONS in the environment of the process it starts, and only there; no user or system file is changed. See java-apps.md.
  • The helper runs as the current user; it uses UI Automation and SendInput, which Windows restricts to the interactive session and the foreground window. SPART-Desk never elevates. It brings the target's window to the front only for an action that has no way in the background left (foreground: "require" turns that off); input Windows still refuses is reported (foreground_required). A window of a process above the runner's integrity level (run as administrator, the credential prompt) is out of reach by Windows' design; acting in it fails with elevation_required. Running the test runner elevated is the user's decision, with the precautions in elevated.md.
  • Application processes are launched detached, tracked by PID, and only processes SPART-Desk launched (or was explicitly told to attach to) are ever terminated. A launch that follows a launcher stub reads the process table with parent ids (through the helper executable, read-only) and stops only processes that still descend from the process it spawned and started after it — a recycled process id is never trusted on its own; the spawned process itself is stopped through its handle. It never terminates processes by name, and never terminates ApplicationFrameHost (which draws every Store app's frame) unless it was targeted explicitly: a Store app is stopped through its own process, and one started by its app ID only while that process is still the one Windows started.
  • Closing a launched application also stops the processes it started since the launch (found the same way). A window another process shows owned by the application's window (a credential prompt, a print dialog) is part of the application's view, but its process is never stopped for it. desktop.shell() stops nothing when closed: Explorer and the shell's hosts are never terminated.

The Windows Sandbox runtime​

Experimental

The Windows Sandbox runtime may change in minor versions until its prerequisites are validated against a live Windows Sandbox.

The Windows Sandbox runtime isolates the application under test in a disposable guest; the host session runs the engine, and the guest runs a copy of the helper as an agent, reached over TCP on the sandbox's own network with a token made for that boot and a firewall rule scoped to the host's address (runtime-windows-sandbox.md). Preparing the guest runs as SYSTEM through wsb exec once: the helper copied in, the firewall rule, and Smart App Control turned off in the guest when guestAppControl: "off" asks for it. Terminating guest processes goes through the agent as the guest's user, and through wsb exec … --run-as System only when the agent cannot be reached.

Input injection​

Real input goes wherever the foreground window is. SPART-Desk resolves a control immediately before acting, checks visibility/enabled state, and clicks at the centre of the control's current bounds; between resolution and the click there is an inherent race in any desktop automation tool. On shared or interactive machines, run automation on a dedicated session or in Windows Sandbox (experimental), and keep raw-coordinate clicks (app.window().mouse.*, desktop_mouse) for cases that need them.

Files written​

  • Failure diagnostics: screenshots and ui-tree.json under diagnostics.directory (default %TEMP%\spart-desk). Screenshots and UI trees can contain sensitive data (customer records, credentials visible on screen). Disable them (screenshotOnFailure: false, snapshotOnFailure: false) or point the directory at a controlled location and clean it up in CI.
  • Traces (desktop.trace.pipe) contain locators, control names and values observed; the same caveat applies.
  • The Windows Sandbox runtime's (experimental) staging folders under %LOCALAPPDATA%\spart-desk\sandbox (SPART_DESK_CACHE_DIR): a copy of the helper, the agent's token for that boot, and copies of the launched applications' folders, shared into the guest read-only.

Text and secrets​

  • fill() sends the value to the application; operation records carry the action name, locator and control, never the typed text, and the MCP layer logs typed text by length only. The agent session does keep typed values in memory as part of an application's recording, and an exported test (desktop_export_test, spart-desk-cli export) contains them as source.
  • textContent(), snapshots and screenshots do expose what the UI shows. Treat traces and diagnostics as sensitive artefacts — and the command line's .spart-desk-cli/ folder, which holds complete snapshots (every control name and value) and screenshots; it ignores itself in git.

Network​

The engine never uses the network, with one exception: the Windows Sandbox runtime (experimental) reaches the helper's agent inside the guest over the sandbox's own virtual switch (TCP, a dynamic port chosen at random, a token made for that boot, no encryption; nothing but this host and the guest is on that network). Nothing is downloaded: the package is the installation. The helper executable ships inside the npm package and uses the network only as that agent, inside the guest.

Supply chain and artifact integrity​

  • The engine has no runtime dependencies; the MCP package depends on @modelcontextprotocol/sdk + zod. Neither has a native Node addon or a post-install script. The optional @spartaficial/spart-desk-ocr-paddle carries ONNX Runtime's native build for Windows, Microsoft's signed files, and no post-install script either.
  • The npm package contains two executables: the helper native/spart-desk-host.exe, about 390 KB, the Windows backend itself, and native/spart-desk-host-x86.exe, the same program built as a 32-bit executable, for 32-bit Java runtimes and the windows of 32-bit processes. Both ship inside the package, signed, with SPART-Desk's mark as their icon and the package's version, company and product as their file properties, and run on the .NET Framework that Windows has. They are built and signed with Azure Artifact Signing (formerly Trusted Signing) in the release pipeline, never on a workstation, so application control (WDAC, AppLocker, Smart App Control) can allow them by publisher; the helper reports its own signature, which doctor prints (windows-security-gates.md). Nothing else that runs is downloaded or provisioned. What the helper took from other projects is listed in notices/third-party.md.

Studio​

npx spart-desk studio starts a second listener: a server on 127.0.0.1, on a random port (or --port), behind the Studio window. It can drive the desktop as a test would, so:

  • the window is opened through a one-time link, with a 256-bit token inside the page it serves that every later request must carry; the token is never written to disk. The window keeps it for as long as it lives and moves to Studio's own address, which serves the page without a token, so a reload works and an address someone copied opens nothing;
  • the link is passed to the browser on its command line (Microsoft Edge or Google Chrome started with --app, else the default browser through Explorer), where a process of the same user can read it; it opens one window, once;
  • requests whose Host is not the server's own address are refused (DNS rebinding), as are requests a browser marks as coming from another site (Sec-Fetch-Site); there is no CORS, and state changes go through POST with a JSON body of at most 1 MiB; past that, what arrives is dropped, not kept, and the request is refused;
  • the page renders every name, value and title from the application as text, never as HTML, under a content security policy that loads nothing from anywhere; the engine makes no network requests and nothing is uploaded;
  • Studio never brings windows to the front. It starts an application only when asked to (--launch, Start application…), with the Java Access Bridge turned on for that process when it is a Java application, and never closes one: what it started keeps running when the session ends. It writes only what Save…, Append to…, Share… and the Pictures tool's Save picture, Rename…, Delete… and Update index are asked to write, every path checked by one rule:
    • Save… and Append to… write only a test's file (.ts, .mts, .cts, .js, .mjs) under the folder the command was started in or one --allow-path names. Append to… writes only to a file that is there and imports SPART-Desk, and writes it whole or not at all.
    • Pictures are written, renamed and deleted only as .png files inside the image folder (images.directory), and the typed index only as its two files there (index.mjs, index.d.mts). The image folder must lie under those same folders, or the tool is read-only. Each is written whole or not at all.
    • A saved picture is served to the page only by a name the folder holds: the name is looked up, never joined to a path.
    • The page sends rectangles, never pixels.
    • A bundle is a folder, and its .zip beside it, under those same folders. It goes into a new or empty folder, written beside it first and then put there whole. A folder that holds a bundle is replaced only when the form says so; a folder of other files never.
    • Open folder shows only the bundle Share… wrote last; the page never names a path to open.
    • Ctrl+Shift+F9 is held for Studio while an application is chosen, to save a state; it is never recorded as a step, and it is let go when the session ends.
  • a redaction pattern, which runs against every text of a bundle, is refused when it could take exponential time: a group repeated by *, + or {n,m} that holds a quantifier or an alternation, a backreference, or more than 200 characters;
  • the user's input is captured only while recording is on, an application is chosen and Studio's window shows the Recorder. The helper executable's studio mode then reads the mouse and the keyboard through Windows' raw input (no keyboard hook, ever; a mouse hook only while picking, to swallow the click that ends it), reports which control took the focus, and keeps a small copy of the screen at each mouse press. That copy is of the screen as it was just before the press: while recording, the helper copies the area around the pointer once it rests, and keeps only the latest copy, in memory, until the next press. Studio looks for saved pictures in the press's copy, in the session's memory. The frame Studio draws is excluded from every capture, screen sharing included. Only events in the application's own windows become steps; the rest is noted as "outside the application". What is typed into a password field is discarded as it arrives. Everything stays in the session's memory until Save… writes the test or a support bundle is exported. Capture stops when recording is paused, when the Recorder's window closes, and with Exit;
  • while an application is chosen, the helper watches where that application's windows are, to take a frame it drew off the screen when the window under it moves; it reads nothing of other applications' windows;
  • the server closes with Exit, with Ctrl+C in the terminal, or on its own 15 seconds after its last window closed.

Anyone running as the same user who can read the terminal or the browser's memory can drive the desktop through it: the boundary the session host and the stdio MCP server have. Recorded values (fill) appear in the generated source unless masked, as with an exported agent test.

A support bundle (Studio's Share…, spart-desk support capture or spart-desk support export) is a folder written where asked, never uploaded. It holds the application's saved states: pictures of every window, observations, and notes written on them. support capture also records the application's executable path (the user profile masked), version, integrity level and loaded module names; it clicks and types nothing. It holds what the session observed: control names and values, screenshots, the raw output of the backend. Redaction is on by default: values, the texts of cells, rows, items and documents, and text read off the screen are replaced by tokens, the same token for the same text, and their places are painted over in screenshots. Captions, ids and bounds stay, and so does any text that is such a caption, wherever else it appears; so may a title that names a customer. Notes are written as typed, tokenised only where they hold a text already redacted. Review a bundle before sending it (studio.md).

Failure evidence is written as the application said it, unless the configuration asks for diagnostics: { redact: true }: then the trace file and every sink, the failure tree and its picture, the step pictures and the report are redacted the same way, with the same tokens within one desktop (assertions-diagnostics.md). What stays in memory and the error that is thrown are not.

Agents (MCP and command line)​

The agent layer adds deterministic controls that need no LLM provider:

  • application allow-list (allowedApplications), sandbox-only mode (experimental), restricted key chords (Alt+F4, Control+Alt+Delete, Meta+*, … by default), confirmation for sensitive actions, a before-action hook;
  • raw mouse input at coordinates (desktop_mouse), attaching to running applications (desktop_attach) and the Windows shell (desktop_shell: taskbar, tray, notifications) are off by default and must be enabled explicitly; the application allow-list never opens the shell;
  • running a command (desktop_run) is running code: it is offered only for the programs --allow-run names, one by one, and its output is fenced as untrusted content. Name only what the agent needs, prefer the sandbox runtime (--sandbox-only), and confirm it (--confirm run);
  • windows other processes show for an application — a dialog owned by its window, a window of a process it started — belong to the application an agent was allowed to launch or attach to, and are automated with it (a credential prompt included, which an elevated session can fill). Keep automation that types secrets out of agent sessions, or confine it with sensitiveActions and beforeAction; an elevated session drives administrative tools, so keep its allow-list narrow (elevated.md);
  • snapshots are capped, refs are re-resolved through the engine, and every result carrying UI text is fenced as <untrusted-ui-content> so on-screen text is treated as data, never as instructions (prompt-injection mitigation);
  • every tool call is logged with runtime, target, outcome and the engine's operation record; typed text by length only (the recording behind desktop_export_test keeps the values, see above).

Transports: the MCP server speaks stdio by default — a child of the MCP client with the client's privileges, no listening socket. The spart-desk-cli session host (spart-desk-mcp --transport http) is the one listener in SPART-Desk: bound to 127.0.0.1 on a random port; every request needs a bearer token that lives in a per-user session file (%TEMP%\spart-desk-cli\<name>.json); non-loopback Host headers are refused (DNS rebinding); the host exits after an idle timeout. Anyone running as the same user can read that file and drive the host — the same trust boundary as the stdio server.

What SPART-Desk does not do​

  • It does not sandbox itself: automation runs with your user's rights.
  • It does not verify the application under test.
  • It does not encrypt diagnostics or traces.
  • It cannot prevent an agent from doing harmful things inside an allowed application; scope allow-lists tightly and prefer the sandbox runtime (experimental).