Features/AI CopilotPreview
An AI that drives
software with no API
Describe a goal in plain English. The Copilot reads the application's live UI tree, plans real UI actions and carries them out step by step, in front of you and on the record.
Plan
- Read the UI treeui_tree
- Choose the customerclick
- Enter the amounttype
- Press Saveclick
- Check it savedassert
- 01
Read
It reads the live UI tree (the buttons, fields, menus and dialogs) rather than guessing from screenshots.
- 02
Plan
A language model picks the next action using what's actually on screen, one step at a time.
- 03
Act and check
It clicks, types, waits for controls and asserts results. Each step appears live in the session sidebar.
What it does.
Grounded in the UI tree
Plans against the structure of the real application, so actions are specific, inspectable and repeatable.
A small, auditable toolset
Click, type, run a named action, wait for a control, assert text, and report the outcome. Nothing else.
Assertions that fail loudly
It can check that a control contains what it should, so a run that goes wrong stops and says so instead of drifting on.
Live, step-by-step feed
Ask, inspect the UI tree, browse actions and workflows from one panel beside the stream. Every step is logged against the run.
Turn a run into a workflow
Capture a completed run as a saved workflow, then replay it with no language-model call.
Your choice of model provider
The planner runs against a configurable provider, including OpenAI-compatible endpoints.
Related guides
Give contractors access to one app without a VPN
A VPN gives network access when someone needs one application. How browser-based app access limits what contractors, vendors and temporary staff can reach.
8 min readAutomating a legacy Windows app that has no API
When the only way into an application is its screen, UI automation is the answer. How the UI tree works, how it differs from screen scraping, and what makes automation reliable.
7 min readPreventing data from leaving a remote session
Clipboard, file transfer, screenshots and photos: where data leaks from remote application sessions, and the controls that help, including broker-level policy and watermarks.