Automate a legacy Windows app
that has no API.
Plenty of valuable software has no API and no export. The only integration point is the interface a person uses. Automating that interface is possible, but how you do it decides whether the result is dependable or brittle.
Three ways to automate a screen
They differ in how they find what to click.
- Coordinates: click at x and y. Simple, and breaks whenever a window moves or resizes.
- Image matching: look for a picture of a button. Survives some movement, fails on theme or scaling changes.
- UI tree: ask the application for its buttons, fields and menus by name and type. Steadiest, when the app exposes them.
What the UI tree gives you
Windows applications expose a structured description of their controls. Automation that reads it can say 'click the button called Save' instead of 'click that spot'. That is why it keeps working when a window is moved, resized or restyled.
Building a workflow
In Onscryn you build a workflow by working in a live session. Steps click, type, wait, branch and extract text against the live UI tree. Variables, loops and error handling cover the common shapes, and selectors can find their target again when a screen shifts.
A workflow can then run on a schedule, from a webhook or on demand, and every run keeps a history.
Using the Copilot
Instead of building steps by hand, you can describe a goal in plain English. The Copilot reads the live UI tree, plans real UI actions and carries them out in front of you. It is in preview, so treat it as something to try on real tasks and review rather than something to leave unattended.
Making it reliable
Add a wait for the thing you expect and an assertion that it happened. Run against a known starting state. Keep the recording on while you tune a workflow, so you can see exactly what it did.
Good candidates for automation
Work that repeats and follows a fixed shape is the best place to start.
- Re-keying data from one system into another.
- Pulling a report out of an application on a schedule.
- Nightly or weekly routines that someone does by hand today.
- Checks that need a person to look at several screens and write down a value.
Poor candidates
Anything that needs real judgement, such as approving an unusual invoice, should stay with a person. So should tasks where the screen differs wildly each time, or where a mistake is expensive and hard to undo.
A good rule is to automate the dull, predictable part and leave a person in charge of the decision.
Triggers and schedules
A workflow can run on a schedule, from a webhook when something else happens, or on demand. Each run keeps a history, so you can see what ran, when, and how it ended. When a step fails, error handling decides whether to stop, retry or carry on.
Keeping people in the loop
Automation does not have to be all or nothing. A workflow can do the typing and stop for a person to confirm. The Copilot's runs are visible in front of you and recorded, so a person can review exactly what it did. Treat early runs as supervised and loosen things as trust builds.
Illustration
A worked example
Imagine a clerk who each morning opens an old inventory system, exports a stock level for twelve products, and types them into a spreadsheet. It takes forty minutes and nobody enjoys it.
A workflow opens the app, reads each value from the live UI tree, waits where the screen is slow, and writes the results out. It runs on a schedule before the clerk arrives. They review the output instead of producing it, and the recording shows what the workflow did if a number looks odd.
Before you automate a task
- Write down every step a person takes, including where they wait.
- Is there a reliable signal that each screen has loaded?
- What should happen when something unexpected appears, such as a dialog?
- What proves the task worked, and can the workflow check it?
- Who reviews the result, and how will they know if a run failed?
- Is the app in a known starting state each time?
Related features
Questions.
It covers similar ground: automating work through an application's interface. It runs on the same streamed session your people use.