handsfree-windows-control
by lijinlar
Web & Frontend DevelopmentBrief
Guide skill for controlling native Windows apps (UIA) and web browsers (Playwright) via the handsfree-windows CLI.
Description
What This Skill Does
Controls native Windows desktop apps and web browsers from the command line using the handsfree-windows CLI. Handles UI automation via Windows UIA for desktop apps and Playwright for browsers. Supports recording and replaying YAML macros that mix desktop and browser steps.
Combining UIA desktop control and Playwright browser automation in a single CLI with a shared macro format removes the need to maintain two separate automation toolchains.
When to Use It
- Automating multi-step data entry in a Windows desktop app
- Running end-to-end browser tests without writing test framework code
- Discovering UI controls in an unfamiliar Windows app before scripting it
- Recording a mixed desktop-and-web workflow as a reusable YAML macro
- Taking browser screenshots or snapshots for verification during automation
Example Workflow
User asks: open the company intranet in the browser, click 'Submit Report', then confirm the desktop notification appeared in Outlook
- 1
Run
hf browser-open --url https://intranet.example.comto open the URL in the managed browser profile - 2
Run
hf browser-snapshot --fmt textto inspect the page and locate the Submit Report button - 3
Run
hf browser-click --text "Submit Report"to click it - 4
Run
hf start --app "Outlook"thenhf tree --title-regex "Outlook" --depth 5to discover notification controls - 5
Run
hf inspect --jsonto confirm the notification element is present
Browser action completed and Outlook notification element confirmed via UIA tree
Requirements
- Python (to run setup.py and scripts)
- Windows operating system
Installation
clawhub install lijinlar/handsfree-windows-controlRun the CLI command, or paste the prompt into Claude Code, Cursor, Codex or whichever coding agent you use.