PyAutoGUI Controller

Control Windows desktop with natural-language commands for screenshots, app launching, typing, clicking, and multi-step UI automation using a local Python co...

This skill appears to be what it says: a local Windows desktop automation controller using pyautogui and an optional Playwright DOM bridge. Before installing or invoking it, consider the following: - It will control your screen, move/click the mouse, type (including via clipboard), take screenshots, and launch arbitrary executables found on the machine — only run it on a machine and account you trust. Treat it like a remote-control tool. - The code writes logs and state to runtime/ and saves learned configs under your home directory (~/.pyautogui-controller). Expect persistent state and profile directories (e.g., runtime/playwright_profile). - The Playwright bridge can navigate to web URLs and may connect to a CDP endpoint if DOM_BRIDGE_CDP_URL is set; that can create network interactions. If you do not need DOM-level automation, ensure PYAUTOGUI_CONTROLLER_USE_DOM is set to disable it, or avoid providing bridge env vars. - There are minor metadata mismatches: SKILL.md declares python/python3 as required but the registry entry lists no required binaries; the code reads multiple env vars that are not declared in registry metadata. Verify and set expected environment variables intentionally rather than relying on defaults. - Recommended precautions: run in a sandbox or throwaway VM to test, inspect runtime/playwright_profile and runtime/logs after running, and avoid providing elevated privileges or network-facing CDP URLs unless necessary. If you need to grant less privilege, consider running the skill with a limited user account and without the DOM bridge enabled. If you want, I can list the specific files that create or write to disk, the env variables the code reads, and the exact paths the skill will scan/modify.

Type: OpenClaw Skill Name: pyautogui-controller Version: 1.0.1 The bundle provides a powerful desktop and browser automation framework using PyAutoGUI and Playwright. It contains high-risk capabilities, including the ability to execute arbitrary local binaries via os.startfile and subprocess.Popen in action/desktop_backend.py, and the ability to attach to existing browser sessions via CDP in bridges/playwright_dom_bridge.py. While these features align with the stated goal of 'advanced automation,' the lack of robust input sanitization in the NLU parser (nlu/parser.py) poses a risk of command injection if the agent is prompted with malicious input. The inclusion of security tools like 'Burp Suite' in the application mapping and the use of hardcoded local paths (C:\Users\dev\Desktop\昱昱\...) suggest a highly specialized and potentially intrusive automation environment.