PinchBench

Run PinchBench benchmarks to evaluate OpenClaw agent performance across real-world tasks. Use when testing model capabilities, comparing models, submitting b...

What to check before installing/using this skill: - Inspect lib_upload.py (and the --register/upload code path) to see exactly what fields are sent to pinchbench.com; do not upload runs containing private data unless you understand what's transmitted. Use --no-upload for local-only testing. - Review the grading code behavior: tasks may contain embedded Python automated checks which the grading engine executes with exec(); only run tasks from trusted sources or review task_*.md files before running. - Be aware the skill reads OpenClaw files in your home (~/.openclaw/agents/*) and can create new agents and workspaces via the openclaw CLI — this may expose agent transcripts or sensitive tool outputs to local processing and (if you upload) to the leaderboard. - The pyproject lists fabric/paramiko (SSH capabilities) not mentioned in the docs — if you install dependencies, consider doing so in an isolated virtualenv or sandbox and review why SSH libraries are needed. - If you want to be cautious: run the scripts in an isolated environment (container or VM), run with --no-upload first, and audit any results JSON for sensitive content before sharing. If you plan to allow uploading, confirm the upload endpoint and privacy policy on pinchbench.com and inspect where tokens are stored by the registration flow.

Type: OpenClaw Skill Name: pinchbench Version: 1.0.0 The skill is classified as suspicious primarily due to a critical Remote Code Execution (RCE) vulnerability. The `scripts/lib_grading.py` file uses `exec()` to run Python code directly extracted from the `Automated Checks` sections within task definition markdown files (`tasks/*.md`). This design allows any malicious task submitted to the benchmarking system to execute arbitrary Python code on the host. Additionally, the skill collects system metadata (OS, CPU, Python version, hostname hash) and benchmark results, then uploads them to `api.pinchbench.com`, which, while stated for a public leaderboard, constitutes data exfiltration to an external endpoint. The `pyproject.toml` also lists `fabric` and `paramiko` (SSH libraries) as dependencies, which are powerful capabilities not explicitly used or justified by the current code.