← Back to Skills Marketplace
1080
Downloads
0
Stars
7
Active Installs
1
Versions
Install in OpenClaw
/install openclaw-scrapling
Description
Advanced web scraping with anti-bot bypass, JavaScript support, and adaptive selectors. Use when scraping websites with Cloudflare protection, dynamic conten...
Usage Guidance
This package is internally coherent for an advanced web scraper — it will run Python code, download browser binaries, and can access arbitrary URLs (including internal network addresses). Before installing:
- Only install if you trust the source (GitHub repo) and you accept that the skill will run code and download ~500MB of browser binaries.
- Expect session cookies and selector caches to be written under the skill directory (~/.openclaw/skills/scrapling/sessions and selector_cache.json). Remove those files if they may contain sensitive tokens.
- Do not pass secrets or site credentials to the tool unless you trust it and the host environment; CLI args (username/password) are stored only if you save sessions.
- If you are concerned about exfiltration or internal network access, run the skill in a restricted environment (network policies, sandbox, or VM) and inspect scrape.py and the installed scrapling package source before use.
- If you need to ensure minimal privilege, avoid enabling stealth/dynamic modes that start a browser or save sessions, and prefer one-off basic HTTP fetches with explicit safe target URLs.
Capability Analysis
Type: OpenClaw Skill
Name: openclaw-scrapling
Version: 1.0.0
The skill is a powerful web scraping tool with legitimate functionality, but it exposes several high-risk capabilities that could be exploited by a compromised AI agent through prompt injection. Specifically, the `scrape.py` script allows arbitrary file writes via the `--output` argument, enables extensive control over network requests (URL, proxy, headers) which could lead to SSRF, and the `SKILL.md` documentation explicitly details how to run custom Python scripts, creating a potential RCE vector if an agent can be prompted to write and execute arbitrary code. While no direct malicious intent or prompt injection is found in the provided files, these capabilities present significant vulnerabilities.
Capability Assessment
Purpose & Capability
Name/description match the code and docs: scrape.py, examples, requirements.txt and skill.json all implement a scraper with stealth/dynamic/adaptive features. Declared required binaries (python3, pip) and Python package dependency (scrapling) are appropriate for the described functionality. Minor version differences in metadata (>=0.3.0 vs >=0.4.0) are not a red flag by themselves.
Instruction Scope
SKILL.md and scrape.py instruct the agent to run local scraping commands and to store sessions/selectors in the skill directory. The instructions allow scraping arbitrary URLs (external or internal), performing logins (username/password passed as CLI args), saving session files, screenshots, and writing outputs. They do not instruct reading unrelated system files or environment variables, but they do permit sending credentials via CLI args and persisting session tokens/cookies to disk.
Install Mechanism
There is no built-in install spec in the registry entry, but the repo includes requirements.txt and documented install steps that call 'pip install -r requirements.txt' and 'scrapling install' which will download browser binaries (~500MB). This is expected for a browser-driven scraper; no obscure external download URLs or shorteners are used in the package itself. Browser downloads will occur at runtime when the helper command is run.
Credentials
The skill declares no required environment variables or credentials, which fits the described purpose. However, the tool accepts credentials via CLI arguments (username/password) and will persist session state (session files and selector_cache.json) in the skill directory. Those behaviors are reasonable for a scraper but mean the skill can store sensitive tokens/credentials if provided.
Persistence & Privilege
always:false (no forced installation). The skill writes files into its own directory (sessions/, selector_cache.json) and downloads browsers into standard caches during 'scrapling install'. This is normal for this class of tool but means data and cookies will persist on disk under the skill and browser cache directories unless cleaned.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install openclaw-scrapling - After installation, invoke the skill by name or use
/openclaw-scrapling - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release of Scrapling skill for advanced, resilient web scraping.
- Supports scraping sites with anti-bot protection, JavaScript-rendered content, and frequent UI changes.
- Features stealth mode to bypass Cloudflare, bot detection, and use browser fingerprint spoofing.
- Handles dynamic content via Playwright-based automation and adaptive selectors for robust scraping across redesigns.
- Includes session management for login-required sites and support for proxies, rate limiting, and custom headers.
- Offers multiple extraction modes (text, markdown, attributes, multi-field) and output formats (JSON, CSV, TXT, MD, HTML).
- Provides both CLI commands and a Python API for flexibility.
Metadata
Frequently Asked Questions
What is OpenClaw Scrapling?
Advanced web scraping with anti-bot bypass, JavaScript support, and adaptive selectors. Use when scraping websites with Cloudflare protection, dynamic conten... It is an AI Agent Skill for Claude Code / OpenClaw, with 1080 downloads so far.
How do I install OpenClaw Scrapling?
Run "/install openclaw-scrapling" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is OpenClaw Scrapling free?
Yes, OpenClaw Scrapling is completely free (open-source). You can download, install and use it at no cost.
Which platforms does OpenClaw Scrapling support?
OpenClaw Scrapling is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created OpenClaw Scrapling?
It is built and maintained by cryptos3c (@cryptos3c); the current version is v1.0.0.
More Skills