← Back to Skills Marketplace
8510
Downloads
9
Stars
95
Active Installs
1
Versions
Install in OpenClaw
/install scrape
Description
Legal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling.
Usage Guidance
Install only if you intend to perform authorized public scraping. Before using the examples, confirm the target site's robots.txt and terms, prefer official APIs, avoid login-protected or personal data, consider failing closed when robots.txt cannot be checked, and avoid logging sensitive URL parameters.
Capability Analysis
Type: OpenClaw Skill
Name: scrape
Version: 1.0.0
The skill bundle is designed for ethical and legal web scraping, providing comprehensive guidelines in `SKILL.md` for compliance (robots.txt, ToS, GDPR/CCPA, rate limiting) and implementing these safeguards in `code.md`. The code uses standard Python libraries to perform web requests, handle rate limits, and log activity, all aligned with the stated purpose. There is no evidence of malicious intent, data exfiltration to unauthorized parties, persistence mechanisms, or prompt injection attempts designed to subvert the agent's purpose; instead, the instructions actively promote responsible behavior.
Capability Assessment
Purpose & Capability
The stated purpose is legal web scraping, and the artifact provides checklist guidance plus example Python patterns for robots.txt checks, session setup, rate-limited requests, retries, and audit logging.
Instruction Scope
The instructions are mostly restrictive and user-protective, but the example robots.txt helper returns allowed if robots.txt cannot be read, so cautious users should fail closed or ask before scraping.
Install Mechanism
The bundle contains only SKILL.md and code.md markdown files, with no executable scripts, install hooks, package installation, or declared runtime secrets.
Credentials
Outbound requests to user-selected websites are inherent to the skill and disclosed; it does not request local credentials, browser profiles, session stores, or unrelated sensitive files.
Persistence & Privilege
The examples include audit logging of scraped URLs and response statuses, which is purpose-aligned but could retain sensitive query strings if users choose poorly scoped targets; no background persistence or privilege escalation is present.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install scrape - After installation, invoke the skill by name or use
/scrape - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release
Metadata
Frequently Asked Questions
What is Scrape?
Legal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling. It is an AI Agent Skill for Claude Code / OpenClaw, with 8510 downloads so far.
How do I install Scrape?
Run "/install scrape" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Scrape free?
Yes, Scrape is completely free (open-source). You can download, install and use it at no cost.
Which platforms does Scrape support?
Scrape is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Scrape?
It is built and maintained by Iván (@ivangdavila); the current version is v1.0.0.
More Skills