← Back to Skills Marketplace
Web Scraper
by
rupertnt034
· GitHub ↗
· v1.0.0
· MIT-0
838
Downloads
1
Stars
2
Active Installs
1
Versions
Install in OpenClaw
/install rupert-web-scraper
Description
Extract and export structured web data like text, tables, and images in JSON, CSV, Markdown, or SQL formats while respecting ethical and legal scraping pract...
Usage Guidance
This skill is an instruction-only web-scraper and appears coherent and low-risk from the manifest (no installs, no credentials). Before using it, consider: 1) legal/ethical risk — ensure you have rights to scrape a target and respect robots.txt and TOS; 2) scope control — restrict crawling to specific allowlists and set conservative depth/rate limits to avoid accidental broad crawls or DOS; 3) sensitive targets — do not allow the agent to crawl internal/private URLs or sites requiring authentication unless you trust the outcome; 4) data exfiltration risk — although the skill does not request credentials, scraped data could still be transmitted elsewhere by the agent/platform, so review platform network/output policies; and 5) testing — try on a small, public test site first. If you want stronger guarantees, require explicit user confirmation for any multi-page crawl and add an allowlist/denylist enforcement step in the SKILL.md or platform policy.
Capability Analysis
Type: OpenClaw Skill
Name: rupert-web-scraper
Version: 1.0.0
The skill bundle contains only metadata and documentation for a web scraping tool. The instructions in SKILL.md describe standard data extraction capabilities and emphasize ethical practices such as respecting robots.txt and implementing rate limiting. No executable code or malicious instructions were found.
Capability Assessment
Purpose & Capability
Name and description match the SKILL.md. Requested capabilities (data extraction, formats, rate-limiting, proxy support, robots.txt) are appropriate for a web scraper and there are no unrelated environment variables, binaries, or install steps.
Instruction Scope
The SKILL.md is high-level and describes typical scraping commands and best practices (respect robots.txt, rate limits, UA rotation). It does not instruct reading local files or environment variables, nor does it point to unexpected external endpoints. However, it is intentionally permissive (e.g., 'crawl [website] depth [n]') and lacks enforcement of safeguards—so an agent using this skill could be directed to crawl wide ranges of sites unless additional runtime limits or confirmations are imposed.
Install Mechanism
No install spec and no code files are present, so nothing is written to disk and there is no third-party package download risk.
Credentials
No environment variables, credentials, or config paths are requested. The lack of declared secrets is proportional to an instruction-only scraper.
Persistence & Privilege
always is false and the skill does not request persistent or elevated privileges. It does not attempt to modify other skills or system-wide settings.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install rupert-web-scraper - After installation, invoke the skill by name or use
/rupert-web-scraper - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release of the Web Scraper Skill.
- Extracts text, tables, and media from websites.
- Supports output to JSON, CSV, Markdown, and SQL.
- Features rate limiting, caching, retries, proxy support, and robust error handling.
- Adheres to ethical scraping standards, including robots.txt and legal compliance.
- Includes simple commands for scraping and exporting data in various formats.
Metadata
Frequently Asked Questions
What is Web Scraper?
Extract and export structured web data like text, tables, and images in JSON, CSV, Markdown, or SQL formats while respecting ethical and legal scraping pract... It is an AI Agent Skill for Claude Code / OpenClaw, with 838 downloads so far.
How do I install Web Scraper?
Run "/install rupert-web-scraper" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Web Scraper free?
Yes, Web Scraper is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Web Scraper support?
Web Scraper is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Web Scraper?
It is built and maintained by rupertnt034 (@rupertnt034); the current version is v1.0.0.
More Skills