← Back to Skills Marketplace
Web Scraping
by
zhangqixin9527
· GitHub ↗
· v1.0.0
· MIT-0
11921
Downloads
6
Stars
66
Active Installs
1
Versions
Install in OpenClaw
/install web-scraping
Description
Extract structured information from websites using web_fetch for simple pages and browser automation for dynamic sites, login-gated flows, pagination, infini...
Usage Guidance
Install only if you intentionally want an agent to scrape sites, including potentially logged-in pages. Before use, confirm the target site, fields to collect, whether account/private data may be accessed, and where results will be stored; avoid using it for sensitive accounts unless you explicitly need that data and can manage retention and site-policy risk.
Capability Assessment
Purpose & Capability
The scraping, browser automation, extraction, and workspace output behavior fit a scraping-focused skill, but login-gated collection raises privacy and account-data stakes.
Instruction Scope
The invocation description appears broad enough to activate for generic website, monitoring, summary, or lookup requests, which can lead to scraping behavior when the user did not clearly ask for it.
Install Mechanism
No install-time execution, package-install risk, or hidden setup behavior was identified from the supplied evidence.
Credentials
Supporting login-gated scraping means the skill may access account-scoped or private data, and the instructions do not show clear consent, site-policy, or data-minimization boundaries.
Persistence & Privilege
Saving scraped results to the workspace is purpose-aligned, but the artifacts do not show clear user confirmation, retention limits, or safeguards for sensitive scraped content.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install web-scraping - After installation, invoke the skill by name or use
/web-scraping - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
- Initial release of the web-scraping skill.
- Supports data extraction from both static and dynamic websites.
- Automatically chooses between direct HTML fetching and browser automation based on site complexity and interaction needs.
- Handles dynamic content, authentication, infinite scroll, pagination, and search/filter flows.
- Outputs structured data in concise bullet lists, JSON, or CSV formats.
- Includes reliability rules for deduplication, error handling, and workspace file management.
Metadata
Frequently Asked Questions
What is Web Scraping?
Extract structured information from websites using web_fetch for simple pages and browser automation for dynamic sites, login-gated flows, pagination, infini... It is an AI Agent Skill for Claude Code / OpenClaw, with 11921 downloads so far.
How do I install Web Scraping?
Run "/install web-scraping" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Web Scraping free?
Yes, Web Scraping is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Web Scraping support?
Web Scraping is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Web Scraping?
It is built and maintained by zhangqixin9527 (@zhangqixin9527); the current version is v1.0.0.
More Skills