← Back to Skills Marketplace
Web Scraping
by
zhangqixin9527
· GitHub ↗
· v1.0.0
· MIT-0
13868
Downloads
6
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install web-scraping
Description
Extract structured information from websites using web_fetch for simple pages and browser automation for dynamic sites, login-gated flows, pagination, infini...
Usage Guidance
Before installing, understand that this skill can guide an agent to scrape dynamic or login-gated pages and save larger results in the workspace. Use it only for sites and fields you intentionally specify, avoid scraping secrets or unnecessary personal data, and review saved files when working with authenticated content.
Capability Assessment
Credentials
Browser/session use and saving scraped results to workspace files can involve sensitive website content, but those behaviors are disclosed and proportionate for a scraping skill.
Install Mechanism
The inspected artifact is a single Markdown skill file with no executable scripts, dependencies, install hooks, or remote payload retrieval.
Instruction Scope
The trigger text is somewhat broad and includes monitoring, summaries, pagination, infinite scroll, and login-gated flows, but the runtime workflow directs the agent to identify target sites and exact fields, test one page first, and extract only requested fields.
Persistence & Privilege
The only persistence described is task output or checkpoints saved under the workspace with clear names; there is no system persistence, privilege escalation, memory poisoning, or tool hijacking.
Purpose & Capability
The skill is narrowly about extracting structured information from websites using web_fetch, browser automation, or web_search discovery, and its capabilities fit that purpose.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install web-scraping - After installation, invoke the skill by name or use
/web-scraping - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
- Initial release of the web-scraping skill.
- Supports data extraction from both static and dynamic websites.
- Automatically chooses between direct HTML fetching and browser automation based on site complexity and interaction needs.
- Handles dynamic content, authentication, infinite scroll, pagination, and search/filter flows.
- Outputs structured data in concise bullet lists, JSON, or CSV formats.
- Includes reliability rules for deduplication, error handling, and workspace file management.
Metadata
Frequently Asked Questions
What is Web Scraping?
Extract structured information from websites using web_fetch for simple pages and browser automation for dynamic sites, login-gated flows, pagination, infini... It is an AI Agent Skill for Claude Code / OpenClaw, with 13868 downloads so far.
How do I install Web Scraping?
Run "/install web-scraping" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Web Scraping free?
Yes, Web Scraping is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Web Scraping support?
Web Scraping is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Web Scraping?
It is built and maintained by zhangqixin9527 (@zhangqixin9527); the current version is v1.0.0.
More Skills