← Back to Skills Marketplace
512
Downloads
0
Stars
1
Active Installs
2
Versions
Install in OpenClaw
/install web-scraper-pro
Description
Intelligent web scraper that fetches any URL and returns clean Markdown content. Triggers on requests like "帮我抓取网页", "获取这个网页内容", "fetch this URL", "scrape th...
Usage Guidance
This skill will send any requested URL (and the resulting page content) to several external conversion services and will call a SkillPay billing API before performing the fetch. Notable issues: (1) the code includes a hard-coded SkillPay API key in the repository — this key authorizes billing calls and should not be embedded in a published skill; (2) fetched content will be transmitted to third parties (markdown.new, defuddle.md, r.jina.ai, Scrapling), so do not use this on sensitive or internal URLs; (3) the skill forces payment verification and may charge or direct users to top-up links that route funds to the key-owner. Before installing, ask the publisher to: remove any embedded API keys and require callers to supply their own billing credentials, document what data is sent to each third party, and provide a privacy/terms link. If you must use it, test in a sandbox, avoid sensitive URLs, and insist on using your own billing credentials (or disable the monetization flow).
Capability Analysis
Type: OpenClaw Skill
Name: web-scraper-pro
Version: 1.1.0
The skill bundle implements a mandatory micro-payment gatekeeper using a third-party service (skillpay.me) and hardcoded API credentials in SKILL.md and payment.py. It uses aggressive prompt instructions to force the AI agent to perform financial transactions (charging 0.001 USDT) before providing the advertised scraping functionality. While no direct system compromise or data theft was observed, the hardcoded 'Secret Key' (sk_d6d26f29...) and the enforcement of a proprietary payment layer are highly irregular for a skill bundle and could be used for unauthorized monetization or user tracking.
Capability Assessment
Purpose & Capability
The name/description (web scraper → convert URL to Markdown) matches the instructions and code for fetching and converting web pages. However, the SKILL.md and payment.py embed a monetization flow (SkillPay) with a hard-coded API key and SKILL_ID; a monetized scraper can be legitimate, but a shipping skill that contains someone else's billing API key rather than asking the deployer to configure their own credentials is unexpected and questionable.
Instruction Scope
Runtime instructions require executing a payment verification step (network call to skillpay.me) before any fetch. The fetch flow sends the target URL (and effectively the page content) to third-party services (markdown.new, defuddle.md, r.jina.ai, and an included Scrapling library). That means user-provided URLs/content will be transmitted to multiple external endpoints — a clear privacy/data-exfiltration risk for sensitive or internal URLs.
Install Mechanism
No installation spec; skill is instruction-only with an included payment.py file. No external archive downloads or unusual install steps were specified. Risk from install mechanism is low.
Credentials
The bundle does not declare required env vars but both SKILL.md and payment.py include a hard-coded BILLING_API_KEY value (sk_...) and a SKILL_ID. Hard-coding a secret API key is a red flag: it grants the skill author (or whoever controls that key) the ability to perform billing operations on that billing endpoint on behalf of users. The only environment variable referenced is SKILLPAY_USER_ID (used as user identifier), which is reasonable, but the embedded key makes the credential model asymmetric and surprising.
Persistence & Privilege
always is false and nothing in the files requests persistent platform-wide privileges or modifies other skills. The payment decorator/context manager enforces payment before function execution but does not change agent configuration beyond its own behavior.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install web-scraper-pro - After installation, invoke the skill by name or use
/web-scraper-pro - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.1.0
Fix SkillPay API integration: correct endpoint, auth header, skill_id, and response handling
v1.0.0
Initial release with multi-layer fetch strategy and SkillPay payment integration
Metadata
Frequently Asked Questions
What is Web Scraper Pro?
Intelligent web scraper that fetches any URL and returns clean Markdown content. Triggers on requests like "帮我抓取网页", "获取这个网页内容", "fetch this URL", "scrape th... It is an AI Agent Skill for Claude Code / OpenClaw, with 512 downloads so far.
How do I install Web Scraper Pro?
Run "/install web-scraper-pro" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Web Scraper Pro free?
Yes, Web Scraper Pro is completely free (open-source). You can download, install and use it at no cost.
Which platforms does Web Scraper Pro support?
Web Scraper Pro is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Web Scraper Pro?
It is built and maintained by LiuSir (@codehourra); the current version is v1.1.0.
More Skills