← Back to Skills Marketplace
1113
Downloads
0
Stars
3
Active Installs
1
Versions
Install in OpenClaw
/install smart-crawler
Description
智能爬虫工具 - 企业级数据采集与反爬虫处理 | Smart Web Crawler - Enterprise data collection with anti-detection
Usage Guidance
This package is a normal web crawler and appears coherent with its description, but take usual precautions: install and run in a controlled environment (Playwright will download browser binaries); review any proxy or database connection strings you provide (they can contain credentials); be aware that the tool makes network requests to target URLs you supply (so don't point it at internal or sensitive endpoints without authorization); and audit any config you create for storage backends (MongoDB/MySQL) before entering secrets. If you plan to run it in an automated agent, consider sandboxing network access and limiting where results can be uploaded.
Capability Analysis
Type: OpenClaw Skill
Name: smart-crawler
Version: 1.0.0
The 'smart-crawler' skill bundle is a legitimate web crawling utility providing standard features for data collection, including support for static (requests) and dynamic (Playwright) pages, proxy rotation, and data extraction via XPath/CSS selectors. The code is well-structured, follows the stated purpose in SKILL.md and README.md, and contains no evidence of malicious intent, data exfiltration, or prompt injection.
Capability Assessment
Purpose & Capability
Name/description (enterprise web crawler with anti-detection) aligns with included code, examples, and requirements. The code implements request-based and Playwright-based crawling, proxy support, extraction, batching and dynamic rendering as claimed.
Instruction Scope
SKILL.md and README instruct installing dependencies (pip and playwright) and running tests; runtime instructions and code only perform crawling, extraction, downloading, and optional proxy use. There are no instructions to read unrelated host files, environment variables, or to send data to unexpected endpoints.
Install Mechanism
There is no automated install spec in the registry entry (instruction-only), but SKILL.md tells users to run pip install -r requirements.txt and playwright install. Dependencies are standard public packages; Playwright will download browser binaries when installed — this is expected but worth noting.
Credentials
The skill declares no required env vars or credentials. Code supports optional proxy pools and mentions databases (in README), but no credentials are requested or embedded. Any credential use would come from how a user configures proxies or DB connectors, not from the skill itself.
Persistence & Privilege
Skill is not always-enabled and does not request elevated platform privileges. It does not modify other skills or system-wide agent settings.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install smart-crawler - After installation, invoke the skill by name or use
/smart-crawler - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release: Enterprise web crawler with anti-detection
Metadata
Frequently Asked Questions
What is Smart Crawler?
智能爬虫工具 - 企业级数据采集与反爬虫处理 | Smart Web Crawler - Enterprise data collection with anti-detection. It is an AI Agent Skill for Claude Code / OpenClaw, with 1113 downloads so far.
How do I install Smart Crawler?
Run "/install smart-crawler" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Smart Crawler free?
Yes, Smart Crawler is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Smart Crawler support?
Smart Crawler is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Smart Crawler?
It is built and maintained by Lv Lancer (@kaiyuelv); the current version is v1.0.0.
More Skills