← 返回 Skills 市场
99
总下载
0
收藏
1
当前安装
1
版本数
在 OpenClaw 中安装
/install skylv-web-scraper
功能描述
Extract and parse web page content including text, links, and images using CSS selectors and regex for flexible data scraping.
使用说明 (SKILL.md)
Web Scraper — 网页内容抓取工具
功能说明
从网页抓取并解析内容,支持多种提取方式。
使用方法
1. 抓取网页全文
用户: 抓取 https://example.com 的内容
执行步骤:
- 使用
web_fetch工具抓取URL - 返回markdown格式的正文内容
2. 提取特定元素
用户: 从 https://news.ycombinator.com 提取所有新闻标题
执行步骤:
- 使用
web_fetch抓取页面 - 分析HTML结构,识别标题元素
- 提取并列表返回
3. 批量抓取
用户: 抓取以下URL列表的内容:
https://url1.com
https://url2.com
https://url3.com
执行步骤:
- 遍历URL列表
- 依次调用
web_fetch - 汇总结果
4. 提取链接
用户: 提取 https://example.com 页面中的所有外链
执行步骤:
- 抓取页面内容
- 解析所有
\x3Ca href>标签 - 过滤出外链(域名不同的链接)
- 列表返回
示例对话
用户: 抓取 https://github.com/trending 今天的热门项目
Agent:
- 调用
web_fetch抓取 GitHub Trending 页面 - 解析项目列表(仓库名、描述、star数)
- 格式化输出:
今日 GitHub 热门项目:
1. owner/repo-name - 项目描述
⭐ 1,234 stars today | 📝 JavaScript
2. ...
注意事项
- 遵守 robots.txt
- 添加适当延迟避免被封
- 处理反爬机制(User-Agent、Cookie等)
- 大规模抓取建议使用代理
依赖
web_fetch工具(OpenClaw内置)- 无需额外安装
安全使用建议
This skill appears to do what it says: fetch pages and extract content using an internal fetch tool. Before installing, consider: 1) the skill's source/homepage is unknown — verify you trust the publisher if provenance matters; 2) scraping can have legal and ethical limits (respect robots.txt, site terms, and copyright); 3) the instructions mention User-Agent, cookies, and proxies — avoid using the skill to bypass access controls or to scrape private/login-required data unless you have explicit permission; and 4) if you plan large-scale crawling, ensure you rate-limit and use appropriate infrastructure (and be mindful of potential IP blocking or service abuse). If you need stricter guarantees, request the skill author/publisher info or a published source repository before use.
能力评估
Purpose & Capability
Name and description match the runtime instructions. The only dependency referenced is an internal `web_fetch` tool, which is appropriate for fetching HTML for a scraper. No unrelated binaries, credentials, or config paths are requested.
Instruction Scope
SKILL.md stays within scraping behavior (fetch page, parse HTML, extract links/images/text, iterate URL lists). It explicitly notes obeying robots.txt and rate-limiting. It also mentions handling anti-scraping tactics (User-Agent, cookies, proxies), which is reasonable for robust scraping but could be used to evade protections; this is an operational/ethical consideration rather than an incoherence.
Install Mechanism
Instruction-only skill with no install spec and no code files; nothing is written to disk or downloaded during install, minimizing install-time risk.
Credentials
No environment variables, credentials, or config paths are requested. The skill does not ask for unrelated secrets or system access.
Persistence & Privilege
always is false and the skill is user-invocable. It does not request persistent or elevated platform privileges.
如何使用
- 确保已安装 OpenClaw(本地或 Docker 部署)
- 在对话框中输入安装命令:
/install skylv-web-scraper - 安装完成后,直接呼叫该 Skill 的名称或使用
/skylv-web-scraper触发 - 根据 Skill 的参数说明提供必要输入,即可获得结构化输出
版本历史
v1.0.0
- Initial release of the web-scraper skill.
- Supports scraping web pages, extracting text, links, and images using CSS selectors and regex.
- Handles full page content, specific element extraction, batch URL processing, and link extraction.
- Includes usage examples, step-by-step instructions, and best practices for responsible scraping.
- Relies on the `web_fetch` tool (built-in). No extra installation required.
元数据
常见问题
Skylv Web Scraper 是什么?
Extract and parse web page content including text, links, and images using CSS selectors and regex for flexible data scraping. 它是一个面向 Claude Code / OpenClaw 的 AI Agent Skill 插件,目前累计下载 99 次。
如何安装 Skylv Web Scraper?
在 OpenClaw 或 Claude Code 对话框中运行命令「/install skylv-web-scraper」即可一键安装,无需额外配置。
Skylv Web Scraper 是免费的吗?
是的,Skylv Web Scraper 完全免费,采用 MIT-0 许可证,可自由下载、安装和使用。
Skylv Web Scraper 支持哪些平台?
Skylv Web Scraper 跨平台运行,可在任意部署了 OpenClaw / Claude Code 的环境中使用(cross-platform)。
谁开发了 Skylv Web Scraper?
由 SKY-lv(@sky-lv)开发并维护,当前版本 v1.0.0。
推荐 Skills