← 返回 Skills 市场
sky-lv

Skylv Web Scraper

作者 SKY-lv · GitHub ↗ · v1.0.0 · MIT-0
cross-platform ✓ 安全检测通过
99
总下载
0
收藏
1
当前安装
1
版本数
在 OpenClaw 中安装
/install skylv-web-scraper
功能描述
Extract and parse web page content including text, links, and images using CSS selectors and regex for flexible data scraping.
使用说明 (SKILL.md)

Web Scraper — 网页内容抓取工具

功能说明

从网页抓取并解析内容,支持多种提取方式。

使用方法

1. 抓取网页全文

用户: 抓取 https://example.com 的内容

执行步骤:

  1. 使用 web_fetch 工具抓取URL
  2. 返回markdown格式的正文内容

2. 提取特定元素

用户: 从 https://news.ycombinator.com 提取所有新闻标题

执行步骤:

  1. 使用 web_fetch 抓取页面
  2. 分析HTML结构,识别标题元素
  3. 提取并列表返回

3. 批量抓取

用户: 抓取以下URL列表的内容:
https://url1.com
https://url2.com
https://url3.com

执行步骤:

  1. 遍历URL列表
  2. 依次调用 web_fetch
  3. 汇总结果

4. 提取链接

用户: 提取 https://example.com 页面中的所有外链

执行步骤:

  1. 抓取页面内容
  2. 解析所有 \x3Ca href> 标签
  3. 过滤出外链(域名不同的链接)
  4. 列表返回

示例对话

用户: 抓取 https://github.com/trending 今天的热门项目

Agent:

  1. 调用 web_fetch 抓取 GitHub Trending 页面
  2. 解析项目列表(仓库名、描述、star数)
  3. 格式化输出:
今日 GitHub 热门项目:

1. owner/repo-name - 项目描述
   ⭐ 1,234 stars today | 📝 JavaScript

2. ...

注意事项

  • 遵守 robots.txt
  • 添加适当延迟避免被封
  • 处理反爬机制(User-Agent、Cookie等)
  • 大规模抓取建议使用代理

依赖

  • web_fetch 工具(OpenClaw内置)
  • 无需额外安装
安全使用建议
This skill appears to do what it says: fetch pages and extract content using an internal fetch tool. Before installing, consider: 1) the skill's source/homepage is unknown — verify you trust the publisher if provenance matters; 2) scraping can have legal and ethical limits (respect robots.txt, site terms, and copyright); 3) the instructions mention User-Agent, cookies, and proxies — avoid using the skill to bypass access controls or to scrape private/login-required data unless you have explicit permission; and 4) if you plan large-scale crawling, ensure you rate-limit and use appropriate infrastructure (and be mindful of potential IP blocking or service abuse). If you need stricter guarantees, request the skill author/publisher info or a published source repository before use.
能力评估
Purpose & Capability
Name and description match the runtime instructions. The only dependency referenced is an internal `web_fetch` tool, which is appropriate for fetching HTML for a scraper. No unrelated binaries, credentials, or config paths are requested.
Instruction Scope
SKILL.md stays within scraping behavior (fetch page, parse HTML, extract links/images/text, iterate URL lists). It explicitly notes obeying robots.txt and rate-limiting. It also mentions handling anti-scraping tactics (User-Agent, cookies, proxies), which is reasonable for robust scraping but could be used to evade protections; this is an operational/ethical consideration rather than an incoherence.
Install Mechanism
Instruction-only skill with no install spec and no code files; nothing is written to disk or downloaded during install, minimizing install-time risk.
Credentials
No environment variables, credentials, or config paths are requested. The skill does not ask for unrelated secrets or system access.
Persistence & Privilege
always is false and the skill is user-invocable. It does not request persistent or elevated platform privileges.
如何使用
  1. 确保已安装 OpenClaw(本地或 Docker 部署)
  2. 在对话框中输入安装命令:/install skylv-web-scraper
  3. 安装完成后,直接呼叫该 Skill 的名称或使用 /skylv-web-scraper 触发
  4. 根据 Skill 的参数说明提供必要输入,即可获得结构化输出
版本历史
v1.0.0
- Initial release of the web-scraper skill. - Supports scraping web pages, extracting text, links, and images using CSS selectors and regex. - Handles full page content, specific element extraction, batch URL processing, and link extraction. - Includes usage examples, step-by-step instructions, and best practices for responsible scraping. - Relies on the `web_fetch` tool (built-in). No extra installation required.
元数据
Slug skylv-web-scraper
版本 1.0.0
许可证 MIT-0
累计安装 1
当前安装数 1
历史版本数 1
常见问题

Skylv Web Scraper 是什么?

Extract and parse web page content including text, links, and images using CSS selectors and regex for flexible data scraping. 它是一个面向 Claude Code / OpenClaw 的 AI Agent Skill 插件,目前累计下载 99 次。

如何安装 Skylv Web Scraper?

在 OpenClaw 或 Claude Code 对话框中运行命令「/install skylv-web-scraper」即可一键安装,无需额外配置。

Skylv Web Scraper 是免费的吗?

是的,Skylv Web Scraper 完全免费,采用 MIT-0 许可证,可自由下载、安装和使用。

Skylv Web Scraper 支持哪些平台?

Skylv Web Scraper 跨平台运行,可在任意部署了 OpenClaw / Claude Code 的环境中使用(cross-platform)。

谁开发了 Skylv Web Scraper?

由 SKY-lv(@sky-lv)开发并维护,当前版本 v1.0.0。

💬 留言讨论