← Back to Skills Marketplace
通用网页抓取工具
by
wuxiangfacai
· GitHub ↗
· v1.0.0
· MIT-0
31
Downloads
0
Stars
1
Active Installs
1
Versions
Install in OpenClaw
/install general-web-scraper
Description
通用网页数据抓取工具 — 支持CSS选择器抓取链接、表格数据提取，输出CSV/JSON格式。无需配置，开箱即用。
README (SKILL.md)
Web Scraper — 通用网页数据抓取工具
AI agent 专用的网页数据抓取工具。输入网址和CSS选择器，自动抓取链接或表格数据，导出为CSV或JSON。
功能
- 链接抓取 — 抓取页面中所有匹配CSS选择器的链接
- 表格抓取 — 自动提取HTML表格数据
- CSV导出 — 默认输出CSV格式
- JSON导出 — 支持JSON格式输出
- 中文友好 — 完整支持中文网页编码
使用方式
# 抓取页面所有链接
python scraper.py https://example.com
# 自定义CSS选择器
python scraper.py https://example.com "a.article-link"
# 导出JSON格式
python scraper.py https://example.com "div.item" --json
# 抓取表格数据
python scraper.py https://example.com "table#data" --table --json
依赖安装
pip install requests beautifulsoup4
适用场景
- 数据采集和调研
- 竞品信息监控
- 市场情报收集
- 内容聚合
Tags
scraping, web, data, python, automation, crawler, data-collection
Usage Guidance
Before installing, confirm you are allowed to collect from the configured sources, avoid private or sensitive data, keep exported reports in a controlled location, and review any scheduled automation so it only runs when and where you expect.
Capability Assessment
Purpose & Capability
The reported behavior, scraping or collecting content and exporting a report, matches a trend-monitoring/content-research purpose. The main caveat is privacy and site-policy handling, not evidence of deception or unrelated capability.
Instruction Scope
The instructions appear to encourage automated collection and export, but the supplied evidence does not show hidden commands, unrelated data access, credential theft, or destructive behavior.
Install Mechanism
No artifact-backed installer concern was supplied; VirusTotal telemetry is clean and the reported finding points to SKILL.md guidance rather than an executable installer.
Credentials
Network scraping and saved exports are proportionate for a content research skill, but users should scope sources, avoid sensitive/private data, and respect robots, terms, and applicable law.
Persistence & Privilege
Automated or recurring scraping can create persistence-like behavior if scheduled by the user or agent, but the supplied evidence does not show hidden background workers or privilege escalation.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install general-web-scraper - After installation, invoke the skill by name or use
/general-web-scraper - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
首次发布：支持CSS选择器链接抓取、表格提取、CSV/JSON导出
Metadata
Frequently Asked Questions
What is 通用网页抓取工具?
通用网页数据抓取工具 — 支持CSS选择器抓取链接、表格数据提取，输出CSV/JSON格式。无需配置，开箱即用。 It is an AI Agent Skill for Claude Code / OpenClaw, with 31 downloads so far.
How do I install 通用网页抓取工具?
Run "/install general-web-scraper" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is 通用网页抓取工具 free?
Yes, 通用网页抓取工具 is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does 通用网页抓取工具 support?
通用网页抓取工具 is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created 通用网页抓取工具?
It is built and maintained by wuxiangfacai (@wuxiangfacai); the current version is v1.0.0.
More Skills