← 返回 Skills 市场

通用网页抓取工具

Name: 通用网页抓取工具
Author: wuxiangfacai

作者 wuxiangfacai · GitHub ↗ · v1.0.0 · MIT-0

cross-platform ✓ 安全检测通过

总下载

当前安装

版本数

在 OpenClaw 中安装

/install general-web-scraper

功能描述

通用网页数据抓取工具 — 支持CSS选择器抓取链接、表格数据提取，输出CSV/JSON格式。无需配置，开箱即用。

使用说明 (SKILL.md)

Web Scraper — 通用网页数据抓取工具

AI agent 专用的网页数据抓取工具。输入网址和CSS选择器，自动抓取链接或表格数据，导出为CSV或JSON。

功能

链接抓取 — 抓取页面中所有匹配CSS选择器的链接
表格抓取 — 自动提取HTML表格数据
CSV导出 — 默认输出CSV格式
JSON导出 — 支持JSON格式输出
中文友好 — 完整支持中文网页编码

使用方式

# 抓取页面所有链接
python scraper.py https://example.com

# 自定义CSS选择器
python scraper.py https://example.com "a.article-link"

# 导出JSON格式
python scraper.py https://example.com "div.item" --json

# 抓取表格数据
python scraper.py https://example.com "table#data" --table --json

依赖安装

pip install requests beautifulsoup4

适用场景

数据采集和调研
竞品信息监控
市场情报收集
内容聚合

Tags

scraping, web, data, python, automation, crawler, data-collection

安全使用建议

Before installing, confirm you are allowed to collect from the configured sources, avoid private or sensitive data, keep exported reports in a controlled location, and review any scheduled automation so it only runs when and where you expect.

能力评估

ℹ Purpose & Capability

The reported behavior, scraping or collecting content and exporting a report, matches a trend-monitoring/content-research purpose. The main caveat is privacy and site-policy handling, not evidence of deception or unrelated capability.

ℹ Instruction Scope

The instructions appear to encourage automated collection and export, but the supplied evidence does not show hidden commands, unrelated data access, credential theft, or destructive behavior.

✓ Install Mechanism

No artifact-backed installer concern was supplied; VirusTotal telemetry is clean and the reported finding points to SKILL.md guidance rather than an executable installer.

ℹ Credentials

Network scraping and saved exports are proportionate for a content research skill, but users should scope sources, avoid sensitive/private data, and respect robots, terms, and applicable law.

ℹ Persistence & Privilege

Automated or recurring scraping can create persistence-like behavior if scheduled by the user or agent, but the supplied evidence does not show hidden background workers or privilege escalation.

如何使用

确保已安装 OpenClaw（本地或 Docker 部署）
在对话框中输入安装命令：/install general-web-scraper
安装完成后，直接呼叫该 Skill 的名称或使用 /general-web-scraper 触发
根据 Skill 的参数说明提供必要输入，即可获得结构化输出

版本历史

v1.0.0

首次发布：支持CSS选择器链接抓取、表格提取、CSV/JSON导出

元数据

Slug general-web-scraper

版本 1.0.0

许可证 MIT-0

累计安装 1

当前安装数 1

历史版本数 1

常见问题