← Back to Skills Marketplace
zt-web-fetcher
by
larriewong27
· GitHub ↗
· v1.0.0
· MIT-0
256
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install zt-web-fetcher
Description
当需要获取网页内容时使用(搜索结果页面、博客文章、文档等)。使用 URL 转 Markdown 服务将网页转换为可读文本。 触发场景:用户说"帮我查一下"、"看看这个链接"、"获取 xx 的信息"等需要上网抓信息的情况。
README (SKILL.md)
网页抓取
当需要获取网页内容时,按以下顺序尝试:
首选方案:URL 转 Markdown 服务
-
markdown.new/ (推荐)
- 用法:在网址前加
https://markdown.new/ - 例如:
https://markdown.new/https://example.com - 适合 Cloudflare 支持的网站
- 用法:在网址前加
-
r.jina.ai/ (备用)
- 用法:在网址前加
https://r.jina.ai/ - 例如:
https://r.jina.ai/https://example.com - 兼容性好
- 用法:在网址前加
-
defuddle.md/ (备用)
- 用法:在网址前加
https://defuddle.md/ - 例如:
https://defuddle.md/https://example.com
- 用法:在网址前加
备选方案:爬虫工具
如果以上服务都无法获取,尝试 Scrapling:
- 地址:https://github.com/D4Vinci/Scrapling
- 用法:
pip install scrapling后使用
使用流程
- 先尝试
r.jina.ai/{url}(最稳定) - 如果失败,尝试
markdown.new/{url} - 再失败,尝试
defuddle.md/{url} - 都失败再考虑 Scrapling
注意事项
- 不需要配置任何搜索 API
- 这些服务会将网页转换为纯文本/ Markdown 格式
- 适合大多数静态网页
- 对于需要登录的页面可能无效
Usage Guidance
This skill is coherent for fetching and converting webpages, but be aware of two practical risks before using it: (1) The URL-to-Markdown services (r.jina.ai, markdown.new, defuddle.md) will fetch the given URL themselves, so do not send private, authenticated, or sensitive URLs you don't want a third party to see. (2) The suggested fallback ('pip install scrapling' from GitHub) would run third-party code in your environment—review that repository and prefer installing only vetted packages. If you need to fetch pages behind login or that contain sensitive data, do not rely on these public converters; instead use a controlled, audited crawler or provide the content directly.
Capability Analysis
Type: OpenClaw Skill
Name: zt-web-fetcher
Version: 1.0.0
The skill bundle provides instructions for an AI agent to fetch web content by utilizing legitimate third-party URL-to-Markdown conversion services such as r.jina.ai, markdown.new, and defuddle.md. It also suggests the use of a well-known open-source Python library (Scrapling) as a fallback. The instructions in SKILL.md are clearly aligned with the stated purpose of web fetching and do not contain any indicators of malicious intent, data exfiltration, or unauthorized execution.
Capability Assessment
Purpose & Capability
Name/description match the SKILL.md: the skill converts webpages to Markdown using r.jina.ai, markdown.new, defuddle.md and—if those fail—suggests a Scraping fallback. It requests no credentials, binaries, or config paths, which is consistent with its stated purpose.
Instruction Scope
Instructions stay within the scraping/formatting scope, but they direct the agent to fetch webpages via third-party URL-to-Markdown services (r.jina.ai, markdown.new, defuddle.md). Those services will receive the target URL (and will fetch page content), so private pages or sensitive URLs would be exposed to third parties. The SKILL.md also suggests 'pip install scrapling' as a fallback—this would cause code to be installed/executed in the runtime environment and should be vetted before use. There is a minor internal inconsistency in which service is 'preferred' vs the ordered try sequence.
Install Mechanism
There is no formal install spec (instruction-only), which is low risk. The fallback recommendation to 'pip install scrapling' points to a GitHub project; installing that package would execute third-party code from a repo (moderate risk) and should be reviewed before installation. The primary flows (using URL-to-Markdown endpoints) do not require installing code.
Credentials
No environment variables, credentials, or config paths are requested. The skill does not ask for unrelated secrets or high-privilege access.
Persistence & Privilege
always is false and there is no install-time modification or persistent configuration. The skill does not request elevated or permanent presence beyond normal invocation.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install zt-web-fetcher - After installation, invoke the skill by name or use
/zt-web-fetcher - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Version 1.0.0 of zt-web-fetcher
- Initial release with detailed instructions for fetching web content using URL-to-Markdown services.
- Describes preferred services (markdown.new, r.jina.ai, defuddle.md) and fallback options.
- Outlines step-by-step usage process and fallback to Scrapling tool if needed.
- Includes usage scenarios and important notes for compatibility and limitations.
- No requirement for search API configuration.
Metadata
Frequently Asked Questions
What is zt-web-fetcher?
当需要获取网页内容时使用(搜索结果页面、博客文章、文档等)。使用 URL 转 Markdown 服务将网页转换为可读文本。 触发场景:用户说"帮我查一下"、"看看这个链接"、"获取 xx 的信息"等需要上网抓信息的情况。 It is an AI Agent Skill for Claude Code / OpenClaw, with 256 downloads so far.
How do I install zt-web-fetcher?
Run "/install zt-web-fetcher" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is zt-web-fetcher free?
Yes, zt-web-fetcher is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does zt-web-fetcher support?
zt-web-fetcher is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created zt-web-fetcher?
It is built and maintained by larriewong27 (@larriewong27); the current version is v1.0.0.
More Skills