← 返回 Skills 市场
ucsdzehualiu

free-web-search-js

作者 ucsdzehualiu · GitHub ↗ · v1.0.0 · MIT-0
cross-platform ⚠ suspicious
68
总下载
0
收藏
0
当前安装
1
版本数
在 OpenClaw 中安装
/install free-web-search-js
功能描述
Playwright驱动的联网搜索工具，自动抓取前三条网页内容，无需API Key，支持国内Bing和海外DDG搜索。
使用说明 (SKILL.md)

SKILL.md

name: free-web-search-js description: Playwright 联网搜索，自动抓取内容，零 API Key version: 28.0.0 trigger_keywords:

  • 搜索
  • 查一下
  • 找一下
  • 最新消息
  • 新闻
  • 教程
  • 是什么
  • search
  • find tools:
  • name: search description: 搜索+自动抓取，国内Bing Playwright，海外DDG HTTP script: scripts/search.js parameters: query: type: string description: "搜索关键词" required: true max: type: integer description: "最大结果数，默认10，上限30" required: false region: type: string description: "区域: auto/cn/intl，默认auto按IP检测" required: false
  • name: fetch description: 给定URL抓取正文，HTTP优先失败自动headed兜底 script: scripts/fetch.js parameters: urls: type: string description: "要抓取的URL，多个用空格分隔" required: true max-len: type: integer description: "单页最大字符数，默认12000" required: false

free-web-search-js

一步式：search → Playwright 搜 → 自动抓内容 → 返回

架构

国内：
  Playwright 打开 Bing → 首页拿 cookie → 搜索框提交
  → 自动抓取 top 3 页面内容
  延迟：首次 3~6s（启动浏览器），后续复用更快

海外：
  纯 HTTP → DDG HTML 解析
  → 自动抓取 top 3 页面内容
  延迟：几百ms~1s

搜索引擎

引擎 协议 区域 说明
Bing CN Playwright 搜索框提交 国内 先访问首页拿 cookie，再搜索框输入提交
搜狗 纯 HTTP 国内 --engine=sogou 可选，⚠ 无 cookie 易被反爬拦截，结果不稳定
DDG HTML Lite 纯 HTTP 海外 html.duckduckgo.com

策略

区域 搜索 抓取
国内 Bing CN (Playwright) 自动抓前 3 条
海外 DDG HTML 自动抓前 3 条

IP 怎么判断

每次搜索时自动检测，三轮探测并行，谁先成功用谁：

轮次 探测服务 逻辑
第1轮 myip.ipip.net / cip.cc 国内可达优先
第2轮 ipinfo.io / ipapi.co 国际探测
第3轮 试连 cn.bing.com 能通大概率国内
兜底 — 默认国内

出口 IP 走代理时可能误判，用 --region=cn 或 --region=intl 手动指定。

去重

智能去重：域名 + 路径主干（忽略 www/m 子域、tracking 参数、尾部斜杠、.html 后缀）。

Bing 跳转 URL（bing.com/ck/）自动解码为直链。

抓取模式

搜索后自动抓取 top N 条 URL 内容（默认 3 条）。

层级 方式 速度 说明
第1层 轻量 HTTP + cheerio ⚡ 秒出 不启动浏览器
第2层 Playwright headed 🟡 慢 完整浏览器，支持 JS 渲染

第1层增强：

  • JSON API 响应：自动检测 Content-Type 并提取结构化内容
  • JSON-LD：提取 \x3Cscript type="application/ld+json"> 中的 articleBody/description
  • NEXT_DATA：提取 Next.js 嵌入数据
  • meta 标签：og:description / description 兜底
  • GBK 编码：自动检测并转换

安装

前置依赖（全部必装）：

依赖 说明 大小/耗时
Node.js >= 18 运行时 —
cheerio HTML 解析 小，秒装
commander CLI 参数解析 小，秒装
iconv-lite GBK 编码转换 小，秒装
playwright 浏览器自动化（Bing 搜索 + 抓取兜底） ~50MB
Chromium Playwright 专用浏览器 ~150MB，需几分钟下载

安装脚本自动检测网络区域，国内使用镜像源加速：

# Windows
powershell -File scripts/setup.ps1

# Linux/macOS
bash scripts/setup.sh

国内镜像：

  • npm: https://registry.npmmirror.com
  • Playwright/Chromium: https://npmmirror.com/mirrors/playwright

手动安装：

cd skills/free-web-search-js
npm install
npx playwright install chromium    # ~150MB，需几分钟

验证环境：node scripts/check-env.js

卸载：node scripts/uninstall.js

性能优化：浏览器守护进程

搜索和抓取可复用浏览器守护进程，提速约 70%：

node scripts/browser-daemon.js &       # 启动
node scripts/browser-daemon.js --status # 状态
node scripts/browser-daemon.js --stop   # 停止

守护进程空闲 10 分钟自动退出。

用法

# 搜索（搜 + 自动抓前3条内容）
node scripts/search.js "白银价格"
node scripts/search.js "how to deploy docker" --max=5
node scripts/search.js "xxx" --region=cn
node scripts/search.js "xxx" --fetch=5          # 抓前5条
node scripts/search.js "xxx" --no-fetch         # 只搜不抓

# 单独抓取（给定 URL）
node scripts/fetch.js "https://example.com/page1" "https://example.com/page2"

已知限制

  • 国内首次搜索较慢：需启动 Chromium（3~6s），后续复用更快
  • Bing CN 即时答案不返回：天气、计算器等即时卡片不走 li.b_algo，搜索结果为 0
  • 搜狗 HTTP 不稳定：无 cookie 纯请求易被反爬拦截，结果可能为空（--engine=sogou 慎用）
  • 部分站点 HTTP 抓不到：需要 JS 渲染的页面——HTTP 失败会自动 headed 重试
  • 部分站点海外不可达：国内专属站点从海外访问可能超时
  • 代理干扰 IP 检测：出口 IP 走代理时可能误判区域，用 --region=cn/intl 手动指定
  • 海外引擎国内不可达：DDG 在国内被墙，国内策略不使用
安全使用建议
This skill appears to do what it says: it runs a Node.js program that queries public IP services and search engines, fetches and parses top result pages, and (optionally) launches/reuses a headful Chromium via Playwright. Before installing, consider: 1) it will download/run Playwright and a ~150MB Chromium browser and may run npm install (network downloads) if dependencies are missing — run in a controlled environment (container/VM) if you have security concerns; 2) the skill will make outbound network requests to multiple third-party services (ipip, cip.cc, ipinfo, ipapi, Bing, DDG, target pages) as part of normal operation; 3) it writes a .browser-endpoint file in the skill directory and can spawn a background process (browser daemon) — stop it when not needed; 4) the source/homepage is unknown, so if you require provenance, review the code locally before running or run it in an isolated environment. If you accept those behaviors, the profile is coherent and consistent with its description.
功能分析
Type: OpenClaw Skill Name: free-web-search-js Version: 1.0.0 The skill bundle provides a functional web search and content scraping tool using Playwright and HTTP. However, it contains a critical command injection vulnerability in `scripts/search.js`, where search result URLs are concatenated into a command string and executed via `child_process.execSync` without any sanitization or escaping. This flaw could allow a malicious search result (e.g., a URL containing shell metacharacters) to achieve Remote Code Execution (RCE) on the host system. While the behavior appears to be a result of poor coding practices rather than intentional malice, the high risk of exploitation warrants a suspicious classification.
能力标签
cryptorequires-sensitive-credentials
能力评估
✓ Purpose & Capability
Name/description (Playwright-driven search + fetch) matches the included files (search.js, fetch.js, browser-daemon.js, setup script, package.json). Dependencies and the need to download Chromium are expected for this functionality. No unrelated credentials or system-level config paths are requested.
ℹ Instruction Scope
SKILL.md and the scripts instruct the agent to perform network calls (search engines and IP probe services), fetch and parse web pages, and optionally run a persistent Chromium daemon. The instructions and code operate within the skill directory (writes .browser-endpoint) and do not attempt to read unrelated system files. Note: runtime helper functions will auto-run npm install for missing dependencies (execSync), and the tool will contact external IP detection services and search engines as part of normal operation.
ℹ Install Mechanism
No platform-level install spec is declared, but SKILL.md includes manual setup steps (npm install, npx playwright install chromium) and provides a setup.sh. The code will attempt to install missing npm packages at runtime via child_process.execSync if dependencies are absent — this is convenient but means the skill can fetch and install packages from npm when first run.
✓ Credentials
The skill declares no required environment variables or credentials and only reads standard environment locations (HOME, LOCALAPPDATA) to find Playwright caches; those uses are consistent with the stated purpose. It does not request tokens/secrets or unusual system config paths.
ℹ Persistence & Privilege
always:false and model invocation not disabled (normal). The skill can start a long-running Chromium daemon and create a .browser-endpoint file in its own directory to enable browser reuse; this is a self-contained persistence mechanism but does not modify other skills or system-wide agent settings.
如何使用
  1. 确保已安装 OpenClaw（本地或 Docker 部署）
  2. 在对话框中输入安装命令：/install free-web-search-js
  3. 安装完成后，直接呼叫该 Skill 的名称或使用 /free-web-search-js 触发
  4. 根据 Skill 的参数说明提供必要输入，即可获得结构化输出
版本历史
v1.0.0
Initial release of free-web-search-js, a one-step Playwright-based web search and content scraping tool, requiring no API keys. - Supports both domestic (Bing CN via Playwright) and overseas (DDG via HTTP) search engines, auto-detecting region and using appropriate engines/strategies. - Automatically crawls and summarizes top 3 search results after searching. - Intelligent duplication removal based on core URLs, with Bing redirect decoding. - Adaptive content extraction: handles API responses, JSON-LD, Next.js data, meta tags, and GBK encoding. - Performance-optimized with optional browser daemon for faster repeated searches. - CLI tools for both search+fetch and standalone fetching by URL. - Includes robust region/IP detection and fallback strategies; manual override available.
元数据
Slug free-web-search-js
版本 1.0.0
许可证 MIT-0
累计安装 0
当前安装数 0
历史版本数 1
常见问题

free-web-search-js 是什么？

Playwright驱动的联网搜索工具，自动抓取前三条网页内容，无需API Key，支持国内Bing和海外DDG搜索。 它是一个面向 Claude Code / OpenClaw 的 AI Agent Skill 插件，目前累计下载 68 次。

如何安装 free-web-search-js？

在 OpenClaw 或 Claude Code 对话框中运行命令「/install free-web-search-js」即可一键安装，无需额外配置。

free-web-search-js 是免费的吗？

是的，free-web-search-js 完全免费，采用 MIT-0 许可证，可自由下载、安装和使用。

free-web-search-js 支持哪些平台？

free-web-search-js 跨平台运行，可在任意部署了 OpenClaw / Claude Code 的环境中使用（cross-platform）。

谁开发了 free-web-search-js？

由 ucsdzehualiu（@ucsdzehualiu）开发并维护，当前版本 v1.0.0。

推荐 Skills
self-improving agent
Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Clau...
⬇ 463368 · by pskoett
Skill Vetter
Security-first skill vetting for AI agents. Use before installing any skill from ClawdHub, GitHub, or other sources. Checks for red flags, permission scope, and suspicious patterns.
⬇ 259467 · by spclaudehome
Polymarket
Query Polymarket prediction markets. Check odds, find trending markets, search events, track price movements.
⬇ 232503 · by joelchance
Self-Improving + Proactive Agent
Self-reflection + Self-criticism + Self-learning + Self-organizing memory. Agent evaluates its own work, catches mistakes, and improves permanently. Use when...
⬇ 200457 · by ivangdavila
Github
Interact with GitHub using the `gh` CLI. Use `gh issue`, `gh pr`, `gh run`, and `gh api` for issues, PRs, CI runs, and advanced queries.
⬇ 191163 · by steipete
ontology
Typed knowledge graph for structured agent memory and composable skills. Use when creating/querying entities (Person, Project, Task, Event, Document), linkin...
⬇ 190189 · by oswalpalash

💬 留言讨论