← Back to Skills Marketplace
Vector Store Shootout
by
Nissan Dookeran
· GitHub ↗
· v1.0.0
371
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install vector-store-shootout
Description
8 vector store implementations behind a common interface — numpy, lancedb, qdrant, pgvector, weaviate, weaviate_hybrid, milvus, lightrag. Use when evaluating...
Usage Guidance
This skill appears to implement the vector stores it claims, but there are multiple practical inconsistencies you should address before installing or running it:
- Dependencies: The code requires many Python packages (requests, numpy, networkx, lancedb, qdrant_client, psycopg2/pymilvus/pyarrow/weaviate-client, etc.) but the skill only declares python3. Don't run it on a production host without creating a pinned virtual environment (venv/conda) and installing and auditing those packages first.
- Network vs local: The SKILL metadata implies no outbound networking, but the code calls embedding endpoints (default: local Ollama at http://localhost:11434) and optionally OpenAI (api.openai.com). If you want purely local operation, run an Ollama instance on localhost and avoid supplying OpenAI keys. If you supply an OpenAI key or use remote DB backends, data (the texts you index/query) will be sent to those services.
- Secrets: The skill doesn't declare required env vars, yet it will accept and use an OpenAI API key if given. Only provide credentials if you trust the code and run it in an isolated environment; do not pass secrets you wouldn't want used for remote embedding/exfiltration.
- Resource effects: Backends create temporary files, DB tables, and collections and delete them on cleanup; verify these operations are acceptable in your environment (especially if you point to an existing Postgres instance or other shared service).
- Safety steps: Run the skill in a disposable VM or container first, pin dependency versions, inspect network traffic to ensure embeddings are sent to endpoints you expect, and consider providing a local embed_fn (test injection) to avoid network calls during evaluation. If you need a complete dependency/install spec and clearer network/credential documentation, request it from the publisher before using in production.
Capability Analysis
Type: OpenClaw Skill
Name: vector-store-shootout
Version: 1.0.0
The skill bundle implements various vector store backends but contains a significant security vulnerability and a metadata discrepancy. Specifically, `scripts/pgvector_store.py` is vulnerable to SQL injection because it uses f-strings to interpolate the `table_name` directly into SQL queries. Furthermore, the `SKILL.md` file incorrectly claims no outbound network access (`network: outbound: false`), whereas multiple files (e.g., `scripts/numpy_store.py`, `scripts/weaviate_store.py`) contain logic to send data to the external OpenAI API (`api.openai.com`) for embedding generation.
Capability Assessment
Purpose & Capability
The skill name/description (eight vector store implementations behind a common interface) aligns with the included code files: numpy, lancedb, qdrant, pgvector, weaviate, milvus, lightrag, etc. Implementations perform expected actions for indexing/search and (where appropriate) persistent storage and cleanup. However, the SKILL metadata only requires python3 while the code imports many third-party libraries (requests, lancedb, qdrant_client, qdrant_client.models, psycopg2, pymilvus, networkx, pyarrow, numpy, etc.), so the declared requirements are incomplete relative to the code.
Instruction Scope
SKILL.md metadata states outbound networking is false / 'All backends run locally', but multiple store implementations make HTTP calls (requests.post to Ollama at http://localhost:11434 and to https://api.openai.com), and client libraries connect to networked services (Qdrant, Weaviate, Milvus, Postgres). The code will write temp directories, create/drop DB tables and collections, and may delete those resources on cleanup — expected for DB backends, but the network claim is misleading. The runtime instructions do not request any credentials explicitly, yet the code supports using an OpenAI API key and will send user text to embedding endpoints if configured, which means user data could leave the host depending on deployment.
Install Mechanism
There is no install spec despite many non-standard runtime dependencies. The skill is distributed as code files but does not declare or install required Python packages, increasing friction and risk (users may install libraries ad-hoc or run code without needed packages). Absence of a pinned dependency list or install steps is disproportionate to the task complexity and makes it unclear what will be installed or required on the host.
Credentials
The registry metadata declares no required environment variables or primary credential, but implementations accept and use an optional OpenAI API key parameter and will call remote embedding services (OpenAI) if provided. The mismatch between 'no credentials required' and code that will use credentials if supplied is confusing and could cause users to inadvertently supply a secret to a skill that didn't declare it. Additionally, network usage is environment-dependent but not made explicit in required configuration.
Persistence & Privilege
The skill does not request elevated platform privileges, does not set always:true, and does not modify other skills. It creates temporary directories and persistent stores (LanceDB, Milvus Lite files, or database tables) as part of backend operation and provides cleanup methods that drop those resources — this behavior is expected for database backends and is scoped to the skill's own resources.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install vector-store-shootout - After installation, invoke the skill by name or use
/vector-store-shootout - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release of vector-store-shootout
- Provides 8 interchangeable vector store backends under a unified Python interface: numpy, lancedb, qdrant, pgvector, weaviate, weaviate_hybrid, milvus, and lightrag.
- Enables easy benchmarking, evaluation, and backend-switching for RAG and vector search applications.
- Simple base class: implement add, search, and delete methods for consistency across backends.
- Includes finding: weaviate hybrid search at alpha=0.1 delivers better results for technical/keyword-heavy content compared to default settings.
- All backends can run locally; outbound network access depends on deployment configuration.
Metadata
Frequently Asked Questions
What is Vector Store Shootout?
8 vector store implementations behind a common interface — numpy, lancedb, qdrant, pgvector, weaviate, weaviate_hybrid, milvus, lightrag. Use when evaluating... It is an AI Agent Skill for Claude Code / OpenClaw, with 371 downloads so far.
How do I install Vector Store Shootout?
Run "/install vector-store-shootout" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Vector Store Shootout free?
Yes, Vector Store Shootout is completely free (open-source). You can download, install and use it at no cost.
Which platforms does Vector Store Shootout support?
Vector Store Shootout is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Vector Store Shootout?
It is built and maintained by Nissan Dookeran (@nissan); the current version is v1.0.0.
More Skills