luci-memory

Setup

Requires an MEMORIES_AI_KEY. On first use, if no key is found, the script will error and ask for one.

When the user provides their key, save it to {baseDir}/.env:

MEMORIES_AI_KEY=sk-their-key-here

After that, everything just works — the key is loaded automatically from .env on every run.

Timezone

All timestamps in Luci-memory are stored and returned in UTC. Skill output labels them with " UTC" so this is unambiguous. The user's local timezone is in USER.md (e.g. Asia/Shanghai). You are responsible for converting in both directions:

1. Reading results. When presenting captured_time to the user, convert from UTC to the user's local timezone. Never show raw UTC labels to the user.

2. Writing filters. --after and --before are interpreted as UTC. If the user says relative dates like "yesterday" or "this morning", convert their local-time intent to a UTC range before passing the dates.

Example (user in Asia/Shanghai, UTC+8, asks "what did I do yesterday" on 2026-04-08):

• Local intent: 2026-04-07 00:00 → 2026-04-08 00:00 (Asia/Shanghai)
• UTC range to pass: --after 2026-04-06T16:00:00 --before 2026-04-07T16:00:00

If USER.md has no timezone and the user uses relative dates, ask them first.

Unified search across personal media and portrait data from the Luci-memory API.

The user's videos go through two processing pipelines that produce different data:

• Media content (personal): video summaries, audio transcripts, visual transcripts, keyframes, images
• People & knowledge (portrait): traits, events with participants, relationships, speeches attributed to speakers

When to use

• User asks to find or search videos, images, or photos
• User asks what was said or shown in a video
• User asks to list recent videos or images
• User asks about media at a specific location or time
• User asks about traits, personality, hobbies, interests
• User asks what events happened, or events involving specific people
• User asks about relationships between people
• User asks about what someone said
• User mentions "luci memory" or wants to use their video memory

Choosing the right type

• About content (what happened, what was said/shown, find media) → use media types (search_video, query_audio, etc.)
• About people (who, traits, relationships, named individuals) → use portrait types (traits, events, speeches, etc.)
• Ambiguous questions like "What happened with Alice last week?" → use both: portrait types to identify the person and events, media types to get detailed video content and transcripts.
• Person name fallback: Portrait data only exists for people who have appeared in at least 5 videos AND been named by the user in the app. If a portrait query by person name returns no results, fall back to media types — search video summaries, audio transcripts, or visual transcripts for mentions of that name instead.

Relevance guidelines

• There is no rerank process — retrieved results may contain items irrelevant to the user's actual intent.
• Always verify relevance: after receiving results, check each item against the user's original query. Only present results that are relevant. Discard anything that doesn't match.
• Refine and retry: if results seem off or too broad, retry with a more specific query, narrower date range, or additional filters. Do not just dump low-quality results to the user.
• Ask the user: if the query is ambiguous or too vague to produce good results, ask the user for more specific conditions before searching. It is better to clarify than to return noise. Do this no more than 1 time.

No hallucination — ground every claim in retrieved data

• Never fabricate what the user did, said, or experienced. Every detail in your answer must come from actual search results.
• Multi-step retrieval: for questions like "what did I do and say at XXX", do NOT answer from a single broad search. Follow this pattern:
  1. Locate: search broadly (search_video, search_events) to find relevant video_ids or event_ids.
  2. Retrieve: once you have IDs, prefer query_audio / query_visual with --video-ids to get complete transcripts. You can also use search_audio / search_visual scoped to those video IDs to find specific moments — use both flexibly as needed.
• Do not stuff keywords into search queries. Each semantic search query should be a short, coherent natural-language query, rather than stacking multiple possible words. You are encouraged to try different ones and query various times though.
• If data is missing, say so. Do not fill gaps with plausible-sounding guesses. "I couldn't find transcript data for that video" is always better than making something up.

How to invoke

Note: --after / --before are UTC. Convert from the user's local timezone first (see Timezone section above).

Returning Images/Keyframes to User

When search results include signed URLs (keyframes, images), follow this pipeline to send them in chat:

1. Download the signed URL to the workspace:

   curl -sL -o /path/to/workspace/image.jpg "\x3Csigned_url>"
   

2. Send via OpenClaw message CLI:
openclaw message send --channel \x3Cchannel> --target \x3Cchat_id> --media /path/to/workspace/image.jpg --message "caption"
3. Cleanup the file after sending:
rm /path/to/workspace/image.jpg
⚠️ Signed URLs expire after ~1 hour. Download promptly.
⚠️ Do NOT use /tmp or paths outside the workspace — some tools block external paths.
⚠️ The image tool only analyzes images — it cannot send them to the user. Use openclaw message send --media instead.

# ============ Media content (personal) ============

# --- Video ---
bash {baseDir}/run.sh --query "cooking in kitchen" --type search_video
bash {baseDir}/run.sh --query "what did I do" --type search_video --location "Heze"
bash {baseDir}/run.sh --query "meeting" --type search_video --after 2025-12-01 --before 2026-01-01
bash {baseDir}/run.sh --type query_video
bash {baseDir}/run.sh --type query_video --location "Suzhou" --after 2025-12-01

# --- Image ---
bash {baseDir}/run.sh --query "sunset" --type search_image
bash {baseDir}/run.sh --query "food" --type search_image --location "Beijing"
bash {baseDir}/run.sh --type query_image

# --- Audio Transcripts (what was said) ---
bash {baseDir}/run.sh --query "talking about work" --type search_audio
bash {baseDir}/run.sh --query "budget" --type search_audio --video-ids VI123,VI456
bash {baseDir}/run.sh --type query_audio --video-ids VI123,VI456

# --- Visual Transcripts (what was shown) ---
bash {baseDir}/run.sh --query "walking in park" --type search_visual
bash {baseDir}/run.sh --type query_visual --video-ids VI123,VI456

# --- Keyframes ---
bash {baseDir}/run.sh --query "person waving" --type search_keyframe
bash {baseDir}/run.sh --type query_keyframe --video-ids VI123,VI456

# ============ People & knowledge (portrait) ============

# --- Traits ---
bash {baseDir}/run.sh --type traits
bash {baseDir}/run.sh --type traits --person "Alice"
bash {baseDir}/run.sh --query "outdoor activities" --type search_traits

# --- Events ---
bash {baseDir}/run.sh --type events
bash {baseDir}/run.sh --type events --person "Alice"
bash {baseDir}/run.sh --type events --person "Alice,Bob"
bash {baseDir}/run.sh --type events --after 2025-12-01 --before 2026-01-01
bash {baseDir}/run.sh --query "cooking in kitchen" --type search_events
bash {baseDir}/run.sh --query "meeting" --type search_events --person "Bob" --after 2025-12-01

# --- Relationships ---
bash {baseDir}/run.sh --type relationships
bash {baseDir}/run.sh --type relationships --person "Alice"

# --- Speeches ---
bash {baseDir}/run.sh --type speeches
bash {baseDir}/run.sh --type speeches --person "Alice"
bash {baseDir}/run.sh --type speeches --event-ids EVT123,EVT456
bash {baseDir}/run.sh --type speeches --person "Alice" --event-ids EVT123

Parameters

Signed URLs

Image and keyframe results include a signed_url field — a temporary (1-hour) direct link to view/download from Google Cloud Storage. No authentication needed, but they expire after 1 hour.

Types reference

Media search types (require --query)

Media query types (list/filter)

Portrait query types (list/filter)

Portrait search types (semantic, require --query)