imsg-media

Full iMessage multimedia pipeline:

• 🎙️ Voice memo → text via Silicon Flow ASR (SenseVoiceSmall, cloud, no local model)
• 🖼️ Image → description/OCR via agent's built-in vision model

Requirements

macOS permissions

• Full Disk Access must be granted to the process running OpenClaw
• Settings → Privacy & Security → Full Disk Access → add your terminal/app
• Without this, imsg cannot read ~/Library/Messages/chat.db and will return permissionDenied

API key (audio only)

• Silicon Flow API key — sign up free at https://siliconflow.cn
• Long-term use: add to ~/.openclaw/.env: SILICON_FLOW_KEY=sk-...
• Quick test / override: pass --api-key sk-... directly to the script
• Image analysis does not require this key

CLI dependency

• imsg CLI: npm install -g imsg

Trigger conditions

Activate this skill when:

• Incoming message text contains the attachment placeholder 
• User says "语音转文字", "转写", "识别语音", "transcribe"
• User says "看图", "识别图片", "读图", "OCR", "截图里写的什么"
• User references a photo/audio/file they just sent via iMessage

Decision flow

Attachment detected?
├── Audio (.m4a / .caf / .wav / .mp3)  → transcribe via Silicon Flow ASR
├── Image (.jpg / .png / .heic / .gif) → read with vision model
└── Unknown / not downloaded            → increase --limit or ask user to resend

Workflow

Step 1 — Get the sender identifier

Always read from the message envelope:

• [iMessage [email protected] ...] → use [email protected]
• [SMS +1234567890 ...] → use +1234567890
• Never hardcode an address

Step 2 — Fetch the attachment

# Run from the skill directory
cd ~/.openclaw/skills/imsg-voice-transcribe

python3 scripts/imsg_voice_transcribe.py fetch \
  --identifier "[email protected]" \
  --limit 50

Returns JSON with file, type (audio or image), and metadata.

If nothing found, try --limit 100.

Step 3a — Audio: transcribe

# One-liner (fetch + transcribe)
python3 scripts/imsg_voice_transcribe.py auto \
  --identifier "[email protected]" \
  --limit 50 --raw

# Or transcribe a specific file
python3 scripts/imsg_voice_transcribe.py transcribe \
  --file /path/to/audio.m4a --raw

# Quick test with explicit API key (no env setup needed)
python3 scripts/imsg_voice_transcribe.py transcribe \
  --file /path/to/audio.m4a --api-key sk-... --raw

Step 3b — Image: analyze

After fetch returns an image path (e.g. {"file": "/path/to/photo.jpg", "type": "image"}):

# Example: fetch image from a sender
python3 scripts/imsg_voice_transcribe.py fetch \
  --identifier "[email protected]" --type image --limit 50
# → {"file": "/Users/.../Messages/Attachments/photo.jpg", "type": "image", ...}

Then in the agent:

1. If HEIC/HEIF: convert first → sips -s format png "input.heic" --out "output.png"
2. Open with the read tool → agent vision model processes it
3. Respond with: what it is, main subject, any text/OCR, notable details

Default image response format:

• What it is: photo / screenshot / document
• Main subject: 1–2 sentences
• Text (OCR): quote key text, or "无明显文字"
• Details: 3–5 bullets
• Follow-up: ask if they want OCR / table extraction / comparison / etc.

Supported formats

Troubleshooting