inSaiAI Intelligent Editing

Comprehensive guide for professional video and audio manipulation using FFmpeg and FFprobe.

Core Concepts

FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play almost anything that humans and machines have created. It is a command-line tool that processes streams through a complex pipeline of demuxers, decoders, filters, encoders, and muxers.

Common Operations

# Basic Transcoding (MP4 to MKV)
ffmpeg -i input.mp4 output.mkv

# Change Video Codec (to H.265/HEVC)
ffmpeg -i input.mp4 -c:v libx265 -crf 28 -c:a copy output.mp4

# Extract Audio (No Video)
ffmpeg -i input.mp4 -vn -c:a libmp3lame -q:a 2 output.mp3

# Resize/Scale Video
ffmpeg -i input.mp4 -vf "scale=1280:720" output.mp4

# Cut Video (Start at 10s, Duration 30s)
ffmpeg -i input.mp4 -ss 00:00:10 -t 00:00:30 -c copy output.mp4

# Fast Precise Cut (Re-encoding only the cut points is complex, so standard re-encoding is safer for precision)
ffmpeg -ss 00:00:10 -i input.mp4 -to 00:00:40 -c:v libx264 -crf 23 -c:a aac output.mp4

# Concatenate Files (using demuxer)
# Create filelist.txt: file 'part1.mp4' \
 file 'part2.mp4'
ffmpeg -f concat -safe 0 -i filelist.txt -c copy output.mp4

# Speed Up/Slow Down Video (2x speed)
ffmpeg -i input.mp4 -filter_complex "[0:v]setpts=0.5*PTS[v];[0:a]atempo=2.0[a]" -map "[v]" -map "[a]" output.mp4

Processing Categories & When to Use

Codecs & Quality

Filters & Manipulation

Inspection & Metadata

Advanced: Complex Filtergraphs

Use filter_complex when you need to process multiple inputs or create non-linear filter chains.

# Example: Adding a watermark at the bottom right
ffmpeg -i input.mp4 -i watermark.png -filter_complex "overlay=main_w-overlay_w-10:main_h-overlay_h-10" output.mp4

# Example: Vertical Stack (2 videos)
ffmpeg -i top.mp4 -i bottom.mp4 -filter_complex "vstack=inputs=2" output.mp4

# Example: Side-by-Side (2 videos)
ffmpeg -i left.mp4 -i right.mp4 -filter_complex "hstack=inputs=2" output.mp4

# Example: Grid (4 videos 2x2)
ffmpeg -i v1.mp4 -i v2.mp4 -i v3.mp4 -i v4.mp4 -filter_complex "[0:v][1:v]hstack=inputs=2[top];[2:v][3:v]hstack=inputs=2[bottom];[top][bottom]vstack=inputs=2" output.mp4

# Example: Fade Transition (Simple crossfade between two clips)
# Requires manual offset calculation, using xfade is better
ffmpeg -i input1.mp4 -i input2.mp4 -filter_complex "xfade=transition=fade:duration=1:offset=9" output.mp4

Pro Editing Tips & Techniques

1. High-Quality GIF Creation

Standard conversion often results in poor colors. Use a palette for best results:

ffmpeg -i input.mp4 -vf "fps=15,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif

2. Audio Mixing (Background Music + Voice)

Mix background music at 30% volume with the main audio:

ffmpeg -i voice.mp4 -i bgm.mp3 -filter_complex "[1:a]volume=0.3[bg];[0:a][bg]amix=inputs=2:duration=first" -c:v copy output.mp4

3. Video Stabilization

Two-pass process to fix shaky footage:

# Pass 1: Analyze
ffmpeg -i shaky.mp4 -vf vidstabdetect -f null -
# Pass 2: Transform
ffmpeg -i shaky.mp4 -vf vidstabtransform,unsharp=5:5:0.8:3:3:0.4 output.mp4

4. Color Correction & Enhancement

Adjust brightness, contrast, and saturation:

# brightness=0.05, contrast=1.1, saturation=1.2
ffmpeg -i input.mp4 -vf "eq=brightness=0.05:contrast=1.1:saturation=1.2" output.mp4

5. Automatic Thumbnail Sheet

Create a 3x3 grid of frames:

ffmpeg -i input.mp4 -vf "select='not(mod(n,100))',scale=320:-1,tile=3x3" -frames:v 1 preview.png

6. Remove Silence from Audio

Automatically cut silent parts from the beginning and end:

ffmpeg -i input.mp4 -af silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB:stop_periods=1:stop_silence=0.1:stop_threshold=-50dB output.mp4

7. Hardsub Burning

Burn SRT/ASS subtitles directly into the video stream:

# Burn SRT
ffmpeg -i input.mp4 -vf "subtitles=subs.srt" output.mp4
# Burn ASS (supports advanced styling)
ffmpeg -i input.mp4 -vf "ass=subs.ass" output.mp4

8. Target File Size Compression

Calculate bitrate to fit a specific file size (e.g., 50MB for 60s video):

# Bitrate = (TargetSize_in_bits) / (Duration_in_seconds)
# 50MB = 400,000 bits. For 60s, bitrate ≈ 6600k
ffmpeg -i input.mp4 -b:v 6000k -maxrate 6000k -bufsize 12000k -c:a aac -b:a 128k output.mp4

9. Scene Change Detection

Extract frames where a scene change is detected (threshold 0.4):

ffmpeg -i input.mp4 -filter_complex "select='gt(scene,0.4)',metadata=print:file=scenes.txt" -vsync vfr scene_%03d.png

10. Extracting Frames at Specific Intervals

Extract one frame every 5 seconds:

ffmpeg -i input.mp4 -vf "fps=1/5" img_%03d.jpg

11. Batch Processing (Shell Snippet)

Convert all .mov files in a directory to .mp4:

for f in *.mov; do ffmpeg -i "$f" "${f%.mov}.mp4"; done

12. Live Streaming (RTMP)

Push a local file to a streaming server (YouTube/Twitch):

ffmpeg -re -i input.mp4 -c:v libx264 -preset veryfast -b:v 3000k -maxrate 3000k -bufsize 6000k -pix_fmt yuv420p -g 60 -c:a aac -b:a 128k -f flv rtmp://a.rtmp.youtube.com/live2/YOUR_STREAM_KEY

Hardware Acceleration

Constraints & Error Handling

• Stream Mapping: Always use -map for complex files to ensure you get the right audio/subtitle tracks.
• Seeking: Put -ss before -i for fast seeking (input seeking), or after -i for accurate seeking (output seeking).
• Format Support: Ensure the output container (extension) supports the codecs you've chosen.