- -82%manual review workload
- 99.2%policy violation recall
- +0.9%added unit processing cost
Background
The platform ingests tens of thousands of videos daily. Moderation had been entirely manual spot-checking: reviewers could only cover about 5%, leaving persistent exposure. Meanwhile operations spent hours tagging videos, writing summaries, splitting chapters and producing subtitles by hand.
The key decision: reuse the transcoding pipeline
The cheapest choice we made was not building separate ingestion for AI.
When a video enters the distributed transcoding system, every frame is decoded anyway. We hung a side branch off that path: frames are sampled by policy, audio is separated, metadata extracted, and everything is dispatched to the AI processing queue.
The result is that adding AI raised unit processing cost by only 0.9% — because the expensive decoding step was already paid for.
Pipeline
Transcode worker ─┬─→ video output (multi-rendition HLS)
└─→ side branch: keyframe sampling + audio separation
↓
ASR transcription ──→ text understanding (summary/tags/chapters/policy terms)
keyframe vision ──→ visual tags / violation signals
↓
multimodal aggregation → tiered verdict → human review workbench
Sampling strategy
Uniform frame sampling is both wasteful and lossy. We use scene-change detection with keyframe priority: dense sampling where the picture changes sharply, sparse sampling over static lecture footage. Same detection quality with roughly 60% fewer frames.
Audio carries most of the signal
For lecture content the audio track holds far more information than the picture. ASR transcripts drive summaries and chaptering as well as moderation, and are indexed for full-text search — which incidentally solved “find the video by what was said in it.”
Tiers, not a binary
Moderation output has four levels:
| Tier | Action | Share |
|---|---|---|
| Clearly compliant | Auto-approved | ~86% |
| Suspected issue | Human review queue | ~12% |
| High risk | Auto-blocked, priority review | ~1.5% |
| Detection failed | Full human review | ~0.5% |
Reviewers only see the last three tiers: 82% less work, while coverage went from 5% spot checks to 100%.
Thresholds are tunable
Different categories tolerate false positives and misses differently. Thresholds are configurable, with a replay tool that immediately shows recall and false-positive rates against historical samples, so operations can make the trade themselves.
Outcomes
- Full-catalogue review replaced 5% spot checks, with 99.2% violation recall
- Manual review workload down 82%; the review team shifted to second-pass and rule operations
- Tags, summaries, chapters and subtitles now generate automatically
- With transcripts indexed, on-site search moved from titles to spoken content
Related
The underlying scheduler is described in Distributed Video Transcoding System; the AI method is part of AI Application Development.