← all projects
Personal Project · 1 min read
Automated Media Content Pipeline
A modular Python pipeline that turns raw video into transcribed, voice-processed, vertical-format clips — headlessly, on a daily cron.
~/automated-media-content-pipeline/architecture5 stages · parallel Whisper + XTTS-v2 · daily cron
Problem
- Turning long-form source video into transcribed, voice-processed, vertical-format clips normally means running several separate tools by hand: download, detect highlights, cut clips, transcribe, resynthesize audio, reassemble — every single day.
Architecture
- The pipeline is split into independent, chainable stages: source retrieval (yt-dlp) → audio-based highlight detection → clip extraction → transcription → vertical-format video assembly.
- Each stage reads and writes to disk independently, so a single stage — e.g. swapping the TTS model — can change without touching the rest of the pipeline.
- State (which source has been processed, current rotation position) is persisted back to the repository itself between runs, rather than an external database, since the whole job is stateless infrastructure by design.
Engineering Decisions
- Chained OpenAI-Whisper for automatic transcription and subtitle generation with Coqui XTTS-v2 for voice-cloned speech synthesis — two separate ML models composed into a single automated workflow.
- FFmpeg handles both clip extraction and final vertical-format assembly, driven entirely from the Python orchestration layer.
- Built a scheduled CI/CD pipeline on GitHub Actions with a daily cron trigger that installs dependencies from a clean runner, executes the full pipeline headlessly, and commits updated rotation state back to the repo automatically.
Challenges
- Running ML inference (Whisper + XTTS-v2) inside a GitHub Actions runner meant working within limited compute and execution-time constraints — required careful dependency pinning and staged execution rather than one long-running script.
- Iterated the architecture across multiple content-source variants, refining the pipeline over 800+ commits to improve reliability and processing speed.
Lessons Learned
- Decoupling pipeline stages early made it far easier to debug failures in isolation and to later extract the TTS stage into its own service (see the Voice Synthesis REST API project).