Automated Media Content Pipeline

Requirements

• Run fully unattended on a daily schedule — no manual trigger, no server to babysit
• Chain two ML models (transcription + voice synthesis) without exceeding GitHub Actions' free-tier compute/time budget
• Survive a completely disposable, stateless runner — no local state persists between runs except what's explicitly committed back to git
• Keep each pipeline stage independently swappable, since the TTS model was already a likely future extraction point

Problem

• Turning long-form source video into transcribed, voice-processed, vertical-format clips normally means running several separate tools by hand: download, detect highlights, cut clips, transcribe, resynthesize audio, reassemble — every single day.

Architecture

• The pipeline is split into independent, chainable stages: source retrieval (yt-dlp) → audio-based highlight detection → clip extraction → transcription → vertical-format video assembly.
• Each stage reads and writes to disk independently, so a single stage — e.g. swapping the TTS model — can change without touching the rest of the pipeline.
• State (which source has been processed, current rotation position) is persisted back to the repository itself between runs, rather than an external database, since the whole job is stateless infrastructure by design.

API Design

• This project has no external API surface of its own — it's a scheduled pipeline invoked by a cron job, not a service other systems call.
• The Voice Synthesis REST API project was deliberately extracted from this pipeline specifically to expose the TTS capability as something callable, once it became clear other tools could reuse it.

Database

• No traditional database. State (which source has been processed, current rotation position) is committed directly back to the git repository between runs, avoiding the operational cost of a managed datastore for a job that runs once a day.

Authentication

None — this is a scheduled, unattended job with no external caller and no user-facing surface, so there's nothing to authenticate. Auth becomes relevant for the Voice Synthesis REST API, which is what got extracted from this pipeline specifically to be callable.

Infrastructure

• Runs entirely on GitHub-hosted Actions runners — no persistent server, no cloud account to manage. Each run starts from a clean image, installs dependencies fresh, and tears down after committing state.

Deployment

• Deployment is the scheduled workflow file itself (.github/workflows/daily-pipeline.yml) — there's nothing to separately deploy; a change to the pipeline takes effect the next time the cron fires or a commit lands on main.

Monitoring

No formal monitoring today — a failed run shows up as a failed GitHub Actions job in the repo's Actions tab, which is sufficient at once-a-day frequency. That wouldn't hold up at higher frequency or if the output fed something user-facing; alerting on job failure is a reasonable next step.

Engineering Decisions

• Chained OpenAI-Whisper for automatic transcription and subtitle generation with Coqui XTTS-v2 for voice-cloned speech synthesis — two separate ML models composed into a single automated workflow.
• FFmpeg handles both clip extraction and final vertical-format assembly, driven entirely from the Python orchestration layer.
• Built a scheduled CI/CD pipeline on GitHub Actions with a daily cron trigger that installs dependencies from a clean runner, executes the full pipeline headlessly, and commits updated rotation state back to the repo automatically.

Challenges & Trade-offs

• Running ML inference (Whisper + XTTS-v2) inside a GitHub Actions runner meant working within limited compute and execution-time constraints — required careful dependency pinning and staged execution rather than one long-running script.
• Iterated the architecture across multiple content-source variants, refining the pipeline over 800+ commits to improve reliability and processing speed.

Performance

• The binding constraint is GitHub Actions' per-job compute and time budget, not the pipeline logic itself — Whisper and XTTS-v2 share the same runner CPU, so total run time is dominated by ML inference rather than I/O or orchestration overhead.

Scaling

• Not built to scale in the traditional sense — it's a once-a-day batch job, not a service under concurrent load.
• The real scaling question is per-run capacity: a longer source video or a second output format directly increases the compute budget a single run needs, bounded by GitHub Actions' time limit.
• The natural next step under higher volume would be moving ML inference to a persistent worker, so scheduled runs trigger only a lightweight job instead of paying cold-start and full inference cost every time.

Lessons Learned

• Decoupling pipeline stages early made it far easier to debug failures in isolation and to later extract the TTS stage into its own service (see the Voice Synthesis REST API project).

Future Roadmap

• Move ML inference to a persistent worker so scheduled runs only trigger a lightweight job, removing the GitHub Actions timeout risk that currently caps how much content a single run can process.
• Add a fallback source list so a single dead or region-locked video doesn't stall the day's run.