Add Text to Video via API: Caption and Overlay Workflows

Why text-on-video is an API design problem, not just a prompt

Adding text to a generated video looks like a one-line prompt change: describe the caption, send the request, get a clip with words in it. In practice, the moment you need captions that are readable, timed, localized, or reusable across scenes, the prompt stops being the interesting part. The architecture is.

There are two viable paths, and they lead to different codebases:

• Native text in generation. You ask the video model to render the text as part of the scene. One request, one asset, no post-processing.
• Clean footage plus overlay. You generate footage without text (or ask for text-free framing), then composite captions and overlays in a second rendering step.

Why does this matter beyond aesthetics? Because everything downstream depends on the choice. If text lives inside the generated pixels, you cannot easily restyle it, translate it, time it to speech, or fix a typo without regenerating the clip. If text lives in a separate overlay layer, you own layout, fonts, and timing, but you now own a renderer and a second request in your pipeline.

The rest of this guide treats text-on-video as what it usually becomes in production: a data problem (what text, where, when) plus a rendering problem (who draws it). Once you separate those, the API calls become straightforward, and you can move between generation backends without rewriting your caption logic.

A useful mental model is to keep three things distinct at all times: the content (the words), the timing (when they appear), and the presentation (font, position, style). Native generation collapses all three into one opaque step. Overlay keeps all three explicit and editable. Neither is universally better; the goal is to know which one you are committing to before you write the integration.

Path one: prompting native on-screen text during generation

Native text means the generation model draws the characters into the video frames. You get a single output asset with text baked in, and your pipeline stays short: build a prompt, submit, poll, deliver.

What to put in the prompt

Model-rendered typography is less predictable than composited text, so treat the prompt as a narrow instruction rather than a place to describe a whole design system. Practical guidance:

• Keep the text short. A few words per shot. Long sentences invite spelling drift and cramped layout.
• Quote the literal string. If the words must appear exactly, put them in quotes and ask for that exact text. Vague descriptions produce vague results.
• Be explicit about placement and style. "Large white text centered in the lower third" is actionable. "Nice title" is not.
• One text element per shot. Asking for a title, a subtitle, and a watermark in the same clip multiplies failure modes.
• State the language. If the copy is non-English, say so, since glyph coverage varies.

What to expect back

Treat the output as a draft, not a spec. Even good results can show warped letterforms, inconsistent spacing, or partial occlusion by motion. Always inspect the rendered frames before delivery; a cheap programmatic check plus a human glance at the first and last text-bearing frames catches most problems.

When this path wins

Native text is the right default when the text is incidental and atmospheric: a sign in a background street, a stylized logo on a product, a short title card that you are willing to regenerate if it comes out wrong. It also wins when you genuinely cannot afford a second rendering step, for example in a tight turnaround flow where one request per clip is the budget.

The cost is control. Every change to wording, style, or timing means a new generation request, and results are not deterministic across retries. Design your prompts and your storage so that a regenerated clip can replace an old one cleanly.

Path two: generating clean footage and compositing text yourself

The overlay path splits the job in two. First you generate footage that is deliberately text-free. Then you composite captions and overlays on top, either with a rendering service or your own media pipeline.

Generate clean first

Ask for composition without text: describe the scene, subject, camera motion, and lighting, and explicitly avoid requesting on-screen words. If the model tends to invent signage or UI text, mention that the frame should contain no readable text. The output here is a base asset — a stable input you can reuse across many text variants.

Composite second

The overlay step consumes two inputs: the base asset and a structured caption/overlay payload. Because you control the renderer, you get properties that prompting cannot guarantee:

• Deterministic fonts and sizes. The same payload renders identically every time, which matters for brand consistency.
• Precise timing. Captions can align to speech, music beats, or scene cuts to the frame.
• Safe-area and layout rules. You can enforce margins, avoid faces and key subjects, and keep text inside platform-specific safe zones.
• Cheap iteration. Fixing a typo or translating a caption is a re-render of the overlay, not a regeneration of the video.
• Composability. Watermarks, lower thirds, and subtitles can be layered independently and toggled per output.

The trade-offs

The overlay path trades model dependency for pipeline complexity. You add a second request, you need somewhere to store intermediate assets, and you own the rendering configuration. In exchange, the text becomes data you can version, test, and swap. For anything with real caption requirements — accessibility subtitles, multi-language variants, templated social formats — that trade is almost always worth it.

A practical pattern is to render overlays non-destructively: keep the base asset and the caption payload as the source of truth, and treat the composited video as a derived artifact you can always regenerate.

Designing a caption payload that survives both paths

If you design your caption data well, the choice between native generation and overlay becomes a rendering detail rather than a rewrite. The goal is one payload that can be serialized into a prompt for native text, or fed to a renderer for overlay.

Core fields

Model each text element as an object with content, timing, and presentation kept separate:

{
  "track": "captions",
  "items": [
    {
      "text": "Everything you need to know",
      "start": 0.0,
      "end": 2.4,
      "style": { "role": "title", "align": "center", "position": "lower_third" },
      "lang": "en"
    }
  ]
}

Key design decisions:

• Time in seconds, floating point. Use a single timebase for the whole project and convert to frames only at render time.
• Style by role, not by absolute values. Name roles like title, caption, and lower_third, and resolve them to fonts and sizes in the renderer. This keeps payloads portable across formats.
• Track names. Separate captions, titles, and watermark tracks so tracks can be enabled or disabled per output.
• Language tag per item. Needed for font selection and for later translation passes.
• No baked-in presentation in the text field. Never put style hints inside the words; that is what the style object is for.

Feeding the two paths from one schema

For native generation, write a small adapter that picks the most important item and turns it into prompt instructions: the literal text, its placement, and a style descriptor. Keep this adapter intentionally lossy — native rendering cannot honor a full subtitle track anyway.

For overlay, pass the payload to your renderer, which resolves roles and positions and draws each item within its time window.

The payoff: your application code builds one caption object, and switching from native to overlay (or back for a specific clip) is a config change, not a data migration.

A working overlay request, step by step

Here is the sequence for the overlay path, expressed as API-shaped steps. Exact endpoint names and parameters depend on the provider, but the shape is stable, and a thin abstraction layer keeps it stable across backends.

Step 1: submit the clean generation

POST /video/generations
{
  "prompt": "A slow dolly shot through a modern office, no on-screen text, natural light",
  "duration_seconds": 6,
  "aspect_ratio": "16:9"
}

Keep prompts text-free and store the returned job identifier.

Step 2: poll until the asset is ready

GET /video/generations/{job_id}

Poll with backoff rather than a tight loop. Treat statuses like queued, running, succeeded, and failed as first-class states in your code, and persist the resulting asset URL or ID so a crash does not lose the generation.

Step 3: submit the overlay render

POST /video/overlays
{
  "source_asset": "asset_abc123",
  "captions": {
    "track": "captions",
    "items": [
      { "text": "Shipping today", "start": 0.4, "end": 2.0, "style": { "role": "title", "position": "lower_third", "align": "center" }, "lang": "en" },
      { "text": "No rebuild required", "start": 2.2, "end": 4.0, "style": { "role": "title", "position": "lower_third", "align": "center" }, "lang": "en" }
    ]
  }
}

Note that the overlay request carries only the source asset plus text data. It does not regenerate footage, which is what makes iteration cheap.

Step 4: retrieve the composited result

Poll the overlay job the same way you polled generation, then fetch the final asset. Store the base asset, the caption payload, and the output together so the composite is reproducible.

In code, wrap steps 1–4 in one function that takes a prompt and a caption payload and returns a final asset. That function is your real interface; the individual endpoints are implementation details you can swap.

Reliability checks before you ship

Text-on-video pipelines fail in quiet ways: a clipped caption, a mistranslated line, a regenerate that never gets cleaned up. Add checks before users see them.

Verify rendered text

• Native path: sample frames at the start and end of each text window and inspect them. Automate a basic legibility check where you can, but keep a human review step for customer-facing copy, since model typography can degrade unpredictably.
• Overlay path: validate the payload before rendering — every item has non-overlapping timing, within the clip duration, with a supported language and role. After rendering, spot-check frames at item boundaries to confirm nothing was cut off.

Make retries safe

• Idempotency keys on submission requests so a retried network call does not create duplicate generation jobs.
• State machine per job: queued → running → succeeded/failed, persisted, so restarts resume instead of restarting.
• Bounded polling with backoff and a timeout that surfaces a clear failure rather than looping forever.
• Store intermediate assets. If the overlay step fails, you should not pay to regenerate footage you already have.

Control cost across the chain

The overlay path has two billable steps — generation and rendering — so instrument both. Track how often you regenerate footage versus only re-rendering overlays; if footage regeneration dominates, your native/overlay split may be wrong for that workload. Cache final assets keyed by (base asset, caption payload) so repeated requests for the same output are free.

Observability

Log the prompt, the caption payload hash, the job IDs, and the final asset ID together. When a user reports a wrong caption, you want to identify within seconds whether the problem was generation, data, or rendering. Keep the caption payload as the versioned source of truth so you can always explain what the renderer was asked to draw.

Choosing your default workflow

Use this as a starting rule, and revisit it as the product grows.

Choose native text when the text is decorative or incidental, the wording is short, you can accept regeneration as the fix for mistakes, and adding a second rendering step is not worth the engineering time for that clip.

Choose the overlay path when text must be exact, timed, translated, restyled, or reused across multiple outputs. Captions, subtitles, lower thirds, and templated marketing formats almost always fall here. The extra request is the price of determinism.

Hybrid is legitimate. Many teams generate clean footage by default and rely on overlay for anything that matters, reserving native text for one-off atmospheric shots. Because both paths consume the same caption payload, you can make that call per clip without special-casing your data.

Whichever you pick, keep the integration behind a thin abstraction: one function that accepts a prompt and a caption payload and returns a final asset. That boundary is what lets you switch generation backends, add rendering options, or move individual clips between workflows without touching the caption logic that took effort to get right.