Tutorial
OmniHuman 2.0 API Guide: 5 High-Value Video Generation Use Cases
A developer's practical walkthrough of image-to-video, audio-driven lip-sync, and avatar workflows
本页内容
What OmniHuman 2.0 Actually Takes In and Gives Back
Before you design a pipeline, get the contract straight. OmniHuman 2.0 is an image-to-video model exposed through a single route with capabilities for image-to-video, audio-to-video, lip-sync, and avatar video. Every one of those capabilities collapses into the same shape: you send a character image and an audio track, you get a video back.
The flow looks like this in practice:
image_url(required) — the character to animate, real or illustrated.audio_url(required) — the speech or singing that drives the lip-sync.prompt(optional) — direction for scene, expression, gestures, and camera.- Output — a video whose length follows the audio and whose aspect ratio and resolution are inherited from the input image.
That last point is the one most integrations miss. You are not choosing a resolution tier or an aspect ratio at request time. You are choosing an image, and the model returns video geometry that matches it. If you want a 9:16 vertical clip for a short-form feed, the input image must already be vertical. If you want a cinematic wide frame, the image must be wide. Planning your asset pipeline around this one fact eliminates a whole class of "the output looks wrong" tickets later.
There is no separate model variant for singing versus speaking, and no separate route for illustrated characters. One route, two required inputs, one optional steering string. The rest of this guide is about choosing those inputs well for five different use cases.
The Two Required Inputs: Preparing Images and Audio
Validation failures on this route are almost always asset failures. Both required inputs have hard constraints, so build a pre-flight step rather than discovering problems in production.
Image constraints
image_url accepts jpeg, png, or webp up to 10MB. The width-to-height ratio must fall between 0.4 and 2.5, and each side must be between 300 and 6000 pixels.
What that means geometrically:
- A 0.4 ratio is a tall sliver (roughly 2:5). A 2.5 ratio is a wide banner (roughly 5:2). Anything more extreme — a full-height screenshot, a panoramic crop — is rejected.
- A 250-pixel-wide thumbnail fails the side minimum even if the ratio is fine.
- An 8000-pixel-wide export from a design tool fails the side maximum even if it is visually reasonable.
Because the output keeps the image's aspect ratio and resolution, feeding a 300-pixel-wide image gives you a 300-pixel-wide video. Feed a clean, well-lit frame at a comfortable size and the model has something to work with. Practically: crop to your target ratio, export at a reasonable resolution inside the side range, and keep the face or subject fully in frame — the model animates what it can see.
Audio constraints
audio_url accepts mp3 or wav, up to 15MB, with a duration between 4 and 300 seconds. The video is exactly as long as the audio, so audio duration is both a creative decision and a cost decision.
A few preparation notes that pay off:
- A 3-second clip is invalid. Merge or pad short lines into a single take that clears the 4-second floor.
- Long narration must be chunked. A 20-minute podcast cannot go through in one call; split it into segments under the 300-second ceiling and generate each segment against the same image so the character stays consistent across cuts.
- Clean audio matters disproportionately. Speech with music beds, heavy reverb, or overlapping speakers gives the lip-sync less to lock onto. Export a dry voice track where possible.
- Watch the file size as well as the duration. A long wav at a high sample rate can hit 15MB before it hits 300 seconds.
Validate both inputs client-side — MIME type, byte size, pixel dimensions, ratio, and audio duration — before you ever send a request. That single guard saves the most money of anything in this guide.
Steering Output with the Optional Prompt
The prompt parameter is a short string, capped at 300 characters. Its job is direction, not description.
Here is the distinction that matters. The model already knows who the character is — that is what image_url is for. Identity, face, clothing, and overall look come from the image. The prompt steers four things:
- Scene — the setting or mood of the moment.
- Expression — smiling, serious, surprised, mid-laugh.
- Gestures — nodding, a small hand movement, a shrug.
- Camera — a slow push-in, a subtle drift, a locked-off frame.
Short, imperative direction works far better than an adjective pile. Compare:
- Weak: "a beautiful professional woman in a bright modern office, high quality, cinematic lighting, 4k, ultra detailed"
- Useful: "smiling while nodding, slow push-in"
The first version spends its character budget describing what the image already provides. The second tells the model what to do over time, which is the only thing the prompt can influence.
A few rules of thumb:
- Pick one or two motions. "Smiling while nodding" is coherent. "Smiling, waving, turning, laughing, walking forward" is a request the model will average into mush.
- Put camera moves last and keep them subtle. "Slow push-in" reads clearly; a fast zoom usually does not.
- Do not try to change identity. Asking for a different hairstyle or outfit fights the image and produces artifacts rather than a new character.
- Stay well under the 300-character cap. Shorter prompts are easier to A/B test, and systematic comparison is how you find your house style.
If the audio already carries strong emotion — a laugh, a shout — let it. The prompt and the audio should agree; conflicting direction shows up as a flat performance.
For the full parameter list alongside the rest of the family, see the OmniHuman 2.0 model page.
Use Case 1 and 2: Talking Avatar and Singing Clip
These two are the direct applications of the route and the fastest to ship.
Talking avatar
A single presenter image plus a narration track. This is the workhorse case: course intros, onboarding videos, FAQ answers, internal announcements.
Asset strategy:
- Use a front-facing or three-quarter portrait, evenly lit, eyes open, mouth closed or neutral. A mid-laugh still frame fights the audio.
- Trim narration to a tight take. Because the video is as long as the audio, every second of "um" and dead air is a second of generated video you did not need.
- Keep a short prompt for consistency: something like "calm, speaking directly to camera" gives you a repeatable baseline across a whole series.
Gotcha: if the source image is cropped too close on the face, head motion can push the subject out of frame. Leave a little headroom.
Singing clip
Same route, different audio. Send a vocal track instead of speech and the lip-sync follows the performance.
Asset strategy:
- The image should look like the moment you want — a stage frame, a studio shot — because the model will not add a set for you.
- Export a clean vocal stem if you have one. Full mixes with dense instrumentation blur the phoneme signal and the mouth shapes get vague.
- Keep clips short. A 30–60 second hook renders a punchy social clip; the same song stretched to the full 300-second ceiling is a longer, costlier render with more chance of drift.
- Prompt for energy rather than choreography: "singing with eyes half-closed, subtle sway" sets a tone without asking for motion the model may not deliver cleanly.
Both cases share one habit worth building early: version your assets. Store the exact image, audio, and prompt used for each render so that when a client asks for a tweak you can change one variable instead of re-guessing the whole setup.
Use Case 3 and 4: Localized Lip-Sync and Product Presenter
These two cases are where the image-plus-audio contract earns its keep, because they reuse assets aggressively.
Localized lip-sync
One character image, many language tracks. Record or synthesize the same script in English, Spanish, Japanese, and so on, then run each audio file against the same image_url. The face stays consistent across the whole set; only the mouth and the performance change.
The workflow that works:
- Lock the image first. Once the presenter frame is approved, never touch it — that is what guarantees visual continuity across languages.
- Keep segment structure identical across languages. If the English track is cut at sentence boundaries, cut the Spanish track in the same places so the shared timeline lines up.
- Use the same prompt string for every language. Varying direction per language introduces performance differences you did not intend, which reviewers will read as quality problems.
- Expect duration drift. Translations are rarely the same length as the source, and the video follows each audio file, so your final language builds will differ by a second or two. Plan the edit around that rather than forcing identical runtimes.
Gotcha: fast, dense speech in any language stresses the lip-sync more than slow speech. If one locale looks noticeably rougher, check whether its track is simply faster, and consider a slightly more measured read.
Product presenter
A human presenter delivering a product pitch, a feature walkthrough, or a promo line. Here the image is doing brand work as much as the audio is doing content work.
Asset strategy:
- Choose a frame that matches the brand: wardrobe, background, and lighting should already look like your marketing. The model animates the frame; it does not restyle it.
- Feed it a scripted, well-recorded voiceover. This is the case where audio quality is most visible, because the audience is being sold to, not just informed.
- Use the prompt for a touch of life: "confident smile, slight nod, slow push-in" reads as a produced spot rather than a talking head.
- Because output resolution follows the input image, export the source frame at the resolution you actually need for the placement. There is no upscale step afterward.
Both use cases benefit from the same discipline: one approved image, many audio variants. That is the cheapest way to produce a large, consistent library.
Use Case 5: Animating Illustrated Characters
The route accepts an illustrated character just as readily as a photograph, and this is where teams often get surprised in both directions — sometimes better than expected, sometimes worse, rarely for the reason they assumed.
When the character is a drawing, a mascot, an anime-style figure, or a 3D render, the image is the entire visual specification. Identity, line weight, color palette, shading style — all of it comes from the file. The prompt cannot make a flat vector mascot look like a painted illustration.
What actually drives perceived quality here:
- Image style consistency. Clean, high-contrast line art with a clearly readable mouth region animates far better than a busy, texture-heavy frame where the mouth is ambiguous.
- Framing. A character with a clearly defined head and shoulders gives the model an obvious region to animate. Full-body shots of small figures at low resolution give it almost nothing.
- Audio clarity. This matters even more than with human subjects. A cartoon mouth has no real-world reference for viewers, so any vagueness in the lip-sync reads as "wrong" immediately. Use clean, well-articulated speech.
- Restraint in the prompt. For stylized characters, ask for small motions — "subtle nod, bright expression" — rather than complex gestures that fight the art style.
A practical pattern for mascot-driven content: keep a library of approved character frames at your target aspect ratio and resolution, one per expression or pose, and pair each with the appropriate audio. Localized marketing, explainer videos, and in-app tutorials can all run through the same handful of frames.
Gotcha: illustrated characters with non-standard proportions — very large heads, tiny mouths, abstract faces — occasionally produce motion that looks uncanny rather than charming. Test one frame and one audio clip before committing to a full campaign; it is the cheapest experiment you will run.
Cost Planning and Pre-Flight Checks
Billing on this route is per second, with estimated_seconds as the price factor. The mental model is simple: cost scales with the length of the audio you submit, because the video is exactly as long as the audio.
Planning the spend
Because billing is tied to output seconds and output length equals audio length, your budget lever is audio trimming. Two habits cover most of it:
- Trim before you render. Strip silence, tighten pauses, and cut retakes. A take with 8 seconds of dead air at the head and tail is 8 seconds of paid video nobody watches.
- Chunk deliberately. A long script becomes several segments under the 300-second ceiling. Chunking does not change total billed seconds much, but it lets you review and approve segment by segment instead of paying for a full render that needs a redo.
- Batch by use case. Localized lip-sync multiplies audio tracks against one image. Estimate that multiplication up front — number of languages times average duration — rather than discovering it after the fact.
Since the per-second rate is a single published tier, estimating a batch is straightforward arithmetic: total audio seconds across the batch, times the per-second rate. Check the current rate on the model page before you build a budget into a proposal, and treat that panel as the source of truth.
Pre-flight checklist
Run every one of these before a request leaves your server:
image_urlresolves, is jpeg/png/webp, and is under 10MB.- Image ratio is between 0.4 and 2.5, and each side is between 300 and 6000 px.
- The frame matches the aspect ratio and resolution you want in the output — no post-crop surprises.
audio_urlresolves, is mp3 or wav, and is under 15MB.- Audio duration is between 4 and 300 seconds.
prompt, if present, is under 300 characters and describes motion rather than identity.estimated_secondsreflects the actual audio duration.
A single validation function that enforces items 1–7 catches nearly every failure before it costs anything.
Common Failure Modes and How to Avoid Them
Most problems on this route fall into three buckets. Here is the checklist.
Rejected inputs
- Ratio out of range. A panorama or a tall screenshot falls outside 0.4–2.5. Crop to a sane frame first.
- Side out of range. Thumbnails below 300 px and exports above 6000 px both fail. Resize into the window.
- File too large. Images over 10MB and audio over 15MB are rejected regardless of duration. Re-encode before upload.
- Wrong format. Images must be jpeg, png, or webp; audio must be mp3 or wav. A .m4a or .flac will not pass.
- Audio too short. Anything under 4 seconds fails. Merge short lines into one take.
Mismatched length expectations
The video is as long as the audio — always. If a render came back longer or shorter than a stakeholder expected, the audio was longer or shorter than they thought. Build duration into your UI: show the audio length and the estimated billed seconds next to the generate button. For scripts over 300 seconds, split the audio and generate segments against the same image, then edit them together.
Official pricing is $0.10 per second of output, and output length follows the audio (4 to 300 seconds).
Weak or unconvincing motion
When the result is technically fine but feels flat, the cause is usually in the source inputs:
- Muddy audio. Music beds, reverb, and overlapping speakers give the lip-sync little to work with. Use a dry vocal or speech track.
- Overloaded prompt. Five simultaneous gestures average into a vague wobble. Cut to one or two motions.
- Prompt describing instead of directing. Identity is the image's job. Keep the prompt about expression, gesture, scene, and camera.
- A frame that fights the audio. A neutral, eyes-closed, or heavily shadowed face gives the model less to animate. Pick a frame that matches the performance.
- Low-resolution source. Output resolution follows the image, so a small source means a soft result at any zoom.
Work through them in that order — audio, prompt, image — because that is roughly the order of impact. Fix one variable per render so you know what changed. With the contract, the assets, and these checks in place, all five use cases run on the same route with nothing but different audio and image strategies behind them. Start from the OmniHuman 2.0 model page to confirm current parameters and rate before you wire up your pipeline.
相关文章
查看全部文章 →-
Tutorial
gpt-6.2-sol in Practice: 5 High-Value Uses of a Text Model in Business
-
Tutorial
gpt-6.2-astra Practical Guide: Integration and Implementation Key Points for 5 High-Value Text Scenarios
A Developer's Checklist from Scenario Selection to Engineering Implementation
-
Tutorial
Tripo H3.1 Practical Application Scenarios: 5 High-Value Use Cases and Implementation Essentials
From text-to-3D to multiview reconstruction: choosing the right mode and parameters by business scenario