Reference-to-Video Short-Form Workflow with vidu-q4-preview on the HiAPI API

Why Reference-to-Video Changes the Short-Form Workflow

Prompt-only generation asks the model to invent everything: subject appearance, wardrobe, framing, motion, and audio. For one-off experiments that is fine. For short-form production, where the same character or presenter has to appear across dozens of clips, prompt-only control breaks down quickly. Small wording changes produce large visual drift, and no amount of prompt engineering reliably pins a face, an outfit, or a voice.

Reference-to-video changes the contract. Instead of describing the subject in words, you hand the model a set of reference assets — multiple images that establish identity, plus a voice reference that establishes how the subject sounds — and let the prompt describe what should happen rather than who is in it. Creative control shifts from token-level wording to asset-level curation, and asset preparation becomes the first engineering step in the workflow, not an afterthought.

That shift maps cleanly onto how short-form content is actually produced:

• Reusable identity. A vetted reference set can be reused across many clips, so a series stays visually consistent without re-prompting.
• Separation of concerns. Writers own the action and pacing in the prompt; asset owners own the look and voice in the reference set.
• Reviewable inputs. When an output looks wrong, you can diff the reference set between runs instead of guessing which adjective moved the result.

Short-form also raises the bar on failure cost. Clips are cheap to distribute and expensive to re-shoot, so a pipeline that produces predictable variation from a fixed reference library is more valuable than a pipeline that occasionally produces a brilliant one-off. Reference-to-video is the mechanism that makes that predictability possible.

What stays the same

You still write a prompt, you still submit an asynchronous-style request, and you still post-process the returned video. The difference is that the reference assets become first-class inputs that you version, validate, and store alongside your prompts.

How the vidu-q4-preview Reference Model Fits on HiAPI

On HiAPI, vidu-q4-preview/reference-to-video is addressed as a model identifier in the same request surface you use for other generation tasks. You do not integrate against a separate vendor SDK or manage a second set of credentials. Your application calls the HiAPI API, the platform routes the request to the reference-to-video model, and the response carries back a generated video asset.

That layering matters for two practical reasons:

1. One integration surface. Authentication, request structure, and error handling stay consistent across the models you use, so adding or swapping a model is a configuration change rather than a new client.
2. Uniform operational tooling. Logging, retries, rate handling, and response parsing live in one place, which keeps the reference-to-video path from becoming a parallel codebase.

What this endpoint is for

The reference-to-video model is intended for clips where the subject identity must be anchored by supplied assets rather than invented from text — short promotional clips, recurring presenter segments, product shots with a consistent model, and similar content where consistency across a series is the point. It is not a general-purpose video editor; it generates a new clip from your references and prompt.

Treat the model identifier as configuration

Model identifiers and their supported parameters change over time as models are added, previewed, and retired. Do not hard-code vidu-q4-preview/reference-to-video into business logic in a dozen places. Keep it in one configuration layer, and verify the current identifier, supported parameters, and expected input shapes against HiAPI's public documentation before you ship. Anything in this guide about request structure is an outline of the shape; the documentation is the source of truth for exact field names.

What this guide does not cover

Pricing is intentionally out of scope. Billing on HiAPI is metered per task according to the model and parameters you submit, and the live figures belong in the platform's price panel, not in a tutorial that goes stale. The same applies to any per-model latency or throughput claims — measure them on your own workload rather than trusting a number copied from a blog post.

Preparing Multi-Image and Voice References

Reference quality dominates output quality. Budget real engineering time for asset preparation, because the model can only be as consistent as the references you give it.

Selecting multi-image references

Use several images that depict the same subject under different but compatible conditions — different angles, slightly different lighting, a mix of close and medium framing. The goal is to give the model enough coverage to resolve identity from multiple viewpoints without introducing contradictions.

Practical selection rules:

• Same subject, same era. Avoid mixing references from looks that differ in hair, facial hair, or styling unless you want the model to average them.
• Consistent color treatment. Wildly different white balance or grading across references pushes the model toward an unstable palette.
• No competing subjects. A reference set with two prominent faces invites blending. One subject per set.
• Sufficient resolution. Downscaled or heavily compressed references lose exactly the detail the model needs.

Preparing the voice reference

A voice reference establishes timbre and delivery. Supply clean audio — low noise floor, no background music, no overlapping speakers. Trim to the segment that best represents the intended delivery, and keep it short enough to be representative rather than a full monologue. If you plan a series in the same voice, lock one voice reference and reuse it.

Normalizing assets

Before anything reaches the request builder, run every asset through a normalization step: consistent image format and dimensions within a set, consistent audio format and sample rate. Normalization removes an entire class of "it worked yesterday" failures and makes cache keys meaningful.

Versioning reference sets

Treat a reference set as an artifact with an identifier:

refsets/
  presenter-a/
    v1/  images/  voice.wav  manifest.json
    v2/  images/  voice.wav  manifest.json

The manifest should record where the assets came from, what normalization was applied, and a hash of each file. When an output drifts, you can tell immediately whether the reference set changed. When it drifts and the set did not change, you know to look at the prompt or at the model version instead. Keep sets small — a handful of well-chosen images beats a large, inconsistent pile — and keep them versioned, because reproducibility is the whole point.

Assembling the Request in Your Codebase

The request for a reference-to-video call has three conceptual parts: the model selection, the prompt describing the action, and the reference payload containing your multi-image and voice assets. Keep each part in its own layer so that changing one does not force you to touch the others.

A workable structure

config/          model id, endpoint, default parameters
refsets/         versioned reference assets + manifest
builder/         assembles prompt + refset + params into a request
client/          HiAPI call, retries, error mapping
handler/         validates and stores the returned video

A request builder in this shape does roughly the following:

1. Load the model identifier from configuration (for example vidu-q4-preview/reference-to-video), never from a literal buried in the caller.
2. Load the reference set by version from the manifest, so the exact assets used are recorded with the run.
3. Attach the prompt text that describes the scene, motion, and pacing.
4. Merge default parameters with any per-run overrides.
5. Emit a request object that the client layer submits to the HiAPI API.

Keep the seams clean

The builder is the only place that knows how the request is shaped. If a field name changes, or if a parameter is added to the reference-to-video model, the change lands in the builder and nowhere else. Prompt text should be a template with slots for variable content — product name, scene beat, call to action — rather than string concatenation scattered across the calling code.

Validate before you send

Fail fast in the builder: assert that the reference set resolves, that the expected number of images is present, that audio is non-empty, and that the prompt is within limits. A malformed request rejected locally costs milliseconds; the same request rejected remotely costs a round trip and, depending on how you meter, a failed task.

Record provenance

Log the model identifier, reference set version, prompt template version, and any parameter overrides with every submitted job. That single log line is what turns "the output got worse" into an answerable question. Verify current identifiers and parameter names against HiAPI's public documentation when you wire this up, and re-check them when you upgrade the model.

Handling the Response and Short-Form Post-Processing

The returned asset is a starting point, not the finished deliverable. Short-form platforms impose their own aspect ratios, durations, loudness norms, and caption conventions, and the returned video and its native audio need to be shaped to fit them.

Store first, transform second

Write the raw returned video to durable storage before touching it. Transformation is lossy and occasionally buggy; if the only copy is the cropped one, a bad crop means a regeneration. Key the stored object by job id plus reference set version so you can trace any published clip back to its inputs.

The post-processing chain

A typical short-form chain looks like this:

• Inspect. Read duration, dimensions, frame rate, and audio loudness from the returned asset and record them in your job metadata.
• Trim. Cut to the target duration, choosing in/out points deliberately rather than slicing at a fixed offset — generation often leaves a beat of instability at the head.
• Reframe. Scale and crop to the platform's aspect ratio. If the subject is off-center, use a subject-aware crop rather than a blind center cut.
• Normalize audio. Apply loudness normalization to the native audio track, and verify the voice reference carried through as intended.
• Caption and brand. Add burned-in captions or overlays as a separate step, so they can be changed without re-encoding the base video.
• Encode. Produce the final profile per destination platform, keeping the intermediate master.

Handle audio explicitly

If the model returns native audio, treat the audio track as an independent artifact. Extract it, measure it, normalize it, and mux it back. That makes it possible to fix a quiet track without re-running generation, and it makes sync problems visible early.

Fail loudly on validation

After processing, run automated checks: duration within bounds, correct aspect ratio, audio present and above a loudness floor, no black frames at the head. Anything that fails goes to a review queue rather than to distribution. Short-form volume makes manual review of every clip impossible, so the validation gate is what keeps bad output from shipping.

Cost, Latency, and Failure-Mode Checklist

Generation is metered per task and takes real wall-clock time, so the operational questions are the same as for any expensive asynchronous job: how do I avoid wasting runs, and how do I diagnose the ones that fail? Use the checklist below when triaging a failed reference-to-video run.

Input causes

• Does the reference set resolve, and is it the version you intended?
• Are the images consistent in subject, styling, and color treatment?
• Is the voice reference clean, free of music and overlapping speech, and non-empty?
• Are dimensions, formats, and sample rates within the documented limits?

Request causes

• Is the model identifier spelled exactly as documented, and still current?
• Are parameter names and value types valid for this model?
• Is the prompt within length limits, and does it describe action rather than re-describing identity already covered by references?
• Are you exceeding any concurrency or rate limits imposed by your own client layer?

Response causes

• Did the request succeed but return an asset that failed your validation gate? Check trimming and head-of-clip stability first; a longer generation window often fixes a clip that starts mid-motion.
• Is the output off-model relative to the references? That is almost always a reference consistency problem, not a prompt problem.
• Are you seeing sporadic timeouts? Add idempotent job tracking so a retry does not become a duplicate charge.

Cost and latency hygiene

• No blind retries. Retry only on transport errors or explicit retryable statuses. A content failure will fail again.
• Validate inputs locally. Every locally caught error is a run you did not pay for.
• Measure latency yourself. Record submit-to-ready time per model and per parameter profile on your own workload rather than assuming a number.
• Cache aggressively. A vetted output can serve many downstream variants; only regenerate the base when references or the scene actually change.
• Cap the blast radius. Batch jobs should have a max-failure threshold that halts the queue when something systemic breaks.

Keep this list next to your logs. Most failed runs resolve to one of the three branches above within a minute.

Putting It Into a Repeatable Pipeline

The final step is turning the workflow into a loop that produces short-form clips on demand from templates and a reference library, with each reviewed output feeding back into the library.

The loop

The pipeline has four stages that run repeatedly:

1. Template + reference selection. A template holds the prompt skeleton, parameter defaults, and post-processing profile. A reference library holds versioned asset sets. A job pairs one of each.
2. Generation. The request builder assembles the call, the client submits it, and the returned asset lands in storage with provenance metadata.
3. Review and post-process. Clips pass the validation gate automatically; anything flagged goes to human review. Approved clips continue through trimming, reframing, audio normalization, and encoding.
4. Feedback. Structured notes from review — "voice drifted," "lighting inconsistent" — become changes to the reference library or the template, not ad-hoc prompt edits. Bump the version so the change is traceable.

What makes it repeatable

• Templates over prompts. Anything that recurs across jobs belongs in a template, not in per-run code.
• Version everything. Reference sets, templates, model identifiers, post-processing profiles. The job log should let you reconstruct exactly what ran.
• Deterministic packaging. Build the queue from data — a spreadsheet of scenes, a content calendar — and let the pipeline expand it into jobs.
• Review as data. Capture review outcomes in a structured store so you can measure which templates and reference sets produce the most approved clips.

Where to start

Begin with one template and one reference set, run a small batch, and tune the reference set before expanding. The pipeline's value comes from the reference library compounding over time: each approved clip validates the assets behind it, and each rejected one points at exactly what to fix.

When you are ready to wire it up, check the current model identifier and request shape in HiAPI's documentation for vidu-q4-preview/reference-to-video, and keep the model selection in configuration so the rest of your pipeline survives the next model release.