Using happyhorse-1.1/reference-to-video via the HiAPI API: Prompt Recipes and a Short-Form Video Workflow

What reference-to-video changes in your pipeline

Text-only video generation asks the model to invent a subject from a description, which means every run can produce a different face, outfit, or product shape. Reference-to-video flips that: you supply one or more images, and the model treats them as the visual ground truth for what appears on screen. The prompt stops being a casting call and becomes a direction sheet.

In a short-form pipeline this shifts where the work lives. With text-only generation you iterate mostly on wording. With happyhorse-1.1/reference-to-video you iterate on the reference set first and the prompt second. If the subject looks wrong, the fix is usually a better image or a different ordering of images, not a longer adjective list.

Where it fits

A typical loop looks like this:

1. Collect and curate a small set of reference images for the subject and scene.
2. Order them by importance (more on this in the next section).
3. Write a shot-sequenced prompt that names stable attributes and pacing.
4. Generate, review against the references, and decide whether to change the images or the words.

The practical payoff is consistency across a series. If you are producing multiple clips featuring the same person, product, or mascot, a stable reference set is what keeps episode three looking like episode one. Treat the images as versioned assets in your repo, not throwaway uploads.

Organizing your reference image set

The model reads your images as a ranked signal. Ordering and selection are part of the prompt, so decide them deliberately.

Pick a primary subject image

The first image should be the clearest, most representative view of the subject: front-facing, well lit, unobstructed, and free of distracting background clutter. This image anchors identity. Everything else is secondary information.

Add a secondary angle only if it earns its place

A second image is useful when it resolves ambiguity the first cannot — a profile view, a different outfit state, or the back of a product. It is harmful when it introduces conflicting details, such as two different hairstyles or two logos. If you would not want the model to average the two, do not include both.

Separate subject from environment

If the scene matters, include an environment reference that shows the setting without competing for attention on the subject. Keep it visually distinct from the subject shots so the model does not blend them.

A simple ordering heuristic

• Position 1: primary subject, clean and canonical.
• Position 2: secondary subject angle, only if needed.
• Position 3+: environment or styling references.

Hygiene rules that save iterations

• Keep the reference set small and internally consistent.
• Avoid images where the subject is tiny, cropped oddly, or heavily filtered.
• Match aspect ratios where you can; wildly different framing makes the model work harder to reconcile views.
• Name files by role (subject-primary.png, scene-street.png) so your team treats them as a set.

When in doubt, cut an image rather than add one.

Wording prompts that preserve subject identity

Identity drift usually comes from vague wording. Generic descriptors like "a person" or "a stylish outfit" give the model permission to improvise. The fix is to name stable, observable attributes explicitly and keep them stable across sentences.

Name what should not change

List the attributes that define the subject: clothing colors and cuts, hair, distinctive accessories, product shape, logo placement. Repeat the critical ones in each shot description rather than assuming continuity.

Separate "locked" from "variable"

Write your prompt in two mental blocks: locked subject attributes, then per-shot action and camera. This makes it easy to see when you have accidentally introduced a contradiction, like describing a red jacket in shot one and a grey one in shot three.

Phrasing patterns that work

• Anchor first: open with the subject and its locked attributes before describing motion.
• Use consistent nouns: if the reference shows a trench coat, call it a trench coat every time, not "jacket" then "coat."
• Prefer concrete over evaluative: "short dark hair, side part" beats "great hair."
• State what stays fixed during motion: "the coat keeps its shape as the subject turns" is more useful than hoping the model infers it.
• Avoid stacking synonyms: piling on adjectives dilutes the signal.

A quick contrast

Weak: "A fashionable woman walking down a street, cinematic."

Stronger: "The woman from the reference images — short dark hair with a side part, camel trench coat, black boots — walks toward the camera at a steady pace; the coat and boots stay consistent throughout."

The second version gives the model a checklist instead of a mood.

Directing shots and pacing in one prompt

Short-form clips live or die on rhythm. A single static description produces a clip that drifts without structure. Write the prompt as an ordered sequence of shots with explicit pacing cues.

Structure as beats

Describe two to four beats, each with a subject state, an action, and a camera note. Keep each beat short. The model handles a clear three-beat sequence better than one sprawling paragraph.

Make pacing explicit

Rhythm words carry real weight: "quick cut," "slow push-in," "hold for a beat," "snap zoom," "settle." Pair them with camera language (dolly, pan, handheld) so motion and timing reinforce each other.

A beat template

Shot 1 (0-2s): [subject + locked attributes] [action], [camera move], [pace cue].
Shot 2 (2-4s): [subject state changes] [action], [camera move], [pace cue].
Shot 3 (4-6s): [resolution beat], [camera settle], [pace cue].

You do not need exact timestamps, but thinking in seconds forces you to justify each beat. If a beat has no clear visual change, cut it.

Common pacing mistakes

• Too many beats for a short clip, which forces the model to rush everything.
• No camera direction, which produces flat, static framing.
• Contradictory pace cues ("slow and energetic") in the same line.

Write the sequence, read it aloud, and cut anything that does not change what is on screen.

Resolution, duration, and budget trade-offs

Before you generate at volume, decide which resource you are spending: pixels, seconds, or iterations. They compete.

The core trade-off

Higher output resolution (such as 1080P) and longer durations both increase the cost of each generation. Since billing follows the output you request — resolution and clip length included — a long 1080P clip is the most expensive thing you can ask for. For short-form, you rarely need both maxed out.

A decision order that works

1. Fix duration first. Short-form usually means a tight window. Lock the length your platform needs, then treat everything else as adjustable.
2. Choose resolution by delivery target. If the clip is downscaled in-feed anyway, a lower resolution may be the smarter default and lets you spend budget on more attempts.
3. Budget iterations, not just outputs. A slightly lower-resolution clip you can generate three times and pick the best of often beats one expensive attempt you cannot afford to redo.

Practical guidance

• Prototype at lower resolution to validate prompts and reference sets.
• Promote to 1080P only after the shot sequence and identity hold up.
• Keep durations honest: a tight clip with clean beats reads better than a long one with filler.

Check the current pricing panel for exact rates; the numbers change and should not be hardcoded into your scripts. Build your request so resolution and duration are parameters you can flip, not constants buried in code.

A runnable request and prompt recipe library

Here is the request shape for happyhorse-1.1/reference-to-video through the HiAPI API, followed by copy-paste prompt recipes tied to the workflow above. Parameter names may vary slightly by SDK version; check the model page for the canonical schema.

Request shape

import os, requests

resp = requests.post(
    "https://api.hiapi.ai/v1/video/generations",
    headers={"Authorization": f"Bearer {os.environ['HIAPI_API_KEY']}"},
    json={
        "model": "happyhorse-1.1/reference-to-video",
        "reference_images": [
            "https://your-cdn.example/subject-primary.png",
            "https://your-cdn.example/subject-angle.png",
            "https://your-cdn.example/scene-street.png",
        ],
        "prompt": "...recipe text...",
        "resolution": "1080P",
        "duration": 6,
    },
)
print(resp.json())

Keep reference images on stable URLs, ordered as discussed: primary subject, secondary angle, environment.

Recipe 1: Product hero, single beat

The product from the reference images — matte black bottle, copper cap, label on the front — sits centered on a stone surface. Slow push-in toward the label, soft studio light, no cuts. The bottle and cap keep their exact shape and color throughout.

Recipe 2: Character walk, three beats

Shot 1 (0-2s): The woman from the reference images — short dark hair with a side part, camel trench coat, black boots — walks toward the camera, medium shot, steady handheld, brisk pace.
Shot 2 (2-4s): Quick cut to a side profile; the coat swings as she turns, camera tracks alongside, same brisk pace.
Shot 3 (4-6s): She stops and faces forward, camera settles into a slow push-in, calm finish. Hair, coat, and boots stay consistent across all shots.

Recipe 3: Environment-led opener

Shot 1 (0-2s): Wide shot of the street from the environment reference, morning light, slow pan left.
Shot 2 (2-4s): The subject from the reference images enters frame from the right — same jacket and bag — camera holds, natural pace.
Shot 3 (4-6s): Tighten to a medium shot as she looks toward the camera; the jacket and bag remain identical to the references.

Swap durations and resolution per clip without editing the prompt text. A dedicated guide with worked examples is available in the docs.

Reviewing outputs and iterating

Review is where most of the quality gain happens, and the key discipline is deciding what kind of problem you are looking at.

Evaluate against the references, not your memory

Open the reference set beside the output. Check identity attributes one by one: face and hair, clothing colors and cuts, product shape, logo placement. Drift you can name is drift you can fix.

Sort failures into two buckets

• Identity or appearance problems usually mean the reference set is weak, inconsistent, or badly ordered. Fix images first; prompting harder rarely rescues a contradictory reference set.
• Composition, motion, or pacing problems usually mean the prompt needs work: clearer beats, stronger camera notes, tighter duration.

A short review loop

1. Score identity, composition, and pacing separately.
2. If identity fails, adjust the reference images before touching the prompt.
3. If rhythm fails, rewrite the beat sequence and cut a beat.
4. Change one variable per iteration so you know what worked.
5. Once stable at lower resolution, promote to 1080P for the final pass.

Keep a small log of prompt versions and reference-set versions alongside each accepted clip. Over a series, that log becomes your real recipe library: reliable prompts, reliable images, predictable output. Iterate on the set and the words, and the model does the rest.