Build · AI product · Generative video
Shipping generative video from model experiment to reliable product
TL;DR
I turned toy-character motion ideas into 5+ production workflows by sequencing image-prep and video models, defining anatomy-aware evaluations, and adding fallback and monitoring. The workflows reached 10K+ users per day with ~60% generation-to-publish conversion.
Skills & stack
- 01Product strategy
- 02Model evaluation & selection
- 03Prompt engineering
- 04Latency, cost, quality tradeoff
- 05Reliability & fallback design
Impact
Shipped 5+ reusable workflows across motion, storytelling, and campaigns. One motion-control workflow reached ~70% publish conversion versus a 56% average, and the product served 10K+ users per day.
- 5+
- production workflows shipped
- 10K+
- users reached per day
- ~60%
- generation-to-publish conversion
- +14 pp
- Knee Slide vs average
Motion control
- Hover to play
Knee Slide ~70% publish conversion vs 56% average
- Hover to play
Hadouken
Campaign
Background
The problem
Toy images vary in quality and anatomy, while video models introduce fidelity, motion, moderation, and reliability failures. I needed to turn a promising demo into a repeatable production workflow.
What I did
Prioritized ideas from community and cultural signals, then sequenced image-prep and video models around controllable inputs. Defined anatomy-specific fidelity gates and shipped with cross-family fallbacks, monitoring, and a release checklist.
How I built it
From demand signal to monitored production workflow
Toy-character video looked like a model problem. In production, it was a product-system problem: messy uploads, unfamiliar anatomy, motion fidelity, moderation limits, latency, and fallback behavior.
I owned the workflow from idea selection through model orchestration, evaluation, deployment, and monitoring. The result was 5+ reusable workflows reaching 10K+ users per day at ~60% generation-to-publish conversion.
Route Demand signal -> Product hypothesis -> Input architecture -> Model sequence -> Evaluation -> Release -> Monitoring
01 / Research
Research here did not mean a survey. It meant deciding what was worth building at all — a clip climbing a subreddit, a scene people quote from a movie, a joke only insiders get, sometimes a hallucinated video that came straight out of prompting.
I used community traction and cultural relevance to prioritize ideas before spending time on model testing.
The funnel
Many candidates, one filter. Reddit finds, scenes from movies, insider jokes and hallucinations all pass through human judgment before anything gets built.
“Kaiju Fight” — reference → shipped
02 / Prototype
Building the model system
The first question was not simply, “Which video model is best?”
It was, “What sequence of models gives the video model a solvable input?”
Generative media platforms
Three vendors rather than one: the same model is priced differently across them, and some make a workflow faster to assemble than others.
-
Fal.ai fal.ai -
Replicate replicate.com -
Runware runware.ai
Workflow · Kaiju Fight
Reference → video DeliveryModel and fallback decisionsGrok, Flux Klein, and when to switch model family instead of model tier
- Grok image edit was versatile, fast, and generally reliable, but could reject some protected-character inputs.
- Flux 9B Klein provided a safer alternate image-preparation route.
- A standard-to-pro fallback helped when the issue was performance.
- A different model family (for example, Kling instead of Seedance) was more useful when the failure came from moderation or IP restrictions.
03 / Implement & iterate
Diagnosing the real failure
Observe the failure -> Identify the constraint -> Change input, prompt, route, or infrastructure -> Test again
The long-leg and takeoff cases initially looked like general hallucinations. Looking across repeated failures revealed a consistent spatial problem: head-position mapping. The problem was the model’s anchor, not the visual style.
The input
A round plush with no legs — the anatomy that head-position mapping cannot survive.
Head-anchored · failure ✕
Head mapped to the fighter's head, so the plush's legs stretch to reach the ground.
Ground-anchored · fix ✓
Feet anchored to the ground, body scaled to 40–60% of the fighter.
The reference-model head-anchoring fixThe prompt that moved the character anchor from the head to the ground
I moved the character anchor to the ground, specified the intended scale, and placed those constraints inside the prompt section responsible for character positioning.
Anchor the character to the ground using its feet, bottom edge, or lowest point. Scale it proportionally from the ground up. Do not align its head with the reference fighter’s head. Element 1 and Element 2 are small toy-sized characters; their total height should be approximately 40–60% of the reference video fighters’ height. Scale down proportionally and keep them grounded.
Changing the spatial anchor fixed the underlying constraint; more surface-level prompt tuning would only have masked it.
The complete evaluation framework
Fidelity gates for humanoid, semi-humanoid, and limbless toy anatomies.
| Metric | A Humanoid | B Semi-humanoid | C Limbless |
|---|---|---|---|
| Character fidelity | High | High — shape + color + markings, not face proportions | High — silhouette + color is the whole identity |
| Skeleton / rig sanity | High | Gating — hallucinated limbs = instant fail (most fragile metric for type B) | Replace with: no phantom limbs added |
| Motion quality | High | Mid — creative locomotion is ok if coherent | Mid — sliding or floating should feel intentional |
| Prompt adherence | High | Mid — credit partial adherence if limbs cannot physically do the move | Low — score intent, not literal execution |
| BG / style consistency | Mid | Mid | Mid — scene-level metric, same weight across all |
04 / Deploy
The release checklistFrom hosted reference assets to production configuration
-
Assets
- Host the reference assets.
- Prepare the demo video and a correctly cropped cover.
-
Pricing
- Convert API cost into the product’s pricing unit.
-
Engine
- Configure the engine, provider, target, and preprocessing policy.
- Bind the user image to the workflow input.
- Connect the engine to the template.
-
Release
- Test the complete experience in beta.
- Configure the production version.
Connecting model quality to product behaviour
I used the product dashboard to connect model testing with workflow behaviour — generation success, fallback behaviour, and whether users moved from generating to publishing. A model could look strong in selected demonstrations and still expose a different pattern at real usage levels.
Reflection
The decisions happen around the model, not inside it
What I learned
- The strongest AI product decisions often happen around the model rather than inside it.
- A cleaner input can matter more than another round of prompt tuning.
- A fallback from another model family can be more useful than a more expensive version of the same model.
- An evaluation system designed around real toy anatomy reveals failures that a generic quality score would miss.
What failed
- The workflow depended fully on vendor availability. When a vendor hit very high latency or CDN errors, we needed to switch vendors flexibly — a lesson that led to switching some of our workflows onto AWS Step Functions.
- Surface-level prompt tuning delayed the real fix on head-anchoring by several rounds.