Co-Director: Agentic Generative Video Storytelling
Yale Song · Yiwen Song · Nick Losier · Nathan Hodson · Ye Jin · Rhyard Zhu · Yan Xu · Daniel Vlasic · Carina Claassen · Jasmine Leon · Khanh G. LeViet · Zack Chomyn · Joe Timmons · Brett Slatkin · Scott Penberthy · Tomas Pfister
We propose an agentic video storytelling framework that formulates generation as a global optimization problem, utilizing Multi-Armed Bandits to navigate a creative latent space and produce high-fidelity, cohesive narratives.
摘要
While diffusion models generate high-fidelity video clips, transforming them into coherent, long-form storytelling engines remains challenging. Current agentic pipelines automate this via chained modules but suffer from semantic drift and cascading failures due to independent, handcrafted prompting. We present Co-Director, a hierarchical multi-agent framework formalizing video storytelling as a global optimization problem. To ensure semantic coherence, we introduce hierarchical parameterization: a multi-armed bandit globally steers creative directions, while local, agent-level self-refinement loops target specific modalities (e.g., storylines, keyframes) to collectively enforce global narrative and visual consistency while mitigating identity drift. This balances the exploration of novel narrative strategies with the exploitation of effective creative configurations. For evaluation, we introduce GenAD-Bench, a 400-scenario dataset of fictional products for personalized advertising. Experiments demonstrate that \moniker significantly outperforms state-of-the-art baselines, offering a principled approach that seamlessly generalizes to broader cinematic narratives. Project webpage: https://co-director-agent.github.io/