ViMax

ViMax is an open-source agentic video generation framework developed by the Hong Kong University Data Science Lab (HKUDS). With over 11,100 GitHub stars and 1,600 forks, it represents a novel approach to video creation where specialized AI agents collaborate as a virtual film production team.

Hosted at github.com/HKUDS/ViMax, ViMax automates the full video production pipeline through four specialized agent roles — Director, Screenwriter, Producer, and Video Generator — each responsible for a distinct phase of the creative process. The project features 362 commits across active development.

• Agentic Film Production - Four specialized AI agents replicate the roles of a real film production team.
• End-to-End Pipeline - From concept to rendered video, all stages are handled automatically.
• Prompt-Driven Workflow - Start with a simple text description and get a fully produced video.
• Modular Agent Runtime - Each agent runs in a sandboxed runtime with specific tools and permissions.
• TUI Interface - Terminal-based user interface for monitoring and controlling the production pipeline.
• Rate Limiting & Resource Management - Built-in rate limiters and media resource management for stable operation.

Role	Responsibility	Output
Director	Creative vision, story concept, visual style	Creative brief and style guide
Screenwriter	Script writing, scene descriptions, dialogue	Full screenplay with scene breakdowns
Producer	Asset management, scheduling, quality control	Production plan and rendered clips
Video Generator	Frame generation, compositing, final render	Completed video file

ViMax's workflow follows a sequential pipeline where each agent receives the output of the previous one. The process begins when the Director agent translates a user prompt into a creative brief. The Screenwriter develops this into a full script with scene descriptions. The Producer plans the production, sourcing assets and managing resources. Finally, the Video Generator renders the frames and composes the final video.

The framework includes an agent loop that supports feedback and iteration, allowing earlier stages to be refined based on later results. This iterative process uses an Observe-Orient-Decide-Act (OODA) loop, similar to military decision-making frameworks, adapted for creative production.

ViMax provides multiple interfaces for interacting with the video generation pipeline:

• Terminal UI (TUI) - Full-screen terminal interface for live monitoring of agent activities, video generation progress, and pipeline status.
• Python API - Programmatic interface for integrating ViMax into larger applications and workflows.
• CLI - Command-line interface for one-shot video generation from text prompts.
• Configuration - YAML-based configuration for model selection, pipeline parameters, and agent behavior customization.

git clone https://github.com/HKUDS/ViMax.git
cd ViMax
pip install -r requirements.txt
python -m vimax --prompt "Your video concept here"

ViMax requires Python 3.10+, PyTorch, and a CUDA-compatible GPU for video generation. CPU-only mode is available for development and testing but production use requires GPU acceleration.