ByteDance Bernini Crafts Videos With Words, Not Pixel Paintbrushes

    
        By vramkickedin    
     | 
    
            June 8, 2026 at 2:44 pm        
    
     | 
    
        2 min read

ByteDance has released Bernini, an open-source framework that unifies video generation and editing through a semantic planning approach. Instead of controlling pixels directly, the system uses a multimodal large language model to plan edits in a conceptual embedding space, then a diffusion transformer renders the final video. The inference code, model weights, and a Gradio demo are now available under an Apache 2.0 license.

The Bernini team separated training into two stages—a planner and a renderer—so each component retains its pretrained strengths while only needing light co-training. This method lets the MLLM’s language understanding, including chain-of-thought reasoning, guide complex edits like altering a subject’s motion or inserting new objects into a scene. The result is a flexible pipeline that handles text-to-video, image editing, and reference-image-guided video manipulation with high consistency.

Semantic planning meets diffusion rendering

Key Features

Unifies video generation and editing in one framework.
MLLM planner uses chain-of-thought reasoning for edits.
DiT-based renderer synthesizes photorealistic video.
Segment-Aware 3D RoPE handles multiple visual inputs.
Planner and renderer trained separately, then co-trained.
Supports text, image, and reference-image guided tasks.
Achieves first-tier results compared to closed-source tools.
Gradio UI for single-GPU or multi-GPU inference.

Researchers and developers who need a video model that understands nuanced editing instructions will benefit most from this release. The Apache 2.0 license also makes it accessible for commercial and academic projects. Hobbyists and small studios with powerful hardware can run the tool locally to produce quality edits without relying on cloud services.

What to know before running it

The current version is optimized for Hopper GPUs like the H100, using FlashAttention-3 for speed, though it will fall back to slower attention methods on other CUDA cards. Video tasks require multiple GPUs with Ulysses sequence parallelism, while single-image generation and editing can run on a single GPU. The team provides both a complete diffusers-format model and separate checkpoint files, along with detailed case files that keep long prompts out of the command line.