Generate Various ControlNets with ComfyUI-Yedp-Action-Director

By vramkickedin | | 2 min read
An Openpose Skeleton Tposing

ComfyUI-Yedp-Action-Director is a custom node that has a fully interactive 3D viewport directly into ComfyUI. Users can load 3D character animations from .FBX, .GLB, or .BVH files, preview them in real-time, and batch-render multiple ControlNet passes.

Developed by Yedp123, who also made ComfyUI-yedp-mocap, this tool streamlines the workflow for artists working with 3D character animations and ControlNet pipelines. It was built to handle up to 16 characters simultaneously and supports animations from sources like Mixamo, making it easier to generate consistent control over images.

Key features for animation workflows

  • Interactive 3D viewport with real-time playback scrubbing and orbit controls.
  • Multi-pass rendering generates six output types: Pose, Depth, Canny, Normal, Shaded, and Alpha.
  • Facial motion capture via webcam or video file using MediaPipe technology.
  • Multi-clip animation sequencer with automatic 0.5-second crossfade transitions between clips.
  • Support for imported 3D environments and custom camera animation paths.
  • Male and female depth mesh toggle for mixed-gender scenes.

Artists creating AI-generated video content can use this tool to maintain consistent character poses and depth information across frames. Game developers working with limited resources may find the batch rendering capabilities useful for generating training data without switching between multiple software applications.

Development notes and practical considerations

The developer notes that this project began as an exploration of ControlNet workflows. Yedp123 states,

'I designed this node with mainly mixamo in mind so I can't tell how it behaves with other services offering animations!'

The tool includes a click-to-select raycasting system and a reset button for users who get lost navigating 3D space. One known issue requires users to resize the node after adding it, as the viewport may not appear immediately at default scale.

Get the Tool

This custom node offers a practical solution for integrating 3D animation directly into image generation pipelines without requiring separate software for pose and depth map creation.

You can find the project ComfyUI-Yedp-Action-Director on GitHub.