ComfyUI-Yedp-Mocap mocap that Saves VRAM

By vramkickedin | | 2 min read
A skeleton in mocap gear holding a webcam

ComfyUI-Yedp-Mocap is a new custom node suite for ComfyUI that performs motion capture directly in your web browser. It handles the detection of poses, hands, and faces by utilizing the user's CPU or integrated graphics instead of the GPU.

Released by developer yedp123, this tool aims to help users with limited video memory generate reference data without occupying precious VRAM. This leaves graphics resources completely free for image generation tasks like Stable Diffusion and AnimateDiff.

Real-Time motion capture features

  • Real-time webcam recording with skeleton overlays for quick feedback.
  • Instant snapshot mode to capture single static poses.
  • Video and image loaders to extract rigs from pre-existing files.
  • Automatic conversion to standard OpenPose format.
  • 3D viewport for directly previewing depth and movement.

Digital artists and hobbyists working on older computers often struggle to run pose estimation and image generation at the same time. This tool solves that issue by offloading the detection work to the browser.

Developer notes and node limitations

The developer created this project to learn about ControlNet after realizing that existing models were trained on OpenPose skeletons, requiring a conversion process. Detection runs via MediaPipe in the browser, which makes it lightweight but dependent on the browser's performance.

There are some limitations to keep in mind. The generated 'Mask' output is volumetric based on bone thickness, so it works better for preventing background hallucinations. Audio export is currently disabled for stability, and the developer noted that 3D pose data might need further refinement. The dev states:

'It might be a bit rough around the edges, but if you want to experiment with it... I'm interested to know if you can make use of it,'.

This release offers a practical solution for anyone requiring motion capture data without dedicated GPU overhead. It integrates easily into existing workflows to help generate reference material efficiently.

Get ComfyUI-Yedp-Mocap on GitHub.