Events

Generating a Physical World

Speaker: Koven Yu, Stanford

Location: 60 Fifth Avenue, Room 527
Videoconference link: https://nyu.zoom.us/j/98633633277

Date: Tuesday, April 29, 2025

Generating an interactive, enlivened, and physical world enables a wide range of applications in entertainment, education, embodied AI, and creative designs. Recent image/video models have shown promise in producing realistic visuals, yet they operate purely at the pixel level and lack underlying physical grounding, leading to failures in physical fidelity and user interactivity. In this talk, I’ll introduce our recent efforts in physical world generation by grounding pixel models onto physical models. This methodology inherently incorporates physical world knowledge about 3D spatial structures and dynamics, simultaneously acquiring visual realism, physical fidelity, and user interactivity. I’ll showcase how this methodology is applied to enable fast generation of diverse worlds, with which users can interact via 3D actions.