Events

CILVR seminar: Robotic Intelligence for Solving Everyday Problems

Speaker: Mahi Shafiullah

Location: 60 Fifth Avenue, Room 7th floor open space
Videoconference link: https://nyu.zoom.us/s/94311463747

Date: Wednesday, April 23, 2025

In recent years, robot models, particularly those trained with large amounts of data, have recently shown an exciting set of real-world locomotion and manipulation capabilities. However, a bottleneck remains in the generalization abilities of such models into the messy, unstructured environments that make up our day to day life. In this talk, I will shine some light on my approach of building towards a robot intelligence that can solve everyday problems in everyday environments with a three-step approach. The first component I will talk about is building systems that can efficiently tap into diverse, real world environments for robot data without any robots, with only $25 of hardware and an iPhone. Then, I will talk about how we designed learning algorithms that can learn from such diverse data in the real world. Finally, I will show how to build a robot memory system that – using off-the-shelf large models without any robot data – can compose generalizable robot policies to perform large-scale, long-horizon mobile manipulation in more than 10 real world home environments. I will also discuss the challenges in the critical path for robot intelligence that remain unsolved, namely dexterity, full sensory perception, and meta-cognitive processes (like planning, introspection, and memory).