Student Q&A with Nur Muhammad "Mahi" Shafiullah, 2023 Apple Scholar in AI/ML

In early March, Apple announced the recipients of their annual Apple Scholars in AI/ML PhD fellowship program. The fellows—who are selected based on their innovative research as well as “their record as thought leaders and collaborators”—receive funding, internship opportunities, and mentorship from Apple researchers. Muhammad "Mahi" Shafiullah, a third year PhD candidate at Courant, was announced as one of the 22 fellows in the 2023 class. 

Mahi was born and raised in Dhaka, Bangladesh. An early interest in math and programming led him to compete in international mathematics and informatics olympiads during high school. He was a member of Bangladesh’s National Math Team from 2011-2014 and the National Informatics Team in 2012 and 2015. In 2015, Mahi relocated to the US to pursue his undergraduate studies at MIT, where he majored in math and computer science. In his junior year, Mahi began his research on the robustness and security of deep learning models with Professor Aleksander Madry. Mahi continued his work at MIT in Professor Madry’s lab until August of 2020, when he completed a Masters of Engineering degree. 

Mahi joined the Courant Institute as a PhD student in September of 2020. His advisor is Professor Lerrel Pinto. We recently caught up with Mahi to discuss his research, his time at NYU, and his hopes for the Apple fellowship: 

Do you recall what first interested you in Machine Learning? 

My entire academic career can be summarized as a journey—one where I try to solve increasingly concrete problems while maintaining intellectual rigor—starting from theoretical math through theoretical computer science, and then into robust machine learning and, finally, machine learning for robotics. I would say the reason for my interest is personal: a desire to change the world right here and right now. Machine Learning seems to be the right topic at the right time. 

Why did you decide to pursue your studies at Courant? 

Besides all the classic reasons like the amazing faculty, a long history of excelling in the mathematical sciences, and the excellent location in the middle of the vibrant ‘Big Apple,’ my primary reasons were twofold. Firstly, my current advisor Lerrel Pinto is one of the most exemplary human beings I have had the opportunity to meet. When I met with all of my potential advisors during visit days at different institutions, I recognized that they were all extremely talented. However, with Lerrel I knew that I would have the space to flourish under his supervision. I am glad to say that my instincts were right. Secondly, the floor plan at the Forbes Building (60 Fifth Avenue) resembles a giant open space—with offices made of glass walls and ample workspace in the middle. Nothing makes me happier than good collaboration, and the space seemed built to facilitate that. This made my decision easier. 

Could you describe your PhD experience a bit? How has it been working with professors and fellow students? 

My PhD experience has been atypical, at least compared to the stories I had heard before I joined Courant. This is mainly because of COVID-19, which carved out a big chunk of my first two years in the program. My first semester (Fall 2020), it was practically only Lerrel and myself coming into our office every day. The two of us alone on the fifth floor began to remind me of Willem Dafoe and Robert Pattinson in The Lighthouse. However as COVID subsided, I got to experience more of what NYU and the department has to offer, meeting with and learning from students in my lab and within CILVR. One fantastic feature of Courant—that no one told me about—is that we also have a fantastic group of Masters students, some of whom share our interest in ML and robotics! It’s been a pleasant surprise that has led to some fulfilling collaborations. I could not have performed the resulting research on my own. 

What is your current research focused on? 

Tl;dr: I work on machine learning for household robots. 

Long version: I want to give robots the ability to learn how to complete tasks by watching humans interact with their environments, and also by practicing a little bit. Robots can solve complicated industrial tasks more easily than household chores because of the Anna Karenina principle: all assembly lines are alike, but every household setup is quirky in its own way. My current research focuses on teaching the robots multimodal action priors through demonstrations. This means that the robot can watch how you interact with your home, and by doing so, potentially learn all the possible interactions. Ideally down the line, these prior observations can become the building blocks of meaningful actions in the household, learned only from unstructured demonstrations by the owner! 

Are you happy with the progress you've made? 

I am absolutely not satisfied by the progress I’ve made so far! And I believe that’s a feeling I share with many people living through a global pandemic these past few years. I lament the lost time, while also acknowledging that this is possibly the best we could have done given the cards that we were dealt. 

Do you have specific goals for the future of your project? 

A specific, non-research yet academic goal that I have is to get to know more of the robot learning community personally. This past December, I attended my first big conference since 2019. I realized that I have severely missed making new connections with people of similar interests, and I plan to actively work to correct that. In terms of a research goal, I would like to contribute to the perception systems available to the robots. If you are going to have a robot that can help you with household chores, then it must first be able to look at your home and extract actionable information from it. However our current ML models are much better at identifying dog breeds and cars than door handles and chairs. I would like to change that, either by collecting more data or by crafting new algorithms that are more suited for indoor scenes. 

What are you looking forward to the most in your new role as a 2023 Apple Scholar in AI/ML? 

I have long admired Apple for their user-centric design and their focus to detail. I am hoping that this fellowship will give me the opportunity to work with mentors from Apple who will share those human-centric values. In scientific research it can be easy to forget that, at the end of the day, we want the applications of our work to make lives better in the real world. Hopefully, working with my mentors at Apple will help me finetune my scientific method and focus my eye towards this end result.

This conversation has been condensed and edited for clarity. Headshot by Alexa Gross, courtesy of Apple.