Big Ideas in Artificial Intelligence
Instructors of record: Yann LeCun, Denis Zorin
Coordinator: Alfredo Canziani
Offered: Spring 2021
This team-taught course provides a high-level overview of the keyideasand technologies that lead to revolutionary changes in Artificial Intelligence (AI) and to the explosive growth in practical applications of AI.Taught by a team of NYU's top experts in artificial intelligence lead by the Turing award winner Yann LeCun, the course will introduce students to a range of topics in fundamentals of AI and its key sub-areas including machine learning, natural language processing, computer vision, as well as its applications in several domains.
The course will be complimentary with respect to “Thinking, Learning, and Consciousness in Humans and Machines”, which considers the impact of AI technologies on society and relationship between human and artificial intelligence. Our course focuses on the key scientific ideas underlying revolutionary advances in AI technology.
The course will consist of a sequence of 14 lectures, approximately 45 min in duration each. The lectures will be pre-recorded and watched asynchronously. There will be a weekly discussion session with the presenters, and an online discussion board. Assessment will be based on brief weekly online questionnaires.
Syllabus of Lectures (tentative)
- Introduction; Deep Learning revolution in AI. (Yann LeCun)
- AI for understanding visual information: computer vision. Machine learning foundations of computer vision (Rob Fergus)
- Vision continued (Laurens van der Maaten)
- Natural language processing: machine learning foundations, language generation, dialog(Kyunghyun Cho)
- Engineering behind AI dialog systems: (Emily Dinan)
- Speech understanding and synthesis (Michael Picheny)
- Autonomous robots, reinforcement learning with robotics applications robotics. (Lerrel Pinto)
- Robotics (Andy Zheng)
- AI and games: how machines learned to beat humans in games (Noam Brown)
- AI for medicine: saving lives with machine learning. (Rajesh Ranganath)
- Machine learning in medical imaging: challenges and opportunities (Krzysztof J. Geras)
- Fairness in AI: (Emily Denton)
- Conclusion: What is next in AI? (Yann LeCun)
Lecture title: Deep Learning revolution in AI.
Lecture topics: Brief history of AI, What AI can do, basic ideas of ML, supervised learning, gradient descent, multilayer networks, ConvNets
- LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444.
Lecturer: Yann LeCun, Silver Professor of Computer Science, Data Science, and Neural Science. VP and Chief Scientist, Facebook; ACM Turing Award Laureate
Yann LeCun is VP & Chief AI Scientist at Facebook and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of Facebook AI Research and of the NYU Center for Data Science. He received an Engineering Diploma from ESIEE (Paris) and a PhD from Sorbonne Université. After a postdoc in Toronto he joined AT&T Bell Labs in 1988, and AT&T Labs in 1996 as Head of Image Processing Research. He joined NYU as a professor in 2003 and Facebook in 2013. His interests include AI machine learning, computer perception, robotics and computational neuroscience. He is the recipient of the 2018 ACM Turing Award (with Geoffrey Hinton and Yoshua Bengio) for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing", a member of the National Academy of Engineering and a Chevalier de la Légion d’Honneur.
Lecture title: AI for understanding visual information: computer vision. Machine learning foundations of computer vision
Lecture topics: Computer vision, object recognition, generative image models,, self-supervised learning.
Imagenet classification with deep convolutional neural networks, Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, NeurIPS 2012.
Deep Residual Learning for Image Recognition, Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. CVPR 2015.
Lecturer: Rob Fergus, Professor of Computer Science + DeepMind
Rob Fergus is a Professor of Computer Science at NYU. He is also a research scientist at Google DeepMind. He has degrees from Cambridge, Oxford and Caltech, all in Electrical Engineering. Before coming to NYU, he spent two years as a postdoc at MIT. His research focuses on machine learning, computer vision and deep learning.
Lecture title: Natural language processing: machine learning foundations, language generation, dialog
Lecture topics: NLP, language generation, machine translation, what happens in dialog systems like GPT.
Lecturer: Kyunghyun Cho, Associate Professor of Computer Science and Data Science
Kyunghyun Cho is an associate professor of computer science and data science at New York University and CIFAR Associate Fellow. He was a postdoctoral fellow at University of Montreal until Summer 2015 under the supervision of Prof. Yoshua Bengio, after receiving PhD and MSc degrees from Aalto University April 2011 and April 2014, respectively, under the supervision of Prof. Juha Karhunen, Dr. Tapani Raiko and Dr. Alexander Ilin. Cho and his collaborators have extensively investigated natural language processing and machine translation. This has resulted in an attention mechanism for artificial neural networks and a new paradigm in machine translation, called ‘neural machine translation.’ It has not only advanced research, but has also been adopted by industry to produce better machine translation systems. He tries his best to find a balance among machine learning, natural language processing, and life, but almost always fails to do so.
Lecture title: AI for Speech Processing: Overview and Recent Developments
Lecture topics: The lecture will cover the basics of four key long-standing problems in speech processing today: Speech Recognition, Speech Synthesis, Speaker Verification, and Spoken Language Understanding, and how these areas have been revolutionized through breakthroughs in Deep Learning
Chorowski, Jan K., Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. "Attention-based models for speech recognition." In Advances in neural information processing systems, pp. 577-585. 2015.
Wang, Yuxuan, R. J. Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang et al. "Tacotron: Towards end-to-end speech synthesis." arXiv preprint arXiv:1703.10135 (2017).
Li, Chao, Xiaokong Ma, Bing Jiang, Xiangang Li, Xuewei Zhang, Xiao Liu, Ying Cao, Ajay Kannan, and Zhenyao Zhu. "Deep speaker: an end-to-end neural speaker embedding system." arXiv preprint arXiv:1705.02304 650 (2017).
Serdyuk, Dmitriy, Yongqiang Wang, Christian Fuegen, Anuj Kumar, Baiyang Liu, and Yoshua Bengio. "Towards end-to-end spoken language understanding." In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5754-5758. IEEE, 2018.
Lecturer: Michael Picheny, Research Professor, Computer Science and Data Science, Formerly IBM T. J. Watson Research Center
Michael Picheny received his B.S., M.S., and Ph.D. degrees from the Massachusetts Institute of Technology, Cambridge. He recently joined New York University in the Courant-Computer Science and the Center for Data Science as a research professor after many years as the senior manager of speech technologies in IBM Research AI at the IBM T.J. Watson Research Center, Yorktown Heights, New York. He has worked in the speech recognition area since 1981, when he joined IBM. He served as an associate editor of IEEE Transactions on Acoustics, Speech, and Signal Processing from 1986 to 1989 and as the chair of the Speech Technical Committee of the IEEE Signal Processing Society from 2002 to 2004. He was a member of the board of the International Speech Communication Association from 2005 to 2013 and was named an International Speech Communication Association fellow in 2014. He was the general chair of the 2011 IEEE Automatic Speech Recognition and Understanding Workshop. He is currently a distinguished industry speaker of the IEEE Signal Processing Society. He has published numerous papers in journals and at conferences on nearly every aspect of speech recognition. He is a Fellow of the IEEE.
Lecture title: Autonomous robots: teaching robots to drive and fly safely
Lecture topics: This lecture will cover broad applications of machine learning techniques for a variety of real-world applications such as robotic manipulation, navigation, and self-driving cars. We will explore the core technology and ideas that makes these applications possible and discuss areas that require additional research. Topics will include: neural networks for predicting robotic action; reinforcement learning for dexterous manipulation; imitation learning from humans.
- Robots Use AI to Learn to Clean Your House
- AI Researchers Devise Cheap Data Collectio Method to Scale Training Robots
- Deep Learning Robot Takes 10 Days to Teach Itself to Grasp
Lecturer: Lerrel Pinto, Assistant Professor of Computer Science
Lerrel Pinto is an assistant professor of Computer Science. His research interests focus on mearning and computer vision for robots. He received a PhD degree from CMU in 2019; prior to that he received an MS degree from CMU in 2016, and a B.Tech in Mechanical Engineering from IIT-Guwahati. His work on large-scale learning for grasping received the Best Student Paper award at ICRA 2016. Several of his works have been featured in popular media like TechCrunch, MIT Tech Review and BuzzFeed among others.
Lecture title: AI and games: how machines learned to beat humans in games.
Lecture topics: Adversarial search and learning, regret minimization, AI for Go, AI for poker.
In order for an AI bot to be successful in the real world, it must be able to understand how humans and other agents might adapt to the behavior of the bot. This ability to adapt poses a serious challenge to learning techniques designed for stationary, single-agent environments. This lecture will focus on adversarial multi-agent settings and describe techniques used to compute equilibria in perfect-information and imperfect-information games. These techniques have led to the creation of champion AI agents in popular games such as Go and Poker.
Lecturer: Noam Brown, Research Scientist, Facebook AI Research NYC
Noam Brown is a Research Scientist at Facebook AI Research. He earned his PhD at Carnegie Mellon University. His research interests focus on multi-agent artificial intelligence and computational game theory. He has applied this research toward creating Libratus and Pluribus, the first AIs to defeat top humans in no-limit poker. Pluribus was named one of Science Magazine’s Top Ten Breakthroughs of the Year for 2019 and Libratus was a finalist for the same award in 2017. Noam received a NeurIPS Best Paper award in 2017, the 2017 Allen Newell Award for Research Excellence, and the 2019 Marvin Minsky Medal for Outstanding Achievements in AI. He was named a 2019 Innovator Under 35 by MIT Tech Review.
Lecture title: AI for medicine: saving lives with machine learning
Lecture topics: Healthcare data, models for medical data, sepsis, covid
Lecturer: Rajesh Ranganath, Assistant Professor of Computer Science
Lecture title: Machine learning in medical imaging: challenges and opportunities
Explainable AI, learning from very large images, applications in medical imaging, breast cancer diagnosis
Longer description: Although deep neural networks have already achieved a good performance in many medical image analysis tasks, their clinical implementation is slower than many anticipated a few years ago. One of the critical issues that remains outstanding is the lack of explainability of the commonly used network architectures imported from computer vision. In my talk, I will provide an overview of the challenges and opportunities in machine learning applications in medical imaging. I will also explain how to train deep neural networks, tailored to medical image analysis, in which making a prediction is inseparable from explaining it.
Wu et al. Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE TMI, 2019.
Lecturer: Krzysztof J. Geras, Assistant Professor of Radiology
Krzysztof is an assistant professor at NYU School of Medicine and an affiliated faculty at NYU Center for Data Science. His main interests are in unsupervised learning with neural networks, model compression, transfer learning, evaluation of machine learning models and applications of these techniques to medical imaging. He previously did a postdoc at NYU with Kyunghyun Cho, a PhD at the University of Edinburgh with Charles Sutton and an MSc as a visiting student at the University of Edinburgh with Amos Storkey. His BSc is from the University of Warsaw.