Brown CS News

This Friday, Professor Michael Littman of Brown CS won the John McCarthy Award from the International Joint Conferences on Artificial Intelligence Organization (IJCAI). The award, which is the top research honor in AI for mid-career scientists, recognizes researchers who have consistently made significant and influential contributions to the field.

In conferring the award, IJCAI cited Michael’s role in developing reinforcement learning and AI planning in partially observable systems, as well as his work in formalized state abstractions.

“In AI,” Michael said, “the problem of reinforcement learning is using experience to select a sequence of actions to take to accomplish a goal.” Sometimes, the outcomes of these actions are uncertain, requiring AI algorithms to test multiple possibilities in order to learn more about the results of each. Every time it explores a new approach, the algorithm learns more about the efficacy of that approach for accomplishing the overarching goal, but it inherently sacrifices the opportunity to exploit past methods that it already knows to be somewhat successful. That tradeoff between exploration and exploitation — or taking advantage of an approach that was somewhat successful in the past — is a key part of reinforcement learning.

Michael’s research focused on “devising approaches that make the exploration-exploitation tradeoff work both well and efficiently,” he explained. For example, his team demonstrated that adopting optimism in the face of uncertainty results in a simple yet provably effective family of AI algorithms that applies in a wide variety of decision-making scenarios. The algorithmic techniques utilized in that process bear strong similarities to “a number of other computational problems in AI and optimization,” Michael said. That finding allowed for important follow-up research efforts on the most promising class of algorithms.

Brown CS Professor George Konidaris recalled meeting Michael as a young PhD student, after he had been reading Michael’s papers for years. “It was almost surprising that he was a real person, in the same way that people can’t quite get their heads around seeing a movie star at a grocery store,” George said. “Despite his fame and reputation, I have always found him unfailingly helpful, funny, and kind.”

“The McCarthy Award is a wonderful honor and perfectly appropriate for Michael, who has made an incredible contribution to the AI community,” George added. “It is astonishing how many original ideas have come out of one man in such a short period of time.”

John McCarthy, the award’s namesake, was a principal founder of the field of artificial intelligence. His work in the development of the LISP programming languages, computer logic, and time-sharing systems resulted in crucial advancements to computer science, making him a pioneer of numerous technologies and languages that remain in use today. Recent past winners of the McCarthy Award include Professor Tuomas Sandholm of Carnegie Mellon University and Professor Daniela Rus of MIT, both of whom have made significant developments in the field of artificial intelligence.

Michael received his doctorate from Brown CS in 1996 and has been a member of the faculty since 2012. Currently co-directing Brown's Humanity-Centered Robotics Initiative, he works mainly in reinforcement learning, but has done research in machine learning, game theory, computer networking, partially observable Markov decision process solving, computer solving of analogy problems, and other areas.

Michael has earned multiple awards for teaching and research and has served on the editorial boards for the Journal of Machine Learning Research and Journal of Artificial Intelligence Research. He served as General Chair of the International Conference on Machine Learning and Program Chair of the Association for the Advancement of Artificial Intelligence (AAAI) Conference in 2013. He's also an AAAI Fellow and is general chair of the Reinforcement Learning and Decision Making Conference, held this year in Providence. He is an ACM Fellow and was an AAAS Leshner Fellow; recently, he received the AIJ Classic Paper Award at the International Joint Conference on Artificial Intelligence. Michael will serve as NSF’s division director for information and intelligent systems for the next two years.

“I’m really grateful that I was nominated for the award and that the committee selected me,” Michael said. “The past recipients are an inspiring collection of researchers.”

For more information, click the link that follows to contact Brown CS Communications Manager Jesse C. Polhemus.