Brown CS News

Click the links that follow for more news items about Michael Littman, Peter Norvig, and the Randy F. Pausch '82 Computer Science Undergraduate Summer Research Award.

The Randy F. Pausch '82 Computer Science Undergraduate Summer Research Award, given this year to Evan Cater to support his work with Brown CS Professor Michael Littman, recognizes strong achievement from young students and offers them the opportunity to partner with faculty and advance work that began in the  undergraduate research program.

A generous gift from Peter Norvig '78 (a Director of Research at Google and a thought leader in the areas of artificial intelligence, natural language processing, information retrieval, and software engineering) established the award, which provides $10,000 annually to support an undergraduate engaged in an intensive faculty-student summer research partnership. The gift honors the life and work of Randy F. Pausch '82, a renowned expert in computer science, human-computer interaction, and design who died of complications from pancreatic cancer in 2008. "His story is inspiring," Peter says, "and this is an opportunity to remember him."

Evan explains that he began collaborating with Michael as a first-year student, working alongside a Master's student on Variational Autoencoders with Deep Q-Networks, a type of reinforcement learning (RL) algorithm. "We couldn't get the idea working," he says, "and the project fizzled out, but it piqued my interest in both RL and Variational Inference. I've revisited these ideas in conjunction over the past couple of years, and this proposal and research project will hopefully synthesize my thoughts on the subject."

And what might that look like? First, some background.

"Any time we have robotics or computer programs interacting with the real world," Evan says, "there exists a lot of uncertainty. When deploying a program that, for example, modulates the power grid, or a self-driving bus that carts children to school, a lot is at stake. Storms can impede driving, hard drives can fail, car accidents happen, and programs can seg-fault. When faced with uncertainty, we want to make sure our Artificial Intelligence can make decisions. The agents interact with the world over time, so we can formalize this problem as learning good policies (or actions) over time, under uncertainty."

To construct agents that operate under uncertainty, he says, it's necessary to categorize uncertainties. For example, epistemic uncertainty is created by a lack of data, whereas aleatoric uncertainty is the result of random changes in the environment.

"My work," Evan says, "is a research study on a unified view of uncertainty in Deep Reinforcement Learning. First, we'll provide a taxonomy and vocabulary to describe the various uncertainties used in reinforcement learning. Next, we'll compare and contrast different contemporary papers, clearly distinguishing the types of uncertainty each paper considers and how the papers interact. The study will also provide some theoretical comparisons between the types of uncertainty. Finally, we'll use the study to inform the design of a new set of algorithms based on the insights ascertained. My hypothesis is that techniques from Variational Inference can be used as a unifying tool for the competing techniques."

With the summer only months away, Evan tells us that he can't wait to explore, read, and tinker: "I'm ecstatic and really thankful to all the students and professors that got me here. I remember being a first-year student sitting in on Michael's lab meetings and catching every tenth word. Flustered, a couple of seniors and PhD students took me under their wings and pointed me in the right direction. I owe everything to those student mentors and Michael for nurturing me as a researcher. Michael has been helpful and understanding, supportive during the rough weeks, and he sparks creativity in all of his students. I often look forward to our exploratory debates and deep dives into 'what is really going on'." 

Evan's eagerness and excitement is exactly what Peter Norvig is looking for. He sees this award as a multiplier that will amplify the value of his gift and extend it through time. "In the past," he says, "we had to build all our own tools, and we didn't have time to combine computer science with other fields. Now, there are so many opportunities to do so. I think it's a wise choice: you invest in things that you think will do good, and educating a student allows them to help add to the things that you're already trying to accomplish." 

For more information, click the link that follows to contact Brown CS Communication Outreach Specialist Jesse C. Polhemus.