Example research questions for students interested in working with me: * REINFORCEMENT LEARNING FROM PEOPLE: The sites (now defunct) "I do dog tricks" and "subservient chicken" allowed visitors to type in English commands and watch a character carry out the tasks. All of the movements and commands were hard coded, however. Is there some way that ordinary users can train rich language models and sophisticated behavior? (Perhaps this goal can be accomplished by combining ideas from Percy Liang's work and human-feedback genetic algorithms.) Related: How would a person teach COACH to turn on a bank of lights to flash in a particular pattern? (Would it help to run a Wizard of Oz study?) * SUPERVISED LEARNING FROM PEOPLE: Machine-learning algorithms typically require very very large datasets to do their jobs. If a user wants to teach a learner to recognize her house, say, that would typically require putting together a vast training set. How can we teach a learner to recognize a target object more quickly? Can we leverage the ability of people to teach underlying principles and use explanations to learn more effectively? Are the elements of Vapnik and Vashist's "privileged information" concept that can help? * AR-HRI: Can we study human robot interaction via simulated robots in augmented reality? * LEARNING TO LEARN: It doesn't make sense for a learning algorithm to spend its energy learning things that are known in advance. How can we design a meta-learning algorithm that is exposed to a series of learning problems and devises a learning algorithm that is tailored for this distribution of problems, only learning the sorts of things that are likely to change? To put it another way, how can an algorithm learn to learn by discovering and exploiting low-dimensional variability? * COMPLEXITY OF AI ALGORITHMS: By and large, the primary use of complexity theory in AI has been to show that every problem we care about is intractable. Maybe we're looking at things wrong. Is there another way to characterize programs that solve "unsolvable" problems? * VARIABLE BINDING IN DEEP NETS: Can a deep net that has variable binding built in do a better job at sound/text/image/music composition than standard Markov models? * PLANNING IN COACH: Is it possible for COACH to act planfully instead of reflexively? * READING PEOPLE: Clever Hans the horse was able to "solve" complex math problems. What it actually did was carefully watch people who knew the answer and would read their reactions to decide when to stop. Computers have exactly the opposite problem. They can perform arithmetic at super human levels, but they are lost when it comes to reading social cues. Can we build a Clever Hans bot that can answer questions by counting and watching people's reactions to decide when to stop? * CHARACTERIZING DIFFICULT POMDPS: Partially observable Markov decision processes (POMDPs) are difficult to learn and difficult to plan in. However, some POMDPs are harder than others. Interestingly, the same POMDPs that are hard for planning appear to be hard for learning algorithms (https://www.aaai.org/ocs/index.php/AAAI/AAAI12/paper/view/4906). Recent algorithms for learning PSR representations of POMDPs have shown some promise (https://web.eecs.umich.edu/~baveja/Papers/aaai2016.pdf). But, do they simply solve the same set of POMDPs that were already known to be easy? * KWIK LEARNING OF MDP CLUSTERS: Lihong Li and Emma Brunskill showed that it is possible to PAC learn a cluster of MDPs. They used the E^3 algorithm to do it. Is it possible to solve this problem in the KWIK-learning framework?