Overview

The goal of AI has been to build complete intelligent agents, yet the field has been fragmented into a collection of problem-specific areas of study. We will first spend a few weeks in lecture covering a new approach to integrating existing AI subfields into a single agent architecture, and remainder of the semester on self-directed, semester-long research projects.

Grading will be based on a mid-semester project proposal, and a substantial open-ended final project. The projects will be multi-disciplinary in nature but students will have the opportunity to work in small groups, so they need not necessarily have expertise in the relevant areas. Graduate students welcome; undergraduates need instructor permission to enroll.

The previous incarnation of this course was a reading seminar; the old website and reading lists are here.

Instructor
George Konidaris
Office: CIT 447
Email: gdk at cs dot brown dot edu

[Back to top]

Schedule

The first class is on Tuesday January 23rd. The class meets on a Tuesday-Thursday schedule, from 1:00pm to 2:20pm in CIT 316.

[Back to top]

Project

Your project will be a substantial creative and original piece of work. I expect you to design and implement theory and experiments studying some algorithm or model that is aligned with the course content - i.e., a model that integrates at least two aspects of AI research in an interesting way.

I expect it to have approximately the length, form, and content of a conference paper at a good AI conference (though it need not be publishable). That means careful, clear, and precise writing, a well-formalized and thoughtfully evaluated point, and thorough referencing throughout. It does not necessarily have to be an original contribution, although that would be nice. If it is not, then I expect at least an original evaluation that is relevant to our topic (e.g., an existing algorithm is tried in a new domain or addresses a setting that speaks to what we have discussed).

The study will be graded on insight, completeness, and clarity. These studies can be completed in groups of between 1 and 4 people. Graduate students should work in groups of at most 2. Undergraduates taking the course should work in groups of at least 2. Mixed groups have more flexibility but should talk to me. :)

The project accounts for 100% of your grade and is due at the end of the reading period (May 5th).

There is an intermediate deadline: a 2-page project proposal due by approximately the end of February, which identifies the topic, names the group, and sketches out what you hope to accomplish in the study. I will use these to discuss the project with each group to make sure they're on an appropriate path.

[Back to top]

Reading

General background for embodiment and general AI (all quite dated):

• Elephants Don't Play Chess, R.A. Brooks.
• Shakey the Robot. N. Nilsson, ed. Technical Note 323, SRI International, April 1985. Video
• The Animat Path to AI, S.W. Wilson 1991.
• Behaviour: perception, action and intelligence - the view from situated robotics, J.C.T. Hallam and C.A. Malcolm, Philosophical Transactions of the Royal Society A (1994). (You'll need to login via Brown.)

MDPs and RL:

• Kaelbling, Leslie Pack, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996): 237-285.

Object-Oriented MDPs:

• Diuk, Carlos, Andre Cohen, and Michael L. Littman. An object-oriented representation for efficient reinforcement learning. Proceedings of the 25th international conference on Machine learning. 2008.

Abstraction in RL:

• Sutton, Richard S., Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial intelligence 112.1-2 (1999): 181-211.
  
  
• Li, Lihong, Thomas J. Walsh, and Michael L. Littman. Towards a Unified Theory of State Abstraction for MDPs. ISAIM. 2006.
• David Abel, D Ellis Hershkowitz, Michael L. Littman. Near Optimal Behavior via Approximate State Abstraction., ICML 2016.

Problem-Specific Abstractions:

• Konidaris, George. On the necessity of abstraction. Current opinion in behavioral sciences 29 (2019): 1-7.

POMDPs:

• Spaan, Matthijs TJ. Partially observable Markov decision processes. In Reinforcement Learning: The State of the Art. Springer, Berlin, Heidelberg, 2012. 387-414.

Natural Language for MDPs:

• Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, Tim Rocktaschel. A Survey of Reinforcement Learning Informed by Natural Language, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence Survey track. Pages 6309-6317.

Further Reading

Readings from the following books are recommended for more in-depth engagement with the topic, though note that all of these are now dated:

• Mind Design II, J. Haugeland, ed.
• Computers & Thought, E.A. Feigenbaum and J. Feldman, eds.
• Mindware, 2nd edition, Andy Clark, ed.
• Being There, Andy Clark.
• Cambrian Intelligence, R.A. Brooks.

• Mind and Cognition, 2nd ed, W.G. Lycan, ed.

These books and readings will be of interest to students who want to understand the probabilistic foundation of AI more deeply:

• Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, by Judea Pearl.
• Probability Theory: The Logic of Science, by E.T. Jaynes.
• Probabilistic Logic, by Nils Nilsson. In Artificial Intelligence 28 (1986), 71-85.

[Back to top]