(Fall 2021)
Overview
The goal of AI has been to build complete intelligent agents, yet the field has been fragmented into a collection of problem-specific areas of study. We will first spend a few weeks in lecture covering a new approach to integrating existing AI subfields into a single agent architecture, and remainder of the semester on self-directed, semester-long research projects.
Grading will be based on a mid-semester project proposal, and a substantial open-ended final project. The projects will be multi-disciplinary in nature but students will have the opportunity to work in small groups, so they need not necessarily have expertise in the relevant areas. All students need special permission to enroll; advanced undergraduate students welcome.
The 2018 incarnation of this course was a reading seminar; the old website and reading lists are here.
Instructor
George Konidaris
Office: CIT 447
Email: gdk at cs dot brown dot edu
Schedule
The first class is on Wednesday September 8th, meeting weekly on a Wednesday from 3-5:30pm in CIT 241 (Swig). The first few sessions will be lectures (which will be recorded and uploaded to Pantopo, and so can be watched asynchronously), after which we will break into project groups, and weekly meetings will be progress check-ins and discussions with each group.
Project
Your project will be a substantial creative and original piece of work. I expect you to design and implement theory and experiments studying some algorithm or model that is aligned with the course content - i.e., a model that integrates at least two aspects of AI research in an interesting way.
I expect it to have approximately the length, form, and content of a conference paper at a good AI conference (though it need not be publishable). That means careful, clear, and precise writing, a well-formalized and thoughtfully evaluated point, and thorough referencing throughout. It does not necessarily have to be an original contribution, although that would be nice. If it is not, then I expect at least an original evaluation that is relevant to our topic (e.g., an existing algorithm is tried in a new domain or addresses a setting that speaks to what we have discussed).
The study will be graded on insight, completeness, and clarity. These studies can be completed in groups of between 1 and 4 people. Undergraduates taking the course should work in groups of at least 3.
The project accounts for 100% of your grade and is due at the end of the reading period (December 12th).
There is an intermediate deadline: a 2-page project proposal due by approximately early October, which identifies the topic, names the group, and sketches out what you hope to accomplish in the study. I will use these to discuss the project with each group to make sure they're on an appropriate path.
Reading
General background for embodiment and general AI (all quite dated):
- Elephants Don't Play Chess, R.A. Brooks.
- Shakey the Robot. N. Nilsson, ed. Technical Note 323, SRI International, April 1985. Video
- The Animat Path to AI, S.W. Wilson 1991.
- Behaviour: perception, action and intelligence - the view from situated robotics, J.C.T. Hallam and C.A. Malcolm, Philosophical Transactions of the Royal Society A (1994). (You'll need to login via Brown.)
Structuralism:
- Kaelbling, Leslie Pack. The Foundation of Efficient Robot Learning. Science 369(6506), pages 915-916, August 2020.
MDPs and RL:
- Kaelbling, Leslie Pack, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996): 237-285.
Object-Oriented MDPs:
- Diuk, Carlos, Andre Cohen, and Michael L. Littman. An object-oriented representation for efficient reinforcement learning. Proceedings of the 25th international conference on Machine learning. 2008.
Abstraction in RL:
- Sutton, Richard S., Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial intelligence 112.1-2 (1999): 181-211.
- Li, Lihong, Thomas J. Walsh, and Michael L. Littman. Towards a Unified Theory of State Abstraction for MDPs. ISAIM. 2006.
- David Abel, D Ellis Hershkowitz, Michael L. Littman. Near Optimal Behavior via Approximate State Abstraction., ICML 2016.
Problem-Specific Abstractions:
- Konidaris, George. On the necessity of abstraction. Current opinion in behavioral sciences 29 (2019): 1-7.
POMDPs:
- Spaan, Matthijs TJ. Partially observable Markov decision processes. In Reinforcement Learning: The State of the Art. Springer, Berlin, Heidelberg, 2012. 387-414.
Natural Language for MDPs:
- Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, Tim Rocktaschel.
A Survey of Reinforcement Learning Informed by Natural Language,
Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Survey track. Pages 6309-6317.
- R. Rodriguez-Sanchez, R. Patel, and G.D. Konidaris. On the Relationship Between Structure in Natural Language and Models of Sequential Decision Processes. In The First Workshop on Language in Reinforcement Learning at ICML 2020, July 2020.
Further Reading
Readings from the following books are recommended for more in-depth engagement with the topic, though note that all of these are now dated:
- Mind Design II, J. Haugeland, ed.
- Computers & Thought, E.A. Feigenbaum and J. Feldman, eds.
- Mindware, 2nd edition, Andy Clark, ed.
- Being There, Andy Clark.
- Cambrian Intelligence, R.A. Brooks.
- Dreyfus, Herbert and Dreyfus, Stuart, Making a Mind versus Modeling the Brain: Artificial Intelligence Back at a Branchpoint. Daedalus 117(1), pages 15-43, 1988.
- Mind and Cognition, 2nd ed, W.G. Lycan, ed.
These books and readings will be of interest to students who want to understand the probabilistic foundation of AI more deeply:
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, by Judea Pearl.
- Probability Theory: The Logic of Science, by E.T. Jaynes.
- Probabilistic Logic, by Nils Nilsson. In Artificial Intelligence 28 (1986), 71-85.