Overview

The goal of AI has been to build complete intelligent agents, yet the field has been fragmented into a collection of problem-specific areas of study. We will first spend a few weeks in lecture covering a new approach to integrating existing AI subfields into a single agent architecture, and remainder of the semester on self-directed, semester-long research projects.

Grading will be based on a mid-semester project proposal, and a substantial open-ended final project. The projects will be multi-disciplinary in nature but students will have the opportunity to work in small groups, so they need not necessarily have expertise in the relevant areas. All students need special permission to enroll; advanced undergraduate students welcome.

Note that all communication will be via the class Slack; I will send out a link to join sometime before the first class. If you have not joined by the first class and can't find the link, please email me for it.

The 2018 incarnation of this course was a reading seminar; the old website and reading lists are here.

Instructor
George Konidaris
Office: CIT 447
Email: gdk at brown dot edu

[Back to top]

Schedule

The first class is on Thursday January 23rd, meeting weekly on Tuesday and Thursday from 10:30am-11:50am in CIT 506. The first few sessions will be lectures, after which we will break into project groups, and weekly meetings will be progress check-ins and discussions with each group.

[Back to top]

Project

Your project will be a substantial creative and original piece of work. I expect you to design and implement theory and experiments studying some algorithm or model that is aligned with the course content - i.e., a model that integrates at least two aspects of AI research in an interesting way.

I expect it to have approximately the length, form, and content of a conference paper at a good AI conference (though it need not be publishable). That means careful, clear, and precise writing, a well-formalized and thoughtfully evaluated point, and thorough referencing throughout. It does not necessarily have to be an original contribution, although that would be nice. If it is not, then I expect at least an original evaluation that is relevant to our topic (e.g., an existing algorithm is tried in a new domain or addresses a setting that speaks to what we have discussed).

The study will be graded on insight, completeness, and clarity. These studies can be completed in groups of ~5 people.

The project accounts for 100% of your grade and is due at the end of the reading period (May 6th).

There is an intermediate deadline: a 2-page project proposal due by approximately mid-March, which identifies the topic, names the group, and sketches out what you hope to accomplish in the study. I will use these to discuss the project with each group to make sure they're on an appropriate path.

[Back to top]

Reading

General background for embodiment and general AI (all quite dated):

• Elephants Don't Play Chess, R.A. Brooks.
• Shakey the Robot. N. Nilsson, ed. Technical Note 323, SRI International, April 1985. Video
• The Animat Path to AI, S.W. Wilson 1991.
• Behaviour: perception, action and intelligence - the view from situated robotics, J.C.T. Hallam and C.A. Malcolm, Philosophical Transactions of the Royal Society A (1994). (You'll need to login via Brown.)

Structuralism:

• Kaelbling, Leslie Pack. The Foundation of Efficient Robot Learning. Science 369(6506), pages 915-916, August 2020.

The data-driven approach:

• Y. LeCun. A Path Towards Autonomous Machine Intelligence. 2022.
• Blaise Agüera y Arcas and Peter Norvig. Artificial General Intelligence is Already Here. Noema Magazine, October 2023.

More cognitive approaches:

• A. Laird, A. Newell, and P.S. Rosenbloom. Soar: An architecture for general intelligence, Artificial Intelligence 33(1), 1987.
• B.M. Lake, T.D. Ullman, J.B. Tenenbaum, and S.J. Gershman. Building machines that learn and think like people, Behavioral and Brain Sciences 40, 2017.

MDPs and RL:

• D. Silver, S. Singh, D. Precup, and R.S. Sutton. Reward is Enough. Artificial Intelligence 299, October 2021.
• Kaelbling, Leslie Pack, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996): 237-285.

Object-Oriented MDPs:

• Diuk, Carlos, Andre Cohen, and Michael L. Littman. An object-oriented representation for efficient reinforcement learning. Proceedings of the 25th international conference on Machine learning. 2008.

Abstraction in RL:

• Sutton, Richard S., Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial intelligence 112.1-2 (1999): 181-211.
  
  
• Li, Lihong, Thomas J. Walsh, and Michael L. Littman. Towards a Unified Theory of State Abstraction for MDPs. ISAIM. 2006.
  
  
• David Abel, D Ellis Hershkowitz, Michael L. Littman. Near Optimal Behavior via Approximate State Abstraction., ICML 2016.

Problem-Specific Abstractions:

• Konidaris, George. On the necessity of abstraction. Current opinion in behavioral sciences 29 (2019): 1-7.

POMDPs:

• Spaan, Matthijs TJ. Partially observable Markov decision processes. In Reinforcement Learning: The State of the Art. Springer, Berlin, Heidelberg, 2012. 387-414.

Natural Language for MDPs:

• Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, Tim Rocktaschel. A Survey of Reinforcement Learning Informed by Natural Language, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence Survey track. Pages 6309-6317.
  
  
• R. Rodriguez-Sanchez, R. Patel, and G.D. Konidaris. On the Relationship Between Structure in Natural Language and Models of Sequential Decision Processes. In The First Workshop on Language in Reinforcement Learning at ICML 2020, July 2020.

Further Reading

Readings from the following books are recommended for more in-depth engagement with the topic, though note that all of these are now dated:

• Mind Design II, J. Haugeland, ed.
• Computers & Thought, E.A. Feigenbaum and J. Feldman, eds.
• Mindware, 2nd edition, Andy Clark, ed.
• Being There, Andy Clark.
• Cambrian Intelligence, R.A. Brooks.
• Dreyfus, Herbert and Dreyfus, Stuart, Making a Mind versus Modeling the Brain: Artificial Intelligence Back at a Branchpoint. Daedalus 117(1), pages 15-43, 1988.

• Mind and Cognition, 2nd ed, W.G. Lycan, ed.

These books and readings will be of interest to students who want to understand the probabilistic foundation of AI more deeply:

• Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, by Judea Pearl.
• Probability Theory: The Logic of Science, by E.T. Jaynes.
• Probabilistic Logic, by Nils Nilsson. In Artificial Intelligence 28 (1986), 71-85.

[Back to top]