An Electronic Student Notebook
Thomas W. Doeppner
Steven P. Reiss
Department of Computer Science
Elli Mylonas
Allen Renear
Scholarly Technology Group
Brown University
Providence, RI
Project Description
One of the great strengths of the modern residential university is face-to-face interaction among students and faculty. This strength can be enhanced through the appropriate use of technology. We propose to develop and deploy an electronic student notebook. Based on relatively inexpensive technology, such a notebook should simplify the taking of class notes, foster better interaction in lectures, promote more informed discussion, and ease collaborative work. Overall, it should provide the students and the faculty with a better in-class experience and should enhance out-of-class interactions.
Students use notebooks today much as they have for the last century. They meticulously take notes while the professor is lecturing. They copy down what is written on the board and then annotate it. As in-class discussions proceed, they spend much of their time recording what others are saying and relating that to the topic at hand. Then they annotate their notes with their own ideas and the comments of others. As they do reading for the course, they will make additional notes on the material read. They may also make notes as they discuss the material out-of-class with their classmates. Into the notebook, they stuff whatever handouts they receive in class, again augmented with their own notes and scribbles.
We are proposing to develop and deploy an electronic student notebook. This will consist of a relatively inexpensive portable computer with a large display upon which the student can write. The notebook will be able to download lecture notes, class handouts, and even full electronic texts. The students will be able to create their own notes or annotate any of the electronic materials that were downloaded. Students will also be able to exchange notes and view other's annotations as appropriate. The instructor will be able to provide new or revised material to the class as a whole and will be able to view and annotate individual student's notes.
This electronic notebook should enhance the interactive learning experience. Students will use the notebook in lectures to receive, organize, and annotate both instructor-supplied lecture notes and other course materials, all of which will be distributed electronically. Students will be free to participate in class without having to take detailed notes. The notes they do jot down will be fully integrated with all the other course materials. Revisions to the instructor's materials are also distributed electronically; student annotations are fully reintegrated. Instructors can give quick quizzes with either anonymous or attributed answers and see the results immediately, allowing them to adjust their lectures accordingly. In discussion sections, participants will be able to walk through documents together, quickly looking at related materials, and making group and private annotations. Transcripts of what was examined will be made automatically, allowing easy review afterwards. Students can take advantage of collaboration tools to ease the crafting of group-produced documents. For example, participants in a seminar course can easily put together an annotated compendium of all papers discussed.
Several technical issues must be dealt with in order to make this vision of an electronic student notebook a reality. Several of these deal with the physical makeup of the notebook itself. Here it is first extremely important that the notebook be inexpensive, easily affordable by most students. The prototype we are currently developing has a list price of around $1000, and we would expect the price to drop in the future. Second, the notebook must be easy for students to use. This means it should have a high-quality display large enough to allow the simultaneous viewing and annotation of close to a page of material and should have both keyboard and pen input with handwriting recognition software. Again, our prototype approaches this, with a 9.4-inch 800x600 display with stylus input. Third, it must be portable with relatively long battery life. Since students will be using it in a variety of locations throughout the day, its batteries should be able to last for close to twelve hours.
A second set of technical issues deal with the notebook from the instructors point of view. It is important that the use of an electronic notebook not overly burden the instructor. Much of the material presented in courses today is available electronically in one form or another. Lecture notes, when lectures are given, are prepared electronically using PowerPoint or a similar tool. Resources are available through web pages, soft copies of papers, and on-line repositories. Class handouts are developed using an appropriate word processor. All these are typically available to the instructor and students through the Internet. It is thus essential that the electronic notebooks have access to the web and the full range of materials that will be used in courses. Our prototype uses a Proxim radio link to connect to a set of relatively inexpensive basestations for this purpose.
A final set of requirements deals with the need to make the electronic student notebook robust and reliable. The notebook must be fast enough that performance issues do not get in the way of note taking, discussions, or just reading course material. Our current prototypes are just barely fast enough, and we expect that next-generation hardware will be quite adequate to meet the anticipated needs. In addition, we are using a combination of push and pull communications techniques to facilitate and speed up the downloading of appropriate material. Both the hardware and the software of the notebook must be reliable enough that it will be available to the students when they need it and will provide reasonable guarantees that information, once written, becomes permanent. Here we are using reliable servers to permanently record and back up information from the student notebooks. The user interface provided by the notebook must be intuitive and easy to use both for technologically-minded students and for those new to technology. Here we will continue to develop and evaluate prototypes to simplify the interface as much as possible. Finally, the notebook must be able to work wherever the student happens to be. Since we can't guarantee network connectivity throughout campus and beyond, we are developing enhanced caching mechanisms that will let students do much of their work without having to be continually connected to the Internet.
The proposed research involves developing appropriate prototypes of the electronic student notebook and then deploying and evaluating their use. The prototype notebooks will be based on off-the-shelf portable devices similar to the larger Windows-CE devices. Our current prototype, for example, is based on the NEC MobilePro 800. Software for the notebook will be a combination of existing software (e.g. web browsers) and software that we will develop for annotations and note taking. The work on annotations will be developed in conjunction with our efforts and the drive toward Internet standards for storing and accessing annotations. We will also develop, as part of this project, appropriate software for network caching and access, and server-based version-managed storage of course materials and annotations. The deployment will first involve a small number of Computer Science courses, and will later be extended into a variety of courses throughout the University. The evaluation efforts will start before the first deployment and will be used to steer the overall research.
The work here will be done as a joint effort between the Brown Computer Science department and the Brown University Scholarly Technology Group, an applied research and development group that supports the application of advanced information technology in academic research, teaching and scholarly communication. A brief description of STG is given in the appendix.
Project Components
The work proposed here has three mutually synergistic components: developing a combination of hardware and software that we call the "Electronic Student Notebook," experimenting with its use in a number of different educational settings and evaluating its effectiveness, and disseminating the results both to the CISE community and to the general higher-education community. Work is already underway in the first two areas. We'll start making results available midway through the funded period. We summarize our plans for each of the areas in the next few paragraphs, then cover them in more detail in the pages that follow.
Electronic Student Notebook
Our notebook will be a lightweight (~2 pounds) battery-operated device with stylus (and, perhaps, keyboard) input, a "reasonably large" high-quality display, and moderate-speed (1-20 Mbps) wireless networking. Students should be able to view and annotate course materials, working with standard page-size units. Because of their limited displays, today's "palm-size" computers are not appropriate for this project. Though we will depend on professionally administered servers for long-term safe storage of all materials, each notebook should be able to hold the materials needed by the typical student over an entire semester or quarter.
The hardware and a portion of the software we use will be off-the-shelf and readily available, though we plan to take advantage of improving technology over the time frame of the work. We'll be adding additional software as part of ongoing research at Brown, in CS and in STG, and elsewhere. The functions of this software are to:
- Provide quick and dependable performance as well as an easy-to-use user interface through the use of caching and naming techniques.
- Support integrated annotation and browsing in conjunction with current and emerging standards such as XML and OEB.
- Furnish versioning that is integrated with the annotation, browsing, and caching support.
- Provide fast and effective communication among notebooks and between notebooks and servers so as to allow easy sharing and distribution of materials.
- Integrate all with strong security measures.
Aiding Education
In this project, students will have notebooks that they use throughout their studies. They bring them to lectures and group meetings such as seminars, discussion sections, project meetings, study sessions, etc. All materials available in electronic form are accessible with the notebook (though we probably cannot handle video and high-quality sound without additional facilities). Since lecture materials (and their updates) are provided to the students via their notebooks in an annotatable form, students needn't worry about copying down everything that is said. Lectures should become less a means for passive information transfer and more a means for actively engaging students, getting them to at least reflect on what is being said if not interact with the instructor. We'll fully support already established techniques as well, such as providing real-time feedback to instructors via quizzes administered via the notebooks [SLA].
Group meetings are a form of collaboration: participants (students with and without instructors) are collaborating to understand something, to build something, etc. Our contention is that the notebook is an enormous aid in this sort of endeavor. As in lectures, students can annotate common materials. But in this case it makes sense to have group annotations, jointly produced and both jointly and privately studied afterwards. Materials and pre-existing annotations can be examined by all participants. New materials can be introduced by any member for all to look at. A record of what has been covered can be produced automatically and made available to all. When available, web access from the notebooks during group meetings allows participants to look up appropriate references or citations during the discussion and to immediately follow-up interesting side issues that arise rather than having to defer these to later. The notebooks can further be used as an electronic whiteboard, providing a common view of annotations.
Disseminating Results
To make our results known and available to others, we'll create a web page to describe the project and distribute the software. In addition, we'll run a number of workshops to instruct our colleagues in other institutions on the use of the technology. We expect to produce several papers describing the hardware and software that will be developed as part of this project. Papers will also be written detailing the results of our experiences with the use of the notebook in classes and the appropriate evaluation of these experiences. These papers, along with appropriate (CS education, education technology) conference presentations, will provide the initial dissemination of results. They will be augmented with an appropriate set of references to web pages that describe the work and allow others to download the software.
Detailed Project Description
Pilot Project
Over the past six months we have designed and built an initial prototype of the electronic notebook in order to determine the feasibility of the proposed project. Using a grant from Microsoft, we obtained forty NEC MobilePro 800 computers running Windows CE along with Proxim RangeLAN2 wireless cards and base stations. Initially, we used the notebooks to view course materials consisting mainly of lecture notes done as PowerPoint slides with notes and then converted into HTML. These were multicast to students at the beginning of class and made generally available through HTTP. Caching is used to ensure that each machine has all the materials needed for class and subsequent assignments without having to be continuously connected to the internet. Recently we have started adding some primitive annotation capabilities, allowing handwritten annotations on the slides and typed annotations in a separate window associated with each slide.
Although we have not yet performed a careful evaluation, our experiences to date have been positive. Students are intrigued with the project and find it worthwhile. Well over half the students participated in every class, despite suffering at times from slow and sometimes unreliable software. Moreover, many of the students have been using the portable computers in their other classes for note taking and viewing the lecture even though those classes do not attempt to provide any direct support for the notebooks.
We are using the preliminary data from this pilot project to drive the requirements and research directions in this proposal. We now better understand the performance capabilities and requirements for the small diskless computers that we are using. We have a good understanding of what will be needed in terms of caching and broadcast technology in order to support such devices in a seamless manner both with and without Internet connectivity. We are starting to see the future requirements for reliable storage, versioning, and detailed annotations as well as what appropriate user interfaces for in-class usage will require.
Moreover, the initial success of this pilot project has let us obtain the promise of additional support from Microsoft as cost sharing for this proposal. This support will cover both additional, next generation hardware and the basic software needed to run, program, and maintain the electronic notebooks.
Technology
In this section we describe the technology component of the project. Though we take advantage of much off-the-shelf technology, a lot of what we are doing uses the fruits of ongoing research at Brown and elsewhere. We will take full advantage of standards where they exist.
Systems Support
The success of the project depends on our continuing development of technology that provides easy-to-use group communication, high-speed, dependable access to course materials, safe storage of data, and the ability to operate for long periods of time without connections to the Internet. In addition, appropriate security must be employed so that users and servers are properly authenticated and all access to data is mediated by proper authorization measures.
Our prototype, described in [MA], caches materials on mobile computers for relatively quick access and for continued functionality even when the server and Internet are not accessible. Materials are distributed both by standard HTTP and by multicast. At the beginning of a lecture, the notes are quickly multicast to the class. Latecomers retrieve materials using HTTP. Students follow links and get updates using HTTP. A novel feature is if the nominal server for a web page does not respond (or does not respond quickly enough), the page may be fetched from the caches of nearby computers. Thus as long as (say) the instructor's computer has cached copies of the lecture notes and of all web pages to be examined during the lecture, the class can function without Internet access.
As we expand the capabilities of the prototype so that it is useful for group discussions, we will be adding better support for the distribution and sharing of materials within small groups. This involves the dynamic creation of multicast groups to allow fast dissemination and the application of recent work on ad hoc networking (http://www.ietf.org/html.charters/manet-charter.html) so that this can be accomplished without dependence on external servers or other network facilities. The notebook's ability to find materials in neighboring caches allows the easing sharing materials within groups, even if they are not connected to the Internet. One simply refers to a document by its URL and it is found either at its home or in a neighboring cache.
Our caches are intended to allow mobile computers to operate for long periods of time without connection to the Internet. Thus we must insure not only that they are loaded with necessary materials when connected to the Internet, but also that new materials are saved on servers with appropriate consistency checking upon reconnection to the Internet after periods of disconnection. We're adapting much of what has been done in Coda [SAT] for this, though our reintegration model can be much simpler since we have no unrestricted access to shared data.
Of extreme importance, though not yet in our prototype, is adequate security. Our initial plans are to take advantage of TLS (transport layer security, formerly known as SSL-secure socket layer) to provide authentication, integrity, and, where appropriate, confidentiality. Much of this depends upon the establishment of a public-key infrastructure (PKI), something that is currently being developed by the University's CIS department. We have yet to decide upon a means for authorization, though we will look for a technique with reasonably wide acceptance.
Software Technology Development
Realizing an electronic notebook will require some research and development on the underlying software. The front end, offered on each of the mobile computers, will have to:
The back end, where the documents and annotations are stored, will have to:
All these are complicated by the requirement that the system must work both when it is directly connected to the server and when it is disconnected. We will address these issues by building on our experiences in software design and development, user-interface toolkits, client-server computing, software-development tools including software-configuration management, object-oriented databases, and web-based software development [DUB, LEJ, LIN, MEY, PAT, REI1, REI2, REI3, REI4, REI5, REI6, REI7, REI8, REI9, REI10, REI11, RU, SAR, SKA]. We will take full advantage of existing technology, particularly where standards, such as WebDAV (http://www.webdav.org) exist.
The front end will be developed as an extension of a web browser. Web browsers exist for most platforms and provide many of the facilities required for displaying most of the types of documents we require. Moreover, web browsers are under continuous development and should offer the capabilities for OEB and more complex XML documents in the near future. On top of the browser, using standard extension mechanisms (e.g. javascript), we will build the mechanisms for entering, viewing, and manipulating annotations. Here we will work in conjunction with STG to implement an appropriate front end on top of a web browser that meets the requirements they specify. We expect that this interface will be developed incrementally, with prototypes being built, evaluated by STG, and then used to define subsequent prototypes. We will also provide, in conjunction with STG, a extension to the browser that lets users specify which version of which documents they want.
The back end will be developed using techniques from software configuration management to store and access multiple versions of interconnected documents. This software will manage and version links as well as the documents themselves. It will also manage annotations, versions of annotations, and the precise attachment of annotations to portions of documents. We will study the interface and storage issues involved in maintaining consistent versions of documents, links, and annotations. Problems typically arise here when the text to which the annotation is attached is changed, deleted, or just moved. We provided several mechanisms that attempt to track the code and move the annotation intelligently. These depend on the type of change to the source, the type of annotation, and user preferences. Finally, we will ensure that the overall system is capable of working in either detached mode or in connected mode. In detached mode, any changes made to the document or annotations will be cached and then later used to create a new version of the document or annotation space. We will use standard merging technology to combine such edits when needed. In connected mode, changes will be recorded by the server, which will ensure immediate consistency among the multiple users. We will provide the ability to broadcast changes to either the document or annotations directly from detached users or from the server to connected users. This will allow a set of users who are communicating locally, but without the server, to continue to work cooperatively.
Data Representation
We need several different ways to enter material into the system so it can be made available on students' notebook computers for annotation and manipulation. The best understood scenario is the lecture course where the lecturer is already using PowerPoint slides (or transparencies that could easily be converted to PowerPoint). The lecturer in such a class will use lecture notes to guide and structure the lecture and keep the students informed of the progress of his/her arguments. Notes in outline format can be interspersed with graphics, images, and equations. This scenario is most prevalent, however, in introductory science classes and large (introductory) lecture courses in domains like economics. Faculty in smaller classes or in humanities classes often do not lecture, and when they do lecture, they typically don't use slides or transparencies. Classes in these courses are much more likely to involve the study and discussion of much larger and more continuous texts which perhaps are part or all of the reading assignment for the course, or perhaps are student assignments to be discussed by the whole class. Continuous texts could also be mixed with the lecture-note format, and with questions for discussion. Finally, all of the previous types of course materials will have to be easily interspersed with interactive materials. As these two situations are both very common at universities, a generally applicable system for using wireless tablet devices must support both. Lecture notes and continuous texts must, therefore be able to handle linking and annotation.
In addition, the system must be able to launch simulations and demonstrations and support classes with large amounts of visual material, adding images to the continuous texts and lecture notes. The visual materials may serve as "thumbnails", sketches providing a hook for annotation, or a reminder of the basic components of a more detailed image that cannot be reproduced with enough detail on a computer screen.
Requirements:
HTML is important as a least common denominator for creating small interlinked document sets with familiar tools. HTML is common and, if valid, relatively simple to process. However, other document standards, like the Open Ebook (http://www.openebook.org) and the Text Encoding Initiative (http://www.tei-c.org), as well as the capability for processing arbitrary XML element sets, will not only allow the creation of richer documents and the development of systems with greater functionality, but, perhaps most importantly, will better position the project to explore currently emerging standards and technologies. These capabilities are particularly important when preparing long documents or more complex websites, as well as for more fine-grained support of discipline-specific features of the text.
The Open Ebook specification for preparing documents for publication on electronic books will be particularly useful to explore. OEB is an XML data specification for handheld reading devices that describes two formats, a more restrictive one that resembles HTML and an extended format that can handle custom XML structures. A notebook system that supports OEB will be able to import and display ebooks from many different publishers. It will also have a ready-made display mechanism for documents in markup other than HTML. And since OEB can handle any XML document in its extended form, it can also support widely used XML/SGML DTDs like TEI, and the W3C XLink structures for linking and annotation.
Usability:
There are several off-the-shelf ways of creating this type of material. For example, since most faculty are comfortable working in MS Word or some other word processor, controlling stylesheets and using an HTML export from these programs will make integration seamless. Exported HTML can then be post-processed to create OEB. More sophisticated formats such as TEI or arbitrary XML require specialized editors, but may appeal to faculty who found the return on their investment in increased annotation functionality worth the extra effort.
Work Plan:
STG has extensive expertise in markup standards and specifications, especially XML, TEI and OEB. It has applied these forms of markup to different types of document, and has worked with the various incarnations of software packages and engines that are used to process XML based documents.
In the first year, we assume the system will be able handle basic HTML or Basic OEB. Courses undertaken with the system in the first academic year will be selected because they use primarily textual material, with some graphics. STG will investigate the structures of the documents that faculty are most likely to want to use, and provide a useful set of conversion steps in order to create valid HTML or Basic OEB. These will be created using standard text-conversion tools, and will not pretend to be generalized conversion tools. They will, however, be able to provide usable data for downloading onto the wireless devices.
In the second year, STG will be monitoring the tools that are created commercially and in the public domain that facilitate XML editing and conversion, in order to enable faculty and students to generate more sophisticated documents. If the system can handle native XML and/or extended OEB by this time, we will work to customize editing and conversion tools for XML data.
In the third year STG will be testing the most sophisticated discipline-specific XML markup and annotations systems our participants can stand, pushing the testbed to its limits in order to explore future these tools.
Annotations
Wireless systems for education raise interesting questions for annotation. Students and others heavily annotate the documents they study: to aid understanding as they read, for emphasis, to facilitate later review, to ask questions, to mark sections for discussion or for sharing, even to indicate that some action should be taken [BAR].
Useful annotation requires precise attachment and the ability to structure the space of linking and annotations, as well as providing metadata on the annotations (type, permissions, rhetorical function, etc.). STG brings expertise in all these areas, and will develop structured annotation protocols and structures that will allow creating, searching, maintaining, and sharing annotations. Sharing is critical for various reasons: students will need to give faculty feedback, questions, and other information, as well as discuss some issues with each other. Portability immediately raises issues of standards, and STG will develop protocols compatible with relevant specifications such as XLink, so they will be as widely usable as possible.
STG will also provide design expertise for annotation interfaces, and a testbed for users. Revealing the existence of annotations becomes non-trivial when the number of annotation becomes large (as it will in a community studying certain core documents). STG will design mechanisms for eliciting, inferring, and using metadata on annotations that will allow a large annotation space to be managed for effective use. The WWP has a large collection of primary documents used at over 100 universities, by an active community. Our analysis of the ways these texts are annotated will inform the design and direction of our work on the Wireless network systems.
Work Plan:
STG's role in developing and enhancing annotation will be primarily one of analysis and protocol development. We will work closely with the system architect to ensure that the annotation scheme for the wireless system conforms to standards, and to understand the features and constraints of the wireless system.
In the first year, we will work with the system architect to see how easily an XLink-compatible linking structure can be introduced into the system, and at what granularity. At the same time, we will develop a rich set of metadata, rhetorical, functional, etc., for categorizing and using annotations.
In the second year, we will continue the work of the first year, refining it and trying to achieve a richer annotation structure that can be attached to more finely specified points within the text. We will also begin to study how annotation is used by the students and faculty (this will be part of the evaluation).
In the third year, refinement continues, as does testing.
Education
The educational component is the most critical part of the project and indeed the reason for its existence. Our hypothesis is that students can get more out of and be more efficient in lectures, group sessions, and individual study with the aid of the electronic notebook. This is achieved by:
- Reducing the drudgery involved in taking notes, handling corrections and revisions to course materials, etc.
- Aiding collaborative efforts by providing appropriate tools and linking together all participants.
- Giving quick access to materials to facilitate discussion.
- Providing instructor-led interactive demos that can be annotated and reviewed.
- Presenting to the instructor information collected from students, such as answers to quizzes.
- Allowing students to interact with the teaching assistants during class to ask simple questions without disrupting the class.
Computer Science
Our initial work will be in Computer Science classes, which both are the type of class natural for us to teach, and provide a forgiving audience for systems under development. We have had a long history of using computers in the classroom at Brown [DOE, BRO1, BRO2, BRO3, STA1, BAZ1, BAZ2, BAK1, BAK2, BAK3]. We were pioneers in the use of algorithm animation for teaching and the use of broadcast and interaction techniques during lecture. However, the facilities required for much of this work are expensive and not easily exportable to other classrooms at Brown, let alone to other institutions. Furthermore, the workstations we've used tend to get in the way rather than help in normal lectures. The electronic notebook goes well beyond what we've done in the past. It is an unobtrusive aid to lectures that in addition provides most of the computation and graphics facilities of which we've taken advantage in the past.
Networking and Operating Systems Courses
The networking course, taught by Thomas Doeppner, was involved the pilot project and will continue to be part of the electronic-notebook experiment. Our goal is to make the traditional lecture more interactive and more interesting and productive for the student. We'll add improved annotation capabilities and better support for updates of course materials (so that the old annotations are properly attached to the new notes). Following on the cited work done at Brown and others' experiences with software for workstation-equipped classrooms, we'll add interactive capabilities, allowing the instructor to give both anonymous and attributed quizzes1 as well as a means for students to request clarifications (again, both anonymously and with attribution) of lecture topics. Similar use of the technology will be made in the introductory operating systems course, also taught by Doeppner.
Theory of Distributed Systems
This is another lecture course, taught by Maurice Herlihy. The format is similar to the networking and operating systems courses, but here we'll benefit from feedback from an additional instructor who is not directly involved with the design of the technology.
Introduction to Algorithms
This class, currently taught by Roberto Tamassia, was the major experiment and beneficiary of our first foray into the use of computers in the classroom [BRO2, BRO3, STA1, BAK1, BAK2, BAK3]. In addition to using the features used in the networking course, we'll distribute animated versions of the algorithms under study to the class (as applets) and, under the direction of the instructor, have them step through the code to better understand its performance [BRI1, BRI2]. Though we've provided similar functionality in the past, we'll also provide capabilities so that students can annotate the executions and review them after class.
Introduction to Compilers
Understanding compilers requires understanding the behavior of a variety of complex data structures and algorithms such as SR-parsing, symbol table structures, basic block code generation, semantic analysis by tree walking, and flow-based optimizations. These are usually taught by showing the dynamics of the algorithms and data structures, by either using animated slides or drawing and erasing a lot on the board. In either case, it is virtually impossible for the students' notes (or static handouts) to convey the actual dynamics of the algorithm. The electronic notebook will enable us to give the students dynamic notes that illustrate how the algorithms behave. Just as important, the students will be able to annotate these notes to aid their private understanding of what is happening. The course is taught by Steven Reiss.
Robots
This is course is undergoing extensive redevelopment, under the direction of Thomas Dean. The notebook will be used in lectures and also both to facilitate collaboration in group robot-design projects and as an instrument for communicating with the robots.
Topics in Distributed Computing
This course, taught by Maurice Herlihy, is a seminar course in which students read and give presentations on a collection of research papers. The talks are primarily critiques of individual papers or surveys of an area. Many of the CS faculty have been less than satisfied with this sort of course, because of the difficulty in keeping students involved in all the readings, not just those they are presenting. We will use the electronic notebooks as a tool for putting together course summaries discussing all the papers. Students have access to annotations from all class participants and will be responsible not only for what they've presented, but all the other papers as well.
Humanities
It is important to address education outside of Computer Science, both with the hope of aiding education in general, but also to learn how faculty and students in other areas use our technology. Furthermore, there are great benefits in scale if most of the university adopts this technology. STG works with a number of faculty in humanities, many of whom are enthusiastic about taking advantage of technology in their classes.
Literary history
The STG-based Brown University Women Writers Project is a digital library of over 200 texts of pre-Victorian women writers. These texts have been prepared over the last 10 years using a sophisticated SGML system optimized for literary texts and is currently in use, delivered over the Web, at over 100 universities and colleges. STG already knows a great deal about the use of these texts in classroom contexts, both with and without enabling digital technology. To provide a testbed for these devices, STG will work with selected Brown English faculty to integrate these devices into the curriculum of a women's literary history course, exploring the issues surrounding its use both in general and with specific reference to the use of annotations in lengthy texts (so far unexplored), as described above.
Creative Fiction and Non-Fiction
Brown University's creative non-fiction hypertext workshops are an ideal testbed for the system. These courses are intensely collaborative, based on the creation and sharing of new texts by the participants throughout the semester. Traditionally, writing workshops rely on the distribution of photocopies of literature to be discussed in class. Students finish their assignments, and produce duplicates as a basis for discussion and critique. In the hypertext workshop, they make copies of their work available on the network, and critique is accomplished in a classroom with projection. The handheld devices would facilitate sharing copies with each participant in the course, allowing them to browse the text independently of their colleagues in the class. Annotations could be added by individuals, the author, or by the group, and would be recorded for later review by the author. Both traditional spoken critique and written annotation will be enhanced by the technology.
Robert Arellano, English and Creative Writing, who teaches EL19 and EL100, the Hypertext Creative Writing courses, will teach the more advanced course using the wireless computers. Students in this class work in a highly collaborative manner already, and because they are writing new works, also tend to explore and exceed the capabilities of the software and hardware they use. This class would profit greatly from working with the wireless computers because they are actively investigating the potential of technology and new media.
Archaeology and Art History
Art History courses depend on a large amount of visual material, interspersed with lecture notes and primary source readings. We will work with an art history or archaeology class to present the visual and textual material to students on their notebook devices. What is most interesting in such classes is the integration of the visual with textual materials, and the investigation of how annotation occurs in this environment [INO].
Sheila Bonde, Art History, has agreed to use the wireless nodes in a new course on monasticism, which will be either a small lecture or a seminar. Its material will comprise primary sources on medieval monasticism (in translation), archaeological evidence, and architectural evidence as well as images. It will also use the material on an existing web site documenting the cathedral Bonde is excavating (http://www.wesleyan.edu/monarch/).
Language Learning
Language classes, at the intermediate and advanced levels especially, combine an emphasis on the details of grammar and syntax with a diffuse body of information on the culture and history of a language. At these levels, classes use anthologies of texts that are read, analyzed, and discussed in class. These anthologies are supported by grammatical and analytical annotations. In electronic form, they can also be supplemented with sound data, to teach pronunciation.
Masako Fidler, Slavic Studies, teaches three courses on the Czech language. Students of Czech, unlike those of more commonly taught languages like Spanish, French or Italian, usually only have a few semesters of courses available to them. At Brown, the second and third semesters are already labeled "Advanced Czech Language and Literature." Students in these classes are not all at the same level, and must all cover a great deal of material. Fidler has been developing a web-based curriculum with an anthology of excerpted texts, lexical, grammatical and interpretive annotation, audio clips of passages and individual words and exercises. She has agreed to teach Czech 135 or 136 (the advanced course) using the electronic notebooks. She will base the class on the web materials, but would profit from the increased interaction that this system would provide to her and her students. Annotations that students create could selectively also be used to enhance the website, thus becoming available to a broader public (http://www.brown.edu/Departments/CLS/Czech).
Afro-American Studies
This course, being devised by Joy James of Afro-American Studies and William McIver of CS and STG, examines the production and use of science and technology by communities across the African Diaspora. Studies of the production and use of technologies have historically focused on these activities within an elite context. People in marginalized communities have often shown remarkable abilities to both appropriate and create new technologies, often as acts of resistance to political, cultural and economic exclusion. Examples include the development of audio technologies for hip-hop music, the development of grass-roots computer net-works in Nigeria, and the use of video and web sites in political movements. They have agreed to teach this course with the electronic notebooks.
Evaluation
STG will manage the evaluation of the project, looking at the technical aspects as well as observing and evaluating the way the hardware and software are deployed and used in classes. They will perform a qualitative evaluation, based on interviews and questionnaires comparing courses taught with the notebooks with earlier versions taught in a more conventional way. Because electronic notebooks will by their very nature change classroom interactions and the way courses unfold, it is at this point of more interest to try to understand these changes, including how the faculty and students adapt to them, than to construct artificial comparisons.
Technical evaluation
To ensure that our tools and practices can be scaled up, and to explore their general significance-avoiding artificial and anomalous behaviors specific to the experimental context-we will carefully evaluate both the actual and potential conformance of project annotation architectures to emerging W3C and industry standards. STG has had extensive experience with the development of annotation tools and practices, and, specifically, the development of W3C Xlink and Xpointer protocols.
For similar reasons, we will also carefully evaluate actual and potential conformance of project document architectures to emerging W3C and industry standards. STG has also had extensive experience with the development of these standards, including SGML, XML, the Text Encoding Initiative (TEI), and the Open Electronic Book.
Evaluation of classroom use
At the beginning of each class, once during it, and at its conclusion we will distribute a questionnaire intended to collect quantitative and qualitative data from students about a) expectations and b) pros and cons. We will analyze how their expectations change. We also want to learn from them how they put the notebooks to use, and what they found particularly useful or irritating.
At the same time, we will select a few students from each class (not more than five) and interview them at the start, midpoint and end of the class, using the same set of questions. In these interviews, we will strive to get a deeper view of how the notebooks affect their learning experience.
We also want to learn from faculty how their teaching style was affected by the use of the notebook devices, as well as how easy it was for them actually to prepare materials for the system and integrate then into their teaching. We will do this by interviewing the faculty at the beginning and the end of the semester, and by observing their classes.
It is the nature of such a project that the first faculty to use it will be early adopters, willing to put up with some technical discomfort, and also creative in their uses of a new system. The project will also provide hands on help to the faculty and the classes using the devices. We will be keeping discussing the progress of each course with the students and staff who are helping the faculty.
The evaluation instruments (written and oral questionnaires) will be developed with some consulting from an experienced evaluator.
Evaluations will be analyzed and written up by STG.
Project Plan and Management
Managing the project involves the coordination of the ongoing development of the electronic notebook with its deployment in classes. Doeppner will be the director of the project. The CS PIs, working with STG, will manage and participate in the development of the notebook. Other developers will include the graduate student funded by the project as well as a number of undergraduate students working for course credit. For the early portion of the project, the PIs and STG will have weekly meetings to review progress and coordinate activities. As the project matures, monthly meetings should suffice.
Faculty who are participating by using the technology in their classes will start working with project staff in the semester before the classes, in order to plan what materials will be converted and made available through the wireless system. For faculty who already have much of their material in electronic form already (most of the CS faculty are in this category, using primarily PowerPoint and FrameMaker), this should be very easy. STG will work with humanities faculty to prepare their materials before their classes begin. STG will also teach them prepare and edit lecture notes on their own.
Deployment of the technology to classes will be staged. We'll first use it in a few CS courses, first just those of the PIs, then those of other faculty. At this point we will use CS facilities for long-term data storage. Once the notebook is reasonably stable, probably by the middle of the second year of the funded period, we'll introduce the technology to a few carefully selected humanities courses. STG will work with the associated faculty, attending some of the courses (this will also be done as part of the project evaluation).
Milestones for the project are summarized in Table 1.
Dissemination of Results
We will disseminate our results partly by making software available and partly by evangelism. We will start within Brown: we'll sponsor forums and workshops in order to get people interested in the technology and instruct them in its use. We hope to have a number of enthusiastic faculty users in a multitude of disciplines who will entice their colleagues at other institutions to use our technology. We'll set up a web site making the software available and containing links to the courses that use it. The relatively low cost of the of the hardware should make this technology accessible to a broad range of schools, universities, and their students.
In the final summer of the funded period we'll sponsor workshops for both the CISE and the higher-education communities. We plan to run both types on both the east and west coasts-at Brown and at Microsoft-organized facilities near Seattle. We will organize a repository of courseware for the student notebooks so that colleagues can exchange and enhance course materials. We will work with companies such as Microsoft and Sun who are working to put up web sites for the exchange of course materials. Microsoft technical evangelists, who are already giving talks around the country about applications of their technology, will help publicize our work as well.
Conclusions:
We are putting together exciting technology and are learning how to utilize it for effective teaching in a broad spectrum of areas. Unlike other approaches, this technology will be easily affordable by most colleges and universities. Not only will we make the technology itself available to our colleagues at other institutions, but we'll show them how it can be used for effective teaching in a variety of disciplines.
Because a major focus of our research will be exploring the adequacy of emerging data standards, the potential for national impact of this project is enormous. There is no better location for influencing the social and economic effects of technological development than in the areas that test and affect the nature of proposed data standards. Over and over we have seen the huge social and economic costs of the failure to standardize, premature standardization, the adoption of inadequate low-function standards, the adoption of inappropriate standards, and the adoption of mismatched combinations of standards. Many of these problems result directly from inadequate research-based knowledge of the issues, or a failure to disseminate, promote, and defend this knowledge in appropriate venues (not just the scientific periodical literature) in time to affect commercial developments. Therefore, the most important way to avoid or minimize these problems is to develop and make available a critical mass of research-based knowledge-before fast-moving commercial developments have made this knowledge irrelevant. This research must be conducted from a commercially disinterested perspective and the results made both generally available, and, more specifically, presented in a compelling form in the standards development bodies themselves. Fortunately, this is work that STG is well suited for, having been extensively involved in standards development and testing for many years, and having researchers currently serving on the various relevant standards bodies in the W3C and OEBF.
Appendix: The Scholarly Technology Group
The Brown University Scholarly Technology Group (STG) is an applied research and development group that supports the application of advanced information technology in academic research, teaching, and scholarly communication. STG pursues this mission by exploring new technologies and practices, developing specialized tools and techniques, and providing consulting and project management services to academic projects.
The common theme of STG projects is their exploration of the technologies that support electronic books and digital libraries, with a particular focus on developing techniques for exploiting high-performance structured data and for integration of technology strategies with disciplinary and pedagogical methodologies. Although STG has particular strengths in the humanities, arts, and social sciences, its areas of focus cut across all disciplines and all academic levels, and include both teaching and research applications, as well as general communication technologies. Three areas of focus of particular importance to STG are:
Document Standards
For nearly 20 years STG staff have been involved in developing and applying advanced, high-function standards for document representation, most importantly standards based on SGML and XML (meta-grammars for document encoding systems) such as the TEI (Text Encoding Initiative Guidelines) and the OEBPS (Open Electronic Book Publication Structure), as well as other W3C standards, including HyTime, Xlink, and Xpointer. STG staff have participated extensively in the exploration, development, and promotion of document standards and continue to play major roles in document standards development and standards research. STG projects generally make extensive and innovative use of these standards.
Textbase Development and Delivery
Many of STG's projects involve the development and electronic delivery of sophisticated academic textbases, and the exploration of new technologies and techniques for textbase development and delivery. One of these projects, the Women Writers Project, is widely recognized as one of the most advanced academic textbases in the world; based on a specialized adaptation of the TEI, the WWP is delivered over the network for classroom and research use to over 125 universities and colleges and provides a unique testbed for studying new technologies and practices.
Hypertext and Hypermedia
For over 15 years STG staff have been involved in exploring educational and research applications of hypertext and hypermedia. STG hypermedia projects are in a wide variety of academic disciplines and levels, include research and library applications as well as classroom teaching, and range from theoretical studies to practical applications.
[BAK1] Baker, J.E., Cruz, I. F., Liotta, G., and Tamassia, R. "The Mocha Algorithm Animation System.," in Proc. Int. Workshop on Advanced Visual Interfaces, pages 248-250, 1996.
[BAK2] Baker, J.E., Cruz, I. F., Liotta, G., and Tamassia, R. "Animating Geometric Algorithms over the Web," in Proc. 12th Annu. ACM Sympos. Comput. Geom., pages C3-C4, 1996.
[BAK3] Baker, J.E., Cruz, I. F., Liotta, G., and Tamassia, R., "Algorithm Animation over the World Wide Web," in Proc. Int. Workshop on Advanced Visual Interfaces, pages 203-212, 1996.
[BAR] Bargeron, D., Gupta, A., Sanocki, E., and Grudin, J., "Annotations for Streaming Video on the Web: System Design and Usage Studies," MSR-TR-98-60, 1998.
[BAZ2] Bazik, J., Tamassia, R., Reiss, S., and van Dam, A., "Software visualization in teaching at Brown University," in Software Visualization: Programming as a Multimedia Experience, ed. J. Stasko, J. Domingue, M. Brown, pp. 383-398, MIT Press, 1998.
[BRO1] Brown, Marc H., Meyrowitz, N., van Dam, A., "Personal Computer Networks and Graphical Animation: Rationale and Practice for Education," ACM SIGCSE Bulletin, 15(1), 296-307, February, 1983.
[BRO2] Brown, M.H., Sedgewick, R., "A System for Algorithm Animation," Computer Graphics, Vol. 18, No. 3, (July 1984), pp.177-186.
[BRO3] Brown, M.H., Sedgewick, R., "Progress report: Brown University Instructional Computing Laboratory," ACM SIGCSE Bulletin, Vol. 16, No. 1, (Feb. 1984).
[BRO4] Brown, Marc H. and Robert Sedgewick, "Techniques for Algorithm Animation," IEEE Software, vol. 2, no. 1, pp. 28-39, 1985.
[BRI1] Bridgeman, S., Goodrich, M.T., Kobouroov, S.G., and Tamassia, R., "SAIL: A System for Generating, Archiving, and Retrieving Specialized Assignments Using LaTeX," Proc. 31st ACM SIGCSE Technical Symposium, 300-304, 2000.
[BRI2] Bridgeman, S., Goodrich, M.T., Kobouroov, S.G., and Tamassia, R., "PILOT: An Interactive Tool for Learning and Grading," Proc. 31st ACM SIGCSE Technical Symposium, 139-143, 2000.
[CAT] Catlin, T., Bush, P., and Yankelovich, N., "Internote: Extending a Hypermedia Framework to Support Annotative Collaboration," Hypertext'89 Proceedings, 1989.
[DOE] Doeppner, T., Sedgewick, R., van Dam, A., "A Computing Environment for Experimental Research in Computer Science," Computer Science Technical Report CS-67, Brown University (December 1980).
[DUB] Duby, Carolyn K., Meyers, S., and Reiss, S.P., "CCEL: a Metalanguage for C++," Proc. Second Usenix C++ Conference, August 1992.
[ENG] Engelbart, D. C., "Authorship Provisions in AUGMENT," in Douglas C. Engelbart, COMPCON '84 Digest: Proceedings of the COMPCON Conference, San Francisco, CA, 1984, pp. 465-472.
[INO] "Inote: An Image Annotation Tool in Java," Institute for Advanced Technology in the Humanities, University of Virginia, http://jefferson.village.virginia.edu/inote/, 1998
[JI] Ji, S., "Information Query in Trace-Based Debugging," Technical Report, Brown University Computer Science Dept., May 1994
[LAL] Laliberte, D., and Braverman, A., "A Protocol for Scalable Group and Public Annotations," NCSA Technical Report, 1997.
[LEJ] Lejter, M., Meyers S., and Reiss, S.P., "Support for Maintaining Object-Oriented Programs," IEEE Trans. on Software Engineering, vol. 18, no. 12, pp. 1045-1052, December 1992.
[LIN] Lin, Yi-Jing and Steven P. Reiss, "Configuration Management in Terms of Modules," Proc. 5th Intl. Workshop on Software Configuration Management, April 1995.
[MA] Ma, L. "A Communication and Document-Management System for an Electronic Notebook," Master's Thesis, Dept. of Computer Science, Brown University, May 2000.
[MAR] Marshall, C.C. "Toward an Ecology of Hypertext Annotation," Proceedings of Hypertext '98. New York: ACM Press, 1998b.
[MEY] Meyers, Scott and Steven P. Reiss, "An Empirical Study of Multiple-View Software Development," Software Engineering Notes, vol. 17, no. 5, pp. 47-57, December 1992.
[OHA] O'Hara, K., "Towards a Typology of Reading Goals," Rank Xerox Technical Report EPC-1996-107, 1996.
[PAT] Pato, Joseph N., Steven P. Reiss, and Marc H. Brown, "An Environment for Workstations," Proc. IEEE Conf. on Software Tools, pp. 112-117, April 1985.
[PHE] Phelps, T. A. and Wilensky, R., "Multivalent Annotations, "Proceedings of the First European Conference on Research and Advanced Technology for Digital Libraries, 1997.
[REI1] Reiss, Steven P., "PECAN: Program Development Systems that Support Multiple Views," IEEE Trans. Soft. Eng., vol. SE-11, pp. 276-284, March 1985.
[REI2] Reiss, Steven P., "Graphical Program Development with PECAN Program Development Systems," Proc. ACM SIGSOFT/SIGPLAN Symposium on Practical Software Development Environments, April 1984.
[REI3] Reiss, Steven P. and Joseph N. Pato, "Displaying Program and Data Structures," Proc. 20th Hawaii Intl. Conf. System Sciences, January 1987.
[REI4] Reiss, Steven P., "Working in the Garden Environment for Conceptual Programming," IEEE Software, vol. 4, no. 6, pp. 16-27, November 1987.
[REI5] Reiss, Steven P. and John T. Stasko, "The Brown Workstation Environment: a user-interface toolkit," in Engineering for Human-Computer Interaction, ed. G. Cockton, pp. 215-232, North-Holland, 1990.
[REI6] Reiss, Steven P., Scott Meyers, and Carolyn Duby, "Using GELO to visualize software systems," Proc. UIST `89, pp. 149-157, November 1989.
[REI7] Reiss, Steven P., "Interacting with the FIELD Environment," Software Practice and Experience, vol. 20, no. S1, pp. 89- 115, June 1990.
[REI8] Reiss, Steven P., "Connecting Tools Using Uessage Passing in the FIELD Environment," IEEE Software, vol. 7, no. 4, pp. 57-67, July 1990.
[REI9] Reiss, Steven P., FIELD: A Friendly Integrated Environment for Learning and Development, Kluwer, 1994.
[REI10] Reiss, Steven P., "Simplifying Data Integration: the Design of the Desert Software Development Environment," Proc. 18th Intl Conf on Software Engineering, pp. 398-407, March, 1996.
[REI11] Reiss, Steven P., "The Desert environment," ACM TOSEM, vol. 8, no. 4, pp. 297-342, October 1999.
[ROS] Roscheisen, M., Mogensen, M., Winograd, T., "Beyond Browsing: Shared Comments, SOAPS, Trails, and On-line Communities," presented at the Third International WWW Conference, 1995.
[RU] Rubin, Robert V., Eric J. Golin, and Steven P. Reiss, "ThinkPad: a Graphical System for Programming-by-Demonstration," IEEE Software, vol. 2, no. 2, pp. 73-78, March 1985.
[SAR] Sarkar, Manojit, Scott S. Snibbe, Oren J. Tversky, and Steven P. Reiss, "Stretching the Rubber Sheet: a Metaphor for Viewing Large Layouts on Small Screens," Proc. ACM SIGGRAPH UIST, pp. 81-91, November 1993.
[SAT] Satyanarayanan, M., Kistler, J.J., Mummert, L.B., Ebling, M.R., Kumar, P., Lu, Q., "Experience with Disconnected Operation in a Mobile Environment," In Proceedings of the ACM Symposium on Mobile and Location-Independent Computing (August 1993). USENIX, Berkeley, pp. 11-28.
[SCH1] Schickler, M.A., Mazer, M.S., and Brooks, C., "Pan-Browser Support for Annotations and Other Meta Information on the World Wide Web," Proceedings of the Fifth International World Wide Web Conference (Paris, France, May 1996), available at http://www5conf.inria.fr/fich_html/papers/P15/Overview.html.
[SKA] Skarra, Andrea H., Stanley B. Zdonik, and Steven P. Reiss, "An Object Server For an Object-Oriented Database System," Proc. Workshop on Object-Oriented Database Systems, September 1986.
[SLA] Slatin, John. "Is There a Class in this Text? Creating Knowledge in the Electronic Classroom," Sociomedia. Ed. Edward Barrett. Cambridge, MIT Press, 1992.
[STA1] Stasko, J.T., "TANGO: A Framework and System for Algorithm Animation," IEEE Computer, vol. 23, no. 9, (September 1990).
[STA2] Stasko, John, "Using Direct Manipulation to Build Algorithm Animations by Demonstration," Proc. CHI '91 Human Factors in Computing Systems, pp. 307-314, 1991.