Background: Pen- and Touch-based User Interfaces and Applications

Current Projects:
Digital Scholarship Lab
Past Projects:
*Pad SDK
Fluid Inking
Music Notepad
ReMarkable Texts
For decades the Brown University Graphics Group has focused on research supporting multi-modal interactive scholarship and education. The three foundation projects - Memex, Sketchpad, and Put-That-There - listed on the home page, inspired the Graphics Group to develop or sponsor such diverse projects as HES (1967), Intermedia (1985), Sketch (1996), Exploratories (1998), MathPad (2004), and the educational game Immune Attack (2004).

Between 2006 and 2009, in collaboration with Microsoft Research, the Microsoft Center on Pen-Centric Computing (MSpcc), focused on fine-grained pen-centric computing and sponsored a workshop that addressed research issues and presented a wide range of pen-based projects. During this period Dana Tennison completed his pen-based educational Organic Chemistry Ph.D. project, ChemPad.

Current work expands the pen-centric focus to include multi-touch interactive large displays (LADS) as well as an integrated development environment for the humanities (WorkTop).

History of the MSpcc (Microsoft Center on Pen-Centric Computing)


The Brown Computer Graphics Group has been at the vanguard of pen-centric computing for over a decade, having created pen-based interfaces for sketching 3D models, writing music, and creating interactive sketches of mathematics and molecules. During this same period Microsoft was also involved with pen-centric computing, starting with Pen Windows in the early 1990s, and more recently through both its creation of the TabletPC platform and through support of research inside Microsoft and at universities. In particular, Microsoft Research and the Tablet PC Product Group provided a series of gifts to the Graphics Group to pursue research in gesture recognition and applications in pen-centric computing. On March 20, 2006, these two paths formally converged through the inauguration of the Microsoft Center for Research on Pen-Centric Computing, which is housed in the Brown's Computer Science Department.

Research Goals

Andy van Dam, the director of the center, believes that "in some cases, the pen is mightier than the keyboard." However, the challenge for the center is to show that pen-based computer interfaces have a proper place alongside keyboard and mouse interfaces for applications for work and for play. Imagine, for instance, attending an introductory biochemistry class in which 2D chemical notations on an electronic whiteboard were instantly transformed into interactive displays of 3D molecules, or responding to a stroke of musical inspiration by jotting down conventional music scorings on a PDA that could interpret the handwritten notation for instant playback or structured editing. As Rick Rashid, Senior Vice President of Research at Microsoft, noted during the inauguration, "pen-based computing has the potential to alter the way students and teachers interact ... positively affect not only the educational process but our working methods and our culture."

Research Staff

During the term of the Center, van Dam's team of full-time researchers and students made a good start to realizing this vision. Together, with support from Microsoft, Brown, and NSF, they established a portfolio of high-profile pen-based applications. ChemPad, a TabletPC application for visualizing molecules, was enthusiastically received by students in Brown Chemistry Professor Matt Zimmt's introductory Organic Chemistry course. Music Notepad, a TabletPC PowerToy demonstration project previously sponsored by Microsoft, has been downloaded by students and professionals around the world and even incorporated into the curriculum in one high school. Other projects, like MathPad2 and Geompad showed similar potential, although they highlighted the need for improvements in recognition algorithms for 2D notations.