"High-quality virtual 3D objects," says Professor Daniel Ritchie of Brown CS, "are a critical resource for many academic disciplines, as well as for industries such as CAD and furniture and displays like the Hololens and MagicLeap. Unfortunately, traditional processes for creating new 3D objects are ill-equipped to meet this demand, and alternate approaches can be time-consuming, expensive, and require expertise, or fail to produce high-quality geometry."
To help remedy this problem, Daniel has just received a National Science Foundation (NSF) CAREER Award to develop 3D generative models that satisfy the criteria of quality, variety, efficiency, interpretability, economy, and universality, enabling scalable synthesis of high-quality objects for the expanding set of applications that demand them. His solution is neurosymbolic 3D models, a new class of generative representation for 3D objects that combines the best features of both symbolic and neural models.
"The main insight," he explains, "is to use a symbolic program to model the logical part structure of a 3D object, and then to use neural networks to refine this structure into high-quality geometry. This representation supports synthesis of new objects, reconstruction of objects from real-world sensor input, and high-level editing of object structure and geometry. It also supports modeling of higher-order object properties, including kinematics and physics."
The end goal is not only to create these high-quality objects but to help democratize 3D content creation and enable massive-scale generation of synthetic 3D training data for vision and robotics. This includes releasing augmented datasets with neurosymbolic versions of objects, improvements to robotic perception systems, and educating a new generation of diverse researchers with the interdisciplinary skillset needed for neurosymbolic 3D modeling.
CAREER Awards are given in support of outstanding junior faculty teacher-scholars who excel at research, education, and integration of the two within the context of an organizational mission, and Daniel joins multiple previous Brown CS winners of the award, including (most recently) George Konidaris, Theophilus A. Benson, Stefanie Tellex, and Jeff Huang.
For more information, click the link that follows to contact Brown CS Communication Outreach Specialist Jesse C. Polhemus.