Panel Summary Panel Summary PANEL SUMMARY: Bowman Program Solicitation NSF 12-580, IIS Core Programs (Small-scale proposals) Computer Graphics & Visualization ------------------------------------------------------------------------------------------------------ OVERVIEW OF THE PROPOSAL: Proposes to create 3D interaction techniques for volumes for use in immersive 3D environments, to study if or how display fidelity impacts the effectiveness of volume visualization, and to create a taxonomy of tasks performed in analysis. INTELLECTUAL MERIT: + Strengths + building on interesting work; make clear what example data sets will be used; people with application science involved + 3D techniques on their taxonomy; + questions they pose are good; the answers are well defined + the first question is important and long standing; if they could settle it with their studies it would be important + taxonomy of techniques to deal with volumes and specify subregions + already have preliminary results + use what they learn to develop low cost next generation 3D vis system to be used by many (assuming that resuls 4.3 (display fidelity) user studies that use the retina display help define possibility ) +Some directions based on previous work funded by NSF. - Weaknesses - put off by images of volume cracker usage..too low quality; how would they use for high quality - how to use volume cracker for (fine) manual segmentation. - even though they talk about what interaction techniques they would use, they do not talk about what interaction technology they would use - they did not have a way to compare their experiments to real thing versus just a simulation; - (plan to use retina display at full or lower resolution to validate in user studies). * Potentially transformative? If these tests help figure out what kinds of displays are most effective in 3D volumetric immersive it could be transformative, but if the results are inconclusive, it will not be. The proposed interaction methods for selection and manipulation were interesting and could be valuable. BROADER IMPACTS: * Impacts on science and society: One thing that stood out is that the whole project was closely designed with domain scientistis, in particular investigation of how form relates to function in insects, in particular in understanding the 3D relationships among parts of the body. So this would be of relevance to at least one application domain, with others possible. * Integration of Research and Education and Broadening Participation: They have clearly defined educational program and recruiting program. They will contribute to an already existing IGERT that trains grad students at the interface between engineering and biology, where this research also resides. They plan to leverage their work by developing educational efforts in K-12 to integrate biomechanics of insects across biology, physics, and chemistry. Further, efforts to recruit underrepresented minorities would include demonstrations and presentations to Association for Women in Computing's day for middle school girls, and the Center for Enhancement of Engineering Diversity (Virginia Tech), that runs summer programs to recruit minorities. * Postdoc Mentoring Plan (if applicable): N/A DATA MANAGEMENT PLAN, including dissemination of results: 3D models and volume data sets created and shared freely with researchers if 1) scientists who are originators of data agree; 2) a research makes a request. Software created on free open source APIs will be shared freely to researchers who submit request. After the end of the project, this software will be made available on project website. Other software created on proprietary APIs will not be shared. Anonymized raw data from evaluations will be shared upon researcher request; after end of project, available on website Publications available in journals and conference proceedings; presentations available on website. if you ask nice they will give to you. Project website for future data access. No redistribution For how long after the end of the project will the public project repository be maintained? Recommendation: In order that people know about the project website, we recommend it is indicated in all publications resulting from this project that there is a project website on which data and software may be shared. COLLABORATION PLAN (if applicable): Collaboration plan well defined in proposal; however it should also explain the collaboration plan with the unpaid domain scientist and across the institutions? RESULTS FROM PRIOR NSF SUPPORT (if applicable): Well described; and useful to carry out this grant HUMAN SUBJECTS (if applicable): IRB submittied at both institutions; hope approved by june. BUDGET (if necessary): 1 grad; 0 dollars in first year. In justification yr 1 grad student not mentioned. What does that mean? otherwise all okay. ADDITIONAL SUGGESTIONS: Elaborate more on collaboration between institutions and with unpaid collaborators PANEL RECOMMENDATION: X__ Highly Competitive (HC) __ Competitive (C) __ Low Competitive (LC) __ Not Recommended for Funding by Panel (NRFP) * Justification, including key strengths and critical weaknesses: well written and has clear description of goals and milestones; proposed user studies on display fidelity innovative and useful; user interactions useful to user community. ----------------------------------------------------------------------- The summary was read by the panel, and the panel concurred that the Panel Summary accurately reflects the panel discussion. PANEL RECOMMENDATION: Highly Competitive Review 1 Rating: Very Good Review: Summary In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit. The proposed activities are (i) running experiments/user studied to determine how display fidelity affects how well users perform various data analysis tasks; (ii) develop a taxonomy to describe volume data analysis tasks in a generic and comprehensive way; (iii) develop new 3D interaction techniques for volume data analysis [based on this taxonomy] The study of the influence of display fidelity on performance in data analysis tasks is interesting. However, I think to completely validate this approach, it is not sufficient to compare various simulated environments to each other, but at least run a few experiments comparing a simulation to the real setup. The proposal suggests several interaction techniques that could be higlhy useful. However, at least in the proposal, the volume cracker interaction technique did not seem to produce high-quality volume images, but that could be improved over the course of the project. The use of this work for the design of a next-generation volume data analysis system has a lot of potential. The PIs and institutions are well qualified to conduct the proposed activities and resources are adequate. I particularly like the strong involvement of various application disciplines. In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts. The proposed work has the potential to impact a large number of science areas (biology, medicine, ...) that produce volumetric data. The proposal has an interesting eduction component including working with an IGERT. Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable Summary Statement I think the proposal is interesting but think that the studies and proposed interaction techniques could still be strengthened. Review 2 Rating: Multiple Rating: (Excellent/Very Good) Review: Summary In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit. Strengths: 1. All three stated goals of the project: relationship of displays to effectiveness of volume visualization, design of 3D interaction techniques, and a taxonomy of volume data analysis tasks, are very important, timely, and significant. 2. The PIs have a long-standing and impressive record of publications and research in these areas, including preliminary results for what they wish to do. 3. Close involvement of the domain scientists is a plus. 4. The proposed activities will greatly enhance our understanding of the best conditions under which volume visualization can be the most effective. Weakness: 1. The interaction environment depends a lot on the interaction technologies that will be used, but that part has not been fully fleshed out. However given the expertise and past record of the PIs this is not as much of a concern. In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts. Strengths: 1. Advancing our understanding of how best to visualize and interact with volume datasets should be of significant help in a number of areas, including of course the areas that are covered by the domain scientists working with the PIs including entomological biomechanics and cell communication and growth. 2. Educational interactions with the IGERT program and recruitment of women to computing. Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable Summary Statement This proposal is planning to answer three fundamental questions through a rigorous program of user studies. First, how does immersivity and display fidelity affect effectiveness of visualization. Second, it will explore innovative 3D techniques to improve the effectiveness of volume visualization. Third, it will create a taxonomy of volume data analysis tasks based on a survey of users. Normally this would be considered ambitious, but the PIs have preliminary results on all of the three areas that seem very promising. Even if this project can execute on two out of these three aims, it would have made a substantial contribution to volume visualization and interaction. Review 3 Rating: Very Good Review: Summary In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit. Strengths: The ability to perform interactive analysis of a volumetric data in an immersive environment is a challenging problem due to several known factors. In the proposed work, PI and co-I's will study the use of immersive visualization with natural 3D interactions to improve the analysis of 3D volumetric data. If successfully executed, a suite of helpful 3D interaction techniques for interactive segmentation of volume data will be developed. Furthermore, a prototype of an interactive volume data analysis system based on the lessons learned from the proposed study will be delivered. The lessons gained from the proposed project will address some of the challenges in the this field of research. The proposed development of a task taxonomy is critical and beneficial in the development of the user studies to be conducted. The early involvement of the domain scientists (via an initial user survey) is effective. The evaluation of the proposed 3D interaction techniques is well defined. The PI and the co-I's clearly have extensive experience and education to achieve the goals defined in the proposal. The list of tasks and the proposed timeline are reasonable. The collaboration with an industry company (Fluidity Software) on the design of the prototype is a plus. Weaknesses: Some parts of the study are too general and could be improved by providing more details: 'We will continue to study different tasks, application domains, and components of display fidelity through similar controlled experimentsà' The design of the next-generation volume data analysis system needs more details. There is a lack of quantification for the four criteria: Insight generation, Accuracy and efficiency, Cost, and Convenience. In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts. The overall discussion on the border impacts to the five review elements is good. Plans to impact students from the K-12 and the university levels are provided. Additional plans for impact to the underrepresented groups are also given. These plans seem to be achievable within the project time frame. Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable Summary Statement This proposal is well written with a clear description of project goals and milestones. The proposed study on the use of immersive visualization with natural 3D interactions to improve the analysis of 3D volumetric data is innovative and useful. The suite of 3D interaction techniques for interactive segmentation of volume data to be developed will be very useful to the scientific community. The proposal consists of an excellent team of experts. The proposed work will make a tangible impact to 3D interactive analysis of volumetric data. Review 4 Rating: Multiple Rating: (Excellent/Very Good) Review: Summary In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit. Interaction with volumes in immersive environments is hypothesized to be measurably helpful, but it is currently not easy to design such interactions nor are there powerful, easy to user interaction techniques to specify regions of interest for data analysis. Their goal is to change that. Therefore, if successful it would make immersive volume data analysis a tool that could be much more widely used and facilitate the analysis. Humans cannot look into volumes to see details, so the interactions cannot mimick ones that we already use in physical life. The proposal attacks a novel combination of problemsà whether there are some combinations of immersive display characteristics and bimanual user interactions that can improve immersive 3D data analysis for multiple domains, leading to the design of a new data analysis system. They have done initial user studies on whether high fidelity displays make a difference, and developed a bimanual symmetric user interface technique for virtual environments. Their work plan is based on generalization of these ideas, on which they have several specific directions they plan to attack. Then user studies will be used to validate their results and/ or lead to modifications. They have a design plan for these user studies as well. Thus their research plan is well reasoned and well organized. It seems possible that their techniques and system might make progress towards novel interactions and analysis with volumetric data. The team seems to be well qualified in the areas they propose to research. The resources available at both institutions are more than adequate to carry out the proposed research. Brown's new environment seems uniquely suited for the experiments that they plan to run. In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts. The team has embedded a domain scientist so every progressive step will also help advance the co-PIs work on physiological mechanisms of insects. So advances will benefit the understand and development of immersive environments for science but it will also benefit science. Thus, success immediately will support success for the domain scientist since he is so integral. Further, use of his data sets and findings in outreach programs to K-12 as well as undergraduates could have the effect of making biomechanics as fascinating as video games! By working with local school systems, the students will be exposed to the latest and most fascinating results and learn about them in the context of exciting immersive technology that is bimanual with movement. Of course the graduate student training in multiple disciplines with help cross fertilization and broader impact of results in both directions. Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable Summary Statement