Interactivity and Selection

CS137 Assignment #5

1. Dates

2. Goals

1. Explore the interactive component of VR visualization design.
2. Learn about existing and postulate new metaphors, devices, and techniques for VR interaction.
3. Consider specific needs of scientific VR users.
4. Learn about storyboard techniques in describing interactive tasks.

3. Readings

From Colin Ware's Visual Thinking for Design:

• Chapter 7: "Visual and Verbal Narrative" (pp. 129-145)
• Chapter 9: "The Dance of Meaning" (pp. 165-182)

4. Assignment

Read through both parts of this assignment in order to understand the eventual Cave implementation in Part B before beginning work on Part A.

4.1 Part A: Interaction Storyboard

Overview

The goal for this assignment is to create a storyboard—a comic book-like series of image frames—outlining a narrative sequence for the VR exploration of at least eight important features of wing kinematics and flow interaction from the list below:

The storyboard should show these features being explored using 2 or more specific visualization devices or strategies from the list below, and one that you devise on your own:

Your storyboard should follow the explorations of an imaginary user navigating the VR space of the Cave and using the your devices and strategies to investigate and compare the features you include in your imagined model. You should devise a user interface to facilitate the selection and use of the various visualization devices within the model. The interface might include real objects, such as wands, buttons etc. and also virtual objects such as color palettes, pull-down menus, etc. Your interface, real and virtual, might simply be an adjustment to the existing tools in the Cave, or it might more unusual, following a design metaphor—such as driving a car—in order to be more intuitively understandable for the imagined user. However, since Part B of this assignment will involve enacting your storyboard in the Cave, you should avoid impracticality of implementation (ask the TA) or distracting complexity (for more info, see "Interface Design Details" below).

You should show a complex data set relating to wing surface variability (shape and elasticity of membrane and bone, angle of a ttack, and camber) as well as airflow surrounding the wing and in the wake (speed, direction, pressure & vorticity/turbulence). Because it is unlikely that all this information can be shown at once in a 4D environment, you must devise an interactive selection strategy, including a visual "script" or "menu" for user choices. Imagine the best way to visually highlight or isolate specific relationships, segments of data, or cause and effect "events" for study.

Procedure

Step 1. Choosing Features and Tools

Using lists and quick sketches, decide which wing/flow features you are going to include in your storyboard sequence and how they might best be visualized. Maintain a priority for legibility and effective interaction between features you might want your user to compare in the model. Take into account moving wings, virtual 3D space and time sequence. You should project yourself imaginatively into the VR experience, anticipating visual factors and interactions that might occur in the Cave. Your choices can certainly draw on your responses to previous assignments (or other people's responses).

As you refine your list of data features and tools, imagine your user conducting an exploratory process in a narrative sequence. The narrative sequence might follow one of these two options:

1. Connected Chain: a cause and effect sequence of force and consequence that supports a hypothesis and a revised hypothesis.
2. Pairings: an exploratory process of selection, combination and comparison of factors in pursuit of answer to a posed question (e.g. What is the role of vorticity in bat flight?) or hypothesis.

An example of a hypothesis/revised hypothesis that could provide narrative structure for your sequence could be:

"Vorticity is shed as discrete vortices at low speeds, but in continuous structures at higher speeds."

The revised hypothesis might be:

"Vortices are shed as discrete vortices at all speeds."

You might want to pose a hypothesis or question that involves several quantities at the same time (musculature, vorticity, camber, pressure, etc.) to provide a more interesting and challenging design problem.

Ideally, the sequence does not answer the question, but posses several possibilities, each with merit, for comparison. The science involved in your hypotheses or questions need only reflect you current understanding of the problems we have been studying.

Step 2. Interface Design Details

The tool set can include real objects (wand, glove, brush, table, other props, etc.), virtual objects (color prism, 3D icons, etc.), body gestures sensed by wiring, voice commands, and/or other interaction devices and techniques. Your interface should be as simple as possible, yet detailed. It should be intuitive in the sense that it can be easily learned, and later, operated without a "guide book". However, it should also be complete in providing a means to accomplish the required functionality. It should enhance the user's sense of connection with the virtual environment and the "realness" of the model. Finally, it should be efficiently usable by an expert, as well, with metaphorical keyboard shortcuts, for example, that will help the expert user work more quickly.

You might try to imagine a "real world" metaphor for the "mood" of your interface (i.e. "scuba diving", "dissection" or "haunted house") to give the experience a vivid, comforting, or magical character.

Concentrate on meaningful functionality and do-ability. Any metaphor is great as long as it's not detracting from a focus on visualization of the force and form. You must also keep in mind good usability by scientists exploring the real data. Consult the books in the Cave for interface options. You might try to make commonly accessed functions easily or intuitively activated. Document your designs in drawing, actual objects, or in 2D or 3D digital graphics. Documentation should be completely detailed and include diagrams for use. The documentation can be appended to your final storyboard.

Your storyboard for this assignment need not demonstrate every feature that you develop. Our final projects will provide an extended opportunity to build models that approach a full range of explorative tools. You can draw on earlier assignments for techniques and visual elements that you use here, and you will ultimately be able to approach a more comprehensive visualization strategy in the final project, using anything that you have developed during the semester. For this week, choose a set fo actions and visualization techniques that will work well together in a short narrative sequence, even if that means leaving some things out.

Step 3. Constructing the Storyboard

Draw/write a storyboard describing the various components of your visualization strategy and how they would be activated, deactivated, compared and studied by a user.

The storyboard should follow a narrative sequence (as explained in Step 1 above). Begin with very simple diagrammatic sketches to develop a strategy for presenting your toolset and its use in time and VR space in approximately 10 FRAMES. An excellent approach for planning your storyboard is to do many drawings on separate sheets of paper, and then "mix-and-match" the various frames, deleting and filling in gaps, until you get a sequence that works. For example, it is difficult to predict how "jump cuts" between frames will work until you actually see them together.

When you have your sequence, you can paste your sketches onto a piece of illustration board (18" x 24" or larger) and develop them as drawings, or redraw them in place on the board. If you are going to use photos, it will probably be easier to sketch with a pencil, make your decisions, and then set-up to take the photos. The storyboard itself should be fully understandable or even evocative of the actual experience in the Cave. Captions are fine, but the pictures should really tell the story. It is important to think of the storyboard itself as a design project. It should be legible and visually involving. Think of possible strategies for suggesting the sequencing of narrative events. Frames within frames can show scaled-up details of the action, "cutaway shots" can connect the user's actions with their result, or with the aspect of the model the viewer is concentrating on. Continuity is an important factor. Study the "Scarecrow" video or the storyboards from previous classes for techniques that might allow the viewer of your storyboard to understand the pacing and sequence of the action, as well as its narrative content in relation to the data set in the model. Comic books, movies or TV ads can also provide a model for interconnection of narrative through "montage" or sequential juxtaposition of images.

Your style of depiction is completely up to you. You can utilize a fully naturalistic drawing or photographic strategy, or simplify the scenario to a diagrammatic representation. You can also integrate these two approaches, but the result must be narratively comprehensible, and include a role for the 4D character of the model in the spatial and temporal environment of the Cave. Screenshots from the Cave itself are fine, but should be printed out and pasted onto a real board, or integrated into a fully realized storyboard in digital form. In any case, only still images are acceptable.

4.2 Part B: VR scenario implementation

Reenact your storyboard in the CAVE, making adjustments to your interface and visualization strategies if necessary. If two visualization elements will not work together, indicate a selectivity mechanism to toggle them in sequence, view them side by side, reduce one to a numbered readout or data screen etc.

You may select a smaller portion of your storyboard if building the whole thing is impractical. However, there should be at least six data features shown in three different configurations (three frames). Plan to use the "Wizard of Oz" technique with an assistant to simulate a run-through of those three different scenarios in real time for the class.

The "Wizard of Oz" technique means that you don't really code all the interaction. You fake it by having someone behind the curtain pressing keyboard keys, while you act as if the program responds to your actions directly. Use layers and frames to simulate the interaction with the environment. In class you will have to act out your scenarios by having someone helping you with the "interaction" part.

Practice your presentation with your partner so he or she knows the "commands" or "actions" that trigger events in your environment. Some interactive changes will be possible, of course: moving objects or point of view, changing colors with the color prism. If these actions are going to play an important part in your reenactment, they should be initially scripted into your storyboard.

If you like, you can build an artificial sense of scientific discovery into your VR scenarios: e.g. your design has facilitated an important insight about the effects of bone shape in keeping the bats aloft!

It is understood that this is an imaginative design process. That means the form and the animation can be a bit crude, and that what's really important is that you've thought through the interaction technique. Nonetheless, it will be easier for the class to understand your ideas if the interaction runs as smoothly as possible in your presentation.

5. Questions (answer briefly)

Below are some short questions intended to help guide you through the assignment.

1. Do your interaction techniques mimic a "real world" metaphor? If so, which one, and why is this an appropriate choice?
2. Consider a compound task in this environment. For example, part 1: find an area of interest, part 2: navigate to this area, part 3: examine this area in detail. How many operations would be involved in this typical compound task given your interface? Would the sequence of operations be a distraction to the scientist?
3. Do you place several functions on the same device? For example, the left wand button moves your model in navigation mode, but also places a streamline in exploration mode? If so, how difficult will it be to remember the operational details of the interface? How can you avoid confusion in this situation?
4. Are common tasks easily or intuitively performed with your interface?
5. Could your interaction design be generalized to work for any VR application? How specific is it to the bat flight program?
6. Will your techniques work when the bat is made very small? How about when it is as large as the Cave, or even bigger?
7. If you developed an intuitive design for novices, is it still unrestrictive and detailed for an expert user? If your design is more appropriate for an experienced user, how long will it take to become experienced?
8. Could a professor operate your interface? Would a professor want to operate your interface? How about a scientist?
9. Would someone get tired operating your interface?
10. Can a scientist focus on a scientific task, or does your interface require a significant amount of thought unrelated to the problem at hand in order to operate it?
11. What particular scientific task does your storyboard represent a user performing?
12. For each scene in your storyboard, how does the interaction of the user help to accomplish this task? For example, what scientifically relevant structure does toggling visibility of some region help to reveal?
13. As you worked on your storyboard, were there any aspects of your interface design that seemed as though they work smoothly? Were you conscious of any unwieldy or awkward elements as you considered the user navigating and performing tasks?
14. As you look objectively at your completed storyboard, can you read the narrative clearly? Are there actions that might be misinterpreted by a viewer? (if so, you should consider adjusting the drawing, or adding more explanatory text in the area below the images)