Single View Reconstruction

Single View Reconstruction, or Single View Modeling is the process of "reconstructing a 3D image from a single 2D image. The main principles are outlined here. Essentially, the process involves defining a three dimensional space from input either from a user or automated (this project uses user input). This particular project requires input from the user to define two things: a rear "wall" and a vanishing point. Here is the input process:

Defining input

• Step 1: load the image




• Step 2: define the rear wall




• Step 3: pick a vanishing point (the gui allows you to continue clicking until the generated lines are to your liking)




• Step 4: Results. The green, yellow, purple and cyan quadrilaterals define the two walls, ceiling, and floor of the 3D space, and the area between them defines the rear wall.




• Step 5: Request a focal depth from the user. In this case, we define focal depth as the ratio of distance from the view-to-rear-wall : rear wall width. In this case, a guess of 5 may work reasonably well.

Reconstruction process

Next, we proceed in a few steps:

• First, we define transformations from each of the four corners of our five walls to the four corners of five faces of a rectangular prism. Given eight corners of a prism: we essentially assign a wall to each of them.


• Next we transform the walls from quadrilaterals to rectangles. This involves creating a 3x3 projective transformation matrix that will, given any input point in our resultant rectangle, output its corresponding point in the original trapezoidal "wall". Once we have recovered that matrix, we transform our wall into a rectangular texture to map onto our 3D prism.


• Finally, we take the five textures we have created, and, using MATLAB's surface texturing capabilities, we texture the faces of our rectangle with the transformed "walls".

Some results:

A drawn hallway:

A photo of an alley:

A checkered hallway:

A color hallway:

Discoveries:

• First... people really stink at drawing 1-point perspective. If you notice from the results above, it is nearly impossible to get a "correct" vanishing point for any of the drawings. Trust me, I tried a lot. I changed my rear wall a lot, I redid the vanishing point countless times, but people just dont line things up well. This is especially true in hallways, where there is often a critical failure in keeping corner lines consistant over, say, doorways.


• Second, MATLAB's surface texturing feature is not that great. There are frequently aftifacts, and they even appear with large (2-3 times the size of the surface to be textured) lextures, as well as with textures that are smaller than the surface, and even with textures calculated to be the exact size of the texture. Matlab and 3D textures don't play very nicely. However, the data for these models (the texture maps and the 3D vertex information) can all be output easily and read in through a more 3D friendly program. That said, the artifacts are unsightly.


• Finally, Matlab's video capture functionality *should* work, I just couldn't get it to. If the matlab file is run on a computer, it displays a video of the camera progressing through the 3D room. This feature is fun. Not critical, but fun, and it is unfortunate that it was not able to be externalized