Projective depth: A geometric invariant for 3D reconstruction from two perspective/orthographic views and for visual recognition

We address the problem of reconstructing 3D space in a projective framework from two views, and the problem of artificially generating novel views of the scene from two given views. We show that with the correspondences coming from four non-coplanar points in the scene and the corresponding epipoles, one can define and reconstruct (using simple linear methods) a projective invariant, referred to as projective depth, that can be used later to reconstruct the projective or affine structure of the scene, or directly to generate novel views of the scene. The derivation has the advantage that the viewing transformation matrix need not be recovered in the course of computations (i.e., we compute structure without motion).