An improved algorithm for computing the three-dimensional structure of a scene from pair of stereo images is given. The spatial relationship between the two images is not known, only the "corresponding" points in the two images are known. This problem of 3D scene reconstruction involves: (1) establishing a one-to-one correspondence between the image plane stereo pairs corresponding to the spatial points-'the correspondence problem', and (2) determining the relative orientation of the two image planes and the depth relation of spatial points with respect to the image planes. This paper assumes the first problem solved [Usikov et al. 1991] and addresses the second problem. There are several approaches to the problem of relative orientation determination [Thompson 1959, Schut 1960, Longuet-Higgins 1981, Horn 1987]. The problem where the two optical axes are parallel and separated by a baseline through the optical centers is well done. The calculation is straight forward using similarity of triangles. For non-parallel stereo systems, closed form solutions may not exist. We present a technique for determining the closed form solution to absolute orientation problem when only the projection points are known. This paper presents a heuristic approach which requires at most six stereoscopic pairs of points for determining the relative orientation and spatial points. It is based on solving a linear system of at most three equations. The new approach is intuitive, non-iterative, and enables a clear understanding of the operations performed and inferences drawn.