A vision system has been developed based on the application of a 3D direct motion estimation algorithm to facilitate autonomous or operated-assisted missions of AUVS and ROVs near the ocean floor. The main capabilities of interest are vision-based vehicle positioning, navigation and trajectory following, as well as mosaicking of sea bed images. The system performance in enabling these functions has been demonstrated on a three-thruster surface vehicle that operates in a 6'×12'×6' water tank, with the bottom surface set up to simulate a sea floor environment. Images from a down-look camera installed on the vehicle are digitized and processed on a Windows NT Dual Pentium-200 to estimate the vehicle's motion, which is transmitted via the serial link to the vehicle control system running on a 386 processor. This information is employed to maintain or move to a desired position, follow a specified trajectory, and to construct in (near) real-time a composite image of the scene surface.