Efficient three-dimensional scene modeling and mosaicing

Tudor Nicosevici, Nuno Gracias, Shahriar Negahdaripour, Rafael Garcia

Research output: Contribution to journalArticle

55 Citations (Scopus)

Abstract

Scene modeling has a key role in applications ranging from visual mapping to augmented reality. This paper presents an end-to-end solution for creating accurate three-dimensional (3D) textured models using monocular video sequences. The methods are developed within the framework of sequential structure from motion, in which a 3D model of the environment is maintained and updated as new visual information becomes available. The proposed approach contains contributions at different levels. The camera pose is recovered by directly associating the 3D scene model with local image observations, using a dual-registration approach. Compared to the standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures, while allowing 3D reconstructions for any type of scene. Motivated by the need to map large areas, a novel 3D vertex selection mechanism is proposed, which takes into account the geometry of the scene. Vertices are selected not only to have high reconstruction accuracy but also to be representative of the local shape of the scene. This results in a reduction in the complexity of the final 3D model, with minimal loss of precision. As a final step, a composite visual map of the scene (mosaic) is generated. We present a method for blending image textures using 3D geometric information and photometric differences between registered textures. The method allows high-quality mosaicing over 3D surfaces by reducing the effects of the distortions induced by camera viewpoint and illumination changes. The results are presented for four scene modeling scenarios, including a comparison with ground truth under a realistic scenario and a challenging underwater data set. Although developed primarily for underwater mapping applications, the methods are general and applicable to other domains, such as aerial and land-based mapping.

Original languageEnglish
Pages (from-to)759-788
Number of pages30
JournalJournal of Field Robotics
Volume26
Issue number10
DOIs
StatePublished - Oct 1 2009

Fingerprint

Cameras
Image texture
Augmented reality
Textures
Lighting
Association reactions
Antennas
Geometry
Composite materials

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications

Cite this

Efficient three-dimensional scene modeling and mosaicing. / Nicosevici, Tudor; Gracias, Nuno; Negahdaripour, Shahriar; Garcia, Rafael.

In: Journal of Field Robotics, Vol. 26, No. 10, 01.10.2009, p. 759-788.

Research output: Contribution to journalArticle

Nicosevici, Tudor ; Gracias, Nuno ; Negahdaripour, Shahriar ; Garcia, Rafael. / Efficient three-dimensional scene modeling and mosaicing. In: Journal of Field Robotics. 2009 ; Vol. 26, No. 10. pp. 759-788.
@article{f1cc0ddd5b174652aeb111f5500c630e,
title = "Efficient three-dimensional scene modeling and mosaicing",
abstract = "Scene modeling has a key role in applications ranging from visual mapping to augmented reality. This paper presents an end-to-end solution for creating accurate three-dimensional (3D) textured models using monocular video sequences. The methods are developed within the framework of sequential structure from motion, in which a 3D model of the environment is maintained and updated as new visual information becomes available. The proposed approach contains contributions at different levels. The camera pose is recovered by directly associating the 3D scene model with local image observations, using a dual-registration approach. Compared to the standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures, while allowing 3D reconstructions for any type of scene. Motivated by the need to map large areas, a novel 3D vertex selection mechanism is proposed, which takes into account the geometry of the scene. Vertices are selected not only to have high reconstruction accuracy but also to be representative of the local shape of the scene. This results in a reduction in the complexity of the final 3D model, with minimal loss of precision. As a final step, a composite visual map of the scene (mosaic) is generated. We present a method for blending image textures using 3D geometric information and photometric differences between registered textures. The method allows high-quality mosaicing over 3D surfaces by reducing the effects of the distortions induced by camera viewpoint and illumination changes. The results are presented for four scene modeling scenarios, including a comparison with ground truth under a realistic scenario and a challenging underwater data set. Although developed primarily for underwater mapping applications, the methods are general and applicable to other domains, such as aerial and land-based mapping.",
author = "Tudor Nicosevici and Nuno Gracias and Shahriar Negahdaripour and Rafael Garcia",
year = "2009",
month = "10",
day = "1",
doi = "10.1002/rob.20305",
language = "English",
volume = "26",
pages = "759--788",
journal = "Journal of Field Robotics",
issn = "1556-4959",
publisher = "John Wiley and Sons Inc.",
number = "10",

}

TY - JOUR

T1 - Efficient three-dimensional scene modeling and mosaicing

AU - Nicosevici, Tudor

AU - Gracias, Nuno

AU - Negahdaripour, Shahriar

AU - Garcia, Rafael

PY - 2009/10/1

Y1 - 2009/10/1

N2 - Scene modeling has a key role in applications ranging from visual mapping to augmented reality. This paper presents an end-to-end solution for creating accurate three-dimensional (3D) textured models using monocular video sequences. The methods are developed within the framework of sequential structure from motion, in which a 3D model of the environment is maintained and updated as new visual information becomes available. The proposed approach contains contributions at different levels. The camera pose is recovered by directly associating the 3D scene model with local image observations, using a dual-registration approach. Compared to the standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures, while allowing 3D reconstructions for any type of scene. Motivated by the need to map large areas, a novel 3D vertex selection mechanism is proposed, which takes into account the geometry of the scene. Vertices are selected not only to have high reconstruction accuracy but also to be representative of the local shape of the scene. This results in a reduction in the complexity of the final 3D model, with minimal loss of precision. As a final step, a composite visual map of the scene (mosaic) is generated. We present a method for blending image textures using 3D geometric information and photometric differences between registered textures. The method allows high-quality mosaicing over 3D surfaces by reducing the effects of the distortions induced by camera viewpoint and illumination changes. The results are presented for four scene modeling scenarios, including a comparison with ground truth under a realistic scenario and a challenging underwater data set. Although developed primarily for underwater mapping applications, the methods are general and applicable to other domains, such as aerial and land-based mapping.

AB - Scene modeling has a key role in applications ranging from visual mapping to augmented reality. This paper presents an end-to-end solution for creating accurate three-dimensional (3D) textured models using monocular video sequences. The methods are developed within the framework of sequential structure from motion, in which a 3D model of the environment is maintained and updated as new visual information becomes available. The proposed approach contains contributions at different levels. The camera pose is recovered by directly associating the 3D scene model with local image observations, using a dual-registration approach. Compared to the standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures, while allowing 3D reconstructions for any type of scene. Motivated by the need to map large areas, a novel 3D vertex selection mechanism is proposed, which takes into account the geometry of the scene. Vertices are selected not only to have high reconstruction accuracy but also to be representative of the local shape of the scene. This results in a reduction in the complexity of the final 3D model, with minimal loss of precision. As a final step, a composite visual map of the scene (mosaic) is generated. We present a method for blending image textures using 3D geometric information and photometric differences between registered textures. The method allows high-quality mosaicing over 3D surfaces by reducing the effects of the distortions induced by camera viewpoint and illumination changes. The results are presented for four scene modeling scenarios, including a comparison with ground truth under a realistic scenario and a challenging underwater data set. Although developed primarily for underwater mapping applications, the methods are general and applicable to other domains, such as aerial and land-based mapping.

UR - http://www.scopus.com/inward/record.url?scp=70450161981&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70450161981&partnerID=8YFLogxK

U2 - 10.1002/rob.20305

DO - 10.1002/rob.20305

M3 - Article

AN - SCOPUS:70450161981

VL - 26

SP - 759

EP - 788

JO - Journal of Field Robotics

JF - Journal of Field Robotics

SN - 1556-4959

IS - 10

ER -