A Real-Time System for Full Body Interaction with Virtual Worlds
Real-time video acquisition is becoming a reality with the most
recent camera technology. Three-dimensional models can be
reconstructed from multiple views using visual hull carving
techniques. However the combination of these approaches to obtain
a moving 3D model from simultaneous video captures remains a
technological challenge. In this paper we demonstrate a complete
system architecture allowing the real-time (≥ 30 fps)
acquisition and full-body reconstruction of one or several actors,
which can then be integrated in a virtual environment. A volume of
approximately 2m3 is observed with (at least) four video
cameras and the video fluxes are processed to obtain a volumetric
model of the moving actors. The reconstruction process uses a
mixture of pipelined and parallel processing, using N individual
PCs for N cameras and a central computer for integration,
reconstruction and display. A surface description is obtained
using a marching cubes algorithm. We discuss the overall
architecture choices, with particular emphasis on the real-time
constraint and latency issues, and demonstrate that a software
synchronization of the video fluxes is both sufficient and
efficient. The ability to reconstruct a full-body model of the
actors and any additional props or elements opens the way for
very natural interaction techniques using the entire body and real
elements manipulated by the user, whose avatar is immersed in a
virtual world.
Images and movies
BibTex references
@Article\{HLS04, author = "Hasenfratz, Jean-Marc and Lapierre, Marc and Sillion, Fran\c{c}ois", title = "A Real-Time System for Full Body Interaction with Virtual Worlds", journal = "Eurographics Symposium on Virtual Environments", pages = "147-156", year = "2004", keywords = "virtual, interaction, virtual", url = "http://artis.inrialpes.fr/Publications/2004/HLS04" }