Abstract
In the last decades, the generation of the 3D avatar has proven to be a significant challenge and has limit commercial usage due to the high costs associated. Improvements in image-based 3D reconstruction and camera hardware allow us to reduce the cost of 3D scanners drastically. This paper presents a fast and complete pipeline for a full body 3D scanning system
Keywords
3D Image Based Scanning, Digitization process, 3D scanner, Body modelling, Avatar creation
Nowadays, various fields such as medical and entertainment industries, education or cultural preservation are using virtual humans. Much work has been done in the computer graphics domain to push the limits of realism of virtual humans further and also to automatize the pipeline for the avatar creation. Human body modelling is a difficult task that requires manual work of designers and animators to create a 3D geometry that represent a person. In this paper, we propose a fast pipeline for human body modelling that uses recent image-based 3D reconstruction approaches.
Figure 1.2: Overview of the pipeline
3D avatar creation – State of the art
The time-consuming manual process of avatar creation has been replaced over time by several techniques. Different methodologies have been proposed and can be classified into three mains categories: creative (Ratner, 2012), reconstructive (Allen et al., 2003) and interpolated methods (Bastioni et al., 2008). We propose a reconstruction based technique that uses an image-based 3D scanner to capture the user in a fast and accurate manner. The setup is movable and can be installed easily in different places. The position of the cameras can also be adjusted to change the acquisition volume. Finally, the post-processing time and the cost of the installation have been significantly decreased.
The new system is based on photogrammetry technologies. It is composed of a large number of compact cameras that are synchronized and controlled by a computer (Figure 2.1). Within less than a second, pictures of the subject are taken by the camera cluster from different angles. This very short delay during the capture minimizes user movements, which drastically reduces the noise in the generated model. The images can then be used for 3D reconstruction. Finally, a virtual skeleton is inserted into the model to be able to animate it.
System description
Our proposed system is composed of a cluster of 80 compact cameras. They have been placed onto a hexagonal support structure. Our acquisition volume covers an adult human, and the number of cameras and their positions have been chosen accordingly. A made-to-measure green fabric has been placed over the support structure to control the light conditions and to facilitate the post-processing of the acquired data.
Figure 2.1: Overview of the image-based 3D scanner
All cameras are connected to a single computer. A dedicated library has been used to control, to synchronize and to take pictures with the camera remotely (CHDK). Custom scripts have been written to remotely control and synchronize the individual cameras, to adjust the zoom, to take shots and to copy back recorded pictures to the controller computer. After a short synchronization step, we can remotely take a synchronized shots. All 80 pictures are taken nearly simultaneously within less than 200 milliseconds. Once the pictures are taken, they are automatically copied to the hard drive of the controller computer for reconstruction.
Image Processing
In order to facilitate the 3D reconstruction of the avatar and to reduce computational cost, we have defined an automatic pipeline to pre-process the taken images. Removing the background increases the quality of the reconstructed model as well as the quality of the texture. Furthermore, using a picture mask reduces the computation time as the quantity of data to be processed on each picture are lower. An automation of this task is crucial for high user number since a manual creation takes a long time per user and image and does not scale.
The background detection can be used to generate masks that facilitate the 3D reconstruction. As a result we obtain a black and white picture, on which the black area represent the area that can be ignored during the reconstruction whereas the white part represent important areas for the reconstruction (Figure 2.2). The mask includes the user, the cameras, and the LED ribbons.
Figure 2.2: Automatic image processing
3D reconstruction
We are able to generate the 3D avatar mesh by using an image based 3D reconstruction software (Agisoft PhotoScan). As input, it requires a set of images. An optional mask can be used for accelerated 3D reconstruction. The process can be divided into four steps: camera alignment, point cloud creation, mesh reconstruction and texturing (Figure 2.3).
The camera alignment consists of two steps. First, features are detected in all images. In a second step, the software tries to match the features pair-wise in the set of images. Therefore, a sufficient overlap of the images is needed. This can be achieved by carefully controlling position and zoom level of the cameras. Several tests have been conducted to develop our current setup.
Figure 2.3: 3D reconstruction pipeline using image-based method
We obtain a fully reconstructed and textured 3D mesh (Figure 2.4). However, small corrections are needed to remove mesh artefacts in the obtained 3D model. First, we apply Laplacian smoothing to reduce the noise and to smooth the mesh. Then, to reduce the number of polygons and to get a regular grid on the 3D mesh, we apply a Quadric Edge Collapse Decimation algorithm.
a. Textured model
b. Cleaned model
Conclusions
In this paper we have presented a 3D Image Based Scanning technique that allows to capture the basic shape and surface of a physical object of almost any size including its colour information. We have applied this technique for generating a virtual clone of a person and we contribute a solution to improve 3D reconstruction by applying colour correction and automatic creation of masks. By using our setup, costs and manual processing are drastically reduced so it can facilitate the access to such device for the public. Moreover, as the system can moved easily and set up in different places, fewer system allow to scan a large population. Finally, the avatar creation does not need design or modelling skills so that a non-expert could operate it.
References
1. Magnenat-Thalmann, N. (2010). Virtual Prototyping and Collaboration in the Clothing Industry. In Modeling and Simulating Bodies and Garments (pp. 161186). Springer London. 2. Allen, B., Curless, B., & Popović, Z. (2003, July). The space of human body shapes: reconstruction and parameterization from range scans. In ACM Transactions on Graphics (TOG) (Vol. 22, No. 3, pp. 587-594). ACM.
3. Bastioni, M., Re, S., & Misra, S. (2008). Ideas and methods for modeling 3D human figures. ACM Bangalore.
4. CHDK: http://chdk.wikia.com/wiki/CHDK
5. Agisoft PhotoSan: http://www.agisoft.com/
6. Baran, I., & Popović, J. (2007, August). Automatic rigging and animation of 3d characters. In ACM Transactions on Graphics (TOG) (Vol. 26, No. 3, p. 72). ACM.
7. Mixamo: https://www.mixamo.com/
8. 3ds Max : http://www.autodesk.com/products/3ds-max/overview
9. Kasap, M., Magnenat-Thalmann, N. (2011). Skeleton-aware size variations in digital mannequins. The Visual Computer, 27(4), 263-274.
Author
MIRALab, University of Geneva, Switzerland - Thematic Area 6