Match Moving

Chun-Wei Liu Yi-Hsin Liu

National Taiwan University

Artifact

Overview

Match moving is widely using in film production. It combines computer generated imageries (CGI) and real video in a seamless way. In this project, we apply free softwares such as ICARUS [1] and Blender [2] to make a simple video by this technique.

Implementation

Data Acquisition. We used Sony HDR-SR1 to shot footages in 720x540 resolution. The frame size should be consistent in the whole process. We simply took a can model [3] in our film.

Tracking. To add CGI in the film, we have to calibrate camera and scene. The first step is tracking the feature points of the input video, which means we identify some specific points with their 2D position in different frames, and further inference their 3D location in the scene.

Calibration. The second step is then solving the 3D location. This step is complicated than tracking but could be split into a pipeline. It includes 3D estimation, bundle adjustment (an optimization step) and geometry fitting. Fortunately, free software like ICARUS [1] can deal with this problem. User would be asked to define an plane and origin to help the calibration. An example of features shows in Figure 1.

ICARUS

Figure 1. ICARUS tracking result. The green spots are the feature points with higher reliability.

Texture mapping. We try to modify the texture of our model to make it an unique one. Thus, we transform the logo ‘Taiwan Beer’to ‘Taiwan University’. Figure 2. shows the texture map, and we will wrap the mesh to fit the it. This step can be referred to [4].

Taiwan Beer Taiwan University

Figure 2. The Chinese charters are ‘Taiwan Beer’ (left) and ‘Taiwan University’ (right).

Computer generated imagery. Now, we have a video sequence and a CGI mesh model. To combine them together, we use free software Blender [2]. And to make the model tractable, we put the beer can near by the kitten which has lots of feature points. We also put two light sources near the beer can as the setting of the real scene. Figure 3 gives us an illustration of the difference between taking the lighting on and off.

Blender Blender

Figure 3. The lighting make the beer can more lively.

Results

NTU Beer

Raw Video

Composite Video

References

  1. “ICARUS - Interactive Calibration And Reconstruction from Image Sequences.” [Link]
  2. “Blender.” [Link]
  3. “Can model in THE OFFICIAL Blender Model Repository.” [Link]
  4. “fhkkta's blender 3D tutorial.” (in Chinese) [Link]