Reconstruction of 3D rigid body motion in a virtual environment from a 2D image sequence

This research presents a procedure for interactive segmentation and automatic tracking of moving objects in a video sequence. The user outlines the region of interest (ROI) in the initial frame; the procedure builds a refined mask of the dominant object within the ROI. The refined mask is used to model a spline template of the object to be tracked. The tracking algorithm then employs a motion model to track the template through a sequence of frames and gathers the 3D affine motion parameters of the object from each frame. The extracted template is compared with a previously stored library of 3D shapes to determine the closest 3D object. If the extracted template is completely new, it is used to model a new 3D object which is added to the library. To recreate the motion, the motion parameters are applied to the 3D object in a virtual environment. The procedure described here can be applied to industrial problems such as traffic management and material flow congestion analysis.