Learning-Based Tracking of Complex Non-Rigid Motion

This paper describes a novel method for tracking complex non-rigid motions bylearning the intrinsic object structure. The approach builds on and extends the studies onnon-linear dimensionality reduction for object representation, object dynamics modeling and particlefilter style tracking. First, the dimensionality reduction and density estimation algorithm isderived for unsupervised learning of object intrinsic representation, and the obtained non-rigidpart of object state reduces even to 2-3 dimensions. Secondly the dynamical model is derived andtrained based on this intrinsic representation. Thirdly the learned intrinsic object structure isintegrated into a particle filter style tracker. It is shown that this intrinsic objectrepresentation has some interesting properties and based on which the newly derived dynamical modelmakes particle filter style tracker more robust and reliable. Extensive experiments are done on thetracking of challenging non-rigid motions such as fish twisting with self-occlusion, largeinter-frame lip motion and facial expressions with global head rotation. Quantitative results aregiven to make comparisons between the newly proposed tracker and the existing tracker. The proposedmethod also has the potential to solve other type of tracking problems.