Video-Based Reconstruction of Animatable Human Characters

by Aaron Bartholomew on January 23, 2011

in Assignment 1

  1. Sentence: This paper presents a performance capture approach that uses multi-view video recordings and incorporates physically-based cloth to reconstruct a rigged, fully-animatable virtual double of a real person in loose apparel.
     
  2. Problem: There is a significant difference in visual quality of virtual human characters in off-line and real-time renderings; this is due to the non-integration of technology to best create these characters (need to merge geometric models, textures, movement information, etc…, each using different software).  As a consequence, the expense makes it difficult/impossible for game developers to implement these technologies in their product.  By integrating these technologies, it will be faster and less costly to make use of high-quality human representation in real-time apps.
     
  3. Key Idea: Capture the skeletal motion, 3D geometry, and dynamics of a character’s apparel to produce a fully-animatable (arbitrarily animated) character.
     
  4. Basic Info: The technique uses a multi-part processing pipeline to form the aforementioned result; the process is:
    1. Capture a multi-view video sequence of the reference performance
    2. Construct the skeleton motion and deforming surface
    3. Determine which areas of the surface are cloth
    4. Fit a statistical (human) body model to the reference
    5. Estimate the collision proxies
    6. Find the optimal cloth simulation parameters (based on the type of material analyzed – stiff or loose?)
    7. Apply arbitrary animations to the result
       
  5. Contribution: This approach provides a streamlined way to produce much higher-quality human characters for real-time applications; with the process, it may soon be possible to render movie-quality characters in video games.  Also, this paper seems to suggest the potential for ‘plugging’ an arbitrary human into virtual environments (creating an accurate avatar), which could be promising for virtual reality.
     
  6. Resources:

Website (download video/paper here) – http://www.vision.ee.ethz.ch/~gallju/projects/HumanAnimation/HumanAnimation.html

Previous post:

Next post: