Lecture 26: Facial Animation

by Mike Gleicher on May 3, 2013

The Approaches

Hand Parameterized Models

  • know what motions a face requires
    • psychologists have figured this out
    • FACS


  • Data driven
  • usually mesh interpolation – but sometimes internal “bones” as well
  • often with a neutral pose and offsets (deltas)
  • sometimes for part of the face (disjoint regions – but need to make sure the pieces fit together)
  • good for manual control
  • useful for automation
  • choose key poses that you know are useful
    • visemes
    • important, often used expressions
    • extremum
  • or be opportunistic
    • whatever poses that you can get

Morphable Models (c.f. Blanz and Vetter)

  • Very similar to blendshapes – just with automatically determined blendshapes
  • Take a pile of data, do PCA to get “eigenfaces”
  • similar in math to blendshapes, but different in practice
    • blendshapes have no semantic meaning
    • covers space of input
    • need to use with automatic methods for control
  • often used as a way of parameterizing a set of faces
    • Blanz and Vetter: 100 (or so) German college students
    • project another face as a linear combination of these faces
      • Tom Hanks and Audrey Hepburn as a linear combination of University Students
    • things you match can be low dimensional (match the images), combinations are of the data – so you get a 3D Tom Hanks from a picture of him

High-Resolution Facial Capture

  • can do 3D reconstruction and capture with high resolution

Facial Capture

  • large numbers of markers
  • hard to get right
  • hard to re-use
  • MOVA solution

Visual Speech

  • Phonemes
  • Visemes
  • Dynamics
  • Co-articulation
    • Smoothing, look-ahead
    • Cohen & Massaro model – dominance functions that decay

The Papers

Rigging Survey

  • Rigs can be just about anything
  • coming up with them is hard and important
  • this paper is a remarkable grab bag of random stuff (says little about any particular thing – and much is meandering and not to the point)
  • basic lesson: you can do anything – and people do

Direct Manipulation Blendshapes

  • blendshapes are cool since the parameters are interpretable
  • high-end blend rigs may have hundreds of parameters
  • may not know what parameters are necessary to change

directly manipulating points is nice

  • result is a linear combination
    • this is a REALLY easy IK problem
  • need to choose which solution
  • minimize how far the sliders are from the zero position (or rest position, or last position)
  • damped minimization
    • keep as close as possible to “rest” position (start of dragging movement)
      • min ||w-w_0||
      • this allows the user to slide sliders as well
    • add in some amount of “keep w small” to avoid drift ||min w||
  • linear constraints, quadratic objective function
    • could do penalty method (they do)
      • turn constraint into a tradeoff
      • cons: lose exact control, lose abilty to make big changes
      • pros: simplicity, stability
    • solve with lagrange multipliers
    • you might not get the absolute minimum – but any difference from the minimum has to be something that is required to meet the constraints


Some big question:

  • is smallest change in the sliders the right objective?
    • arguably yes (since the sliders are the UI)
    • isn’t the “most natural pose” or anything like that
    • other methods adapt the objective function to try to keep in the space of “natural” results
  • what about over-determined cases
    • unlikely – since model has lots of degrees of freedom
    • nearly over-determines (pull apart two points that have to be next to each other)
      • small movement in a point requires big movement of something far away
      • damping helps a lot
      • soft constraints help a lot (tradeoff – big changes in slider vs. meeting constraints)

Example-Based Facial Rigging

  • input
    • set of sparse examples
      • what blend values these approximately equate to
    • an example of a good model
  • output
    • a blendable model of the example character
    • expressions correspond to the original, but the examples are reproduced
  • challenges
    • make blendshapes match semantics (same deformation gradients)
    • make blendshapes match examples
  • Do everything in the deformation gradient space
    • figure out how to deform the rest position mesh to the blendshapes
    • blendshapes need to blend to make examples
      • too few constraints
    • blendshapes should have similar gradients as the target (where they are known)
  • Bilinear alternation (find blend shapes given guess of weights, then updates weights)
Print Friendly, PDF & Email

Previous post:

Next post: