# Lecture 26: Facial Animation

by on May 3, 2013

## The Approaches

### Hand Parameterized Models

• know what motions a face requires
• psychologists have figured this out
• FACS

### Blendshapes

• Data driven
• usually mesh interpolation – but sometimes internal “bones” as well
• often with a neutral pose and offsets (deltas)
• sometimes for part of the face (disjoint regions – but need to make sure the pieces fit together)
• good for manual control
• useful for automation
• choose key poses that you know are useful
• visemes
• important, often used expressions
• extremum
• or be opportunistic
• whatever poses that you can get

### Morphable Models (c.f. Blanz and Vetter)

• Very similar to blendshapes – just with automatically determined blendshapes
• Take a pile of data, do PCA to get “eigenfaces”
• similar in math to blendshapes, but different in practice
• blendshapes have no semantic meaning
• covers space of input
• need to use with automatic methods for control
• often used as a way of parameterizing a set of faces
• Blanz and Vetter: 100 (or so) German college students
• project another face as a linear combination of these faces
• Tom Hanks and Audrey Hepburn as a linear combination of University Students
• things you match can be low dimensional (match the images), combinations are of the data – so you get a 3D Tom Hanks from a picture of him

### High-Resolution Facial Capture

• can do 3D reconstruction and capture with high resolution

### Facial Capture

• large numbers of markers
• hard to get right
• hard to re-use
• MOVA solution

### Visual Speech

• Phonemes
• Visemes
• Dynamics
• Co-articulation
• Cohen & Massaro model – dominance functions that decay

## The Papers

### Rigging Survey

• Rigs can be just about anything
• coming up with them is hard and important
• this paper is a remarkable grab bag of random stuff (says little about any particular thing – and much is meandering and not to the point)
• basic lesson: you can do anything – and people do

### Direct Manipulation Blendshapes

• blendshapes are cool since the parameters are interpretable
• high-end blend rigs may have hundreds of parameters
• may not know what parameters are necessary to change

directly manipulating points is nice

• result is a linear combination
• this is a REALLY easy IK problem
• need to choose which solution
• minimize how far the sliders are from the zero position (or rest position, or last position)
• damped minimization
• keep as close as possible to “rest” position (start of dragging movement)
• min ||w-w_0||
• this allows the user to slide sliders as well
• add in some amount of “keep w small” to avoid drift ||min w||
• linear constraints, quadratic objective function
• could do penalty method (they do)
• turn constraint into a tradeoff
• cons: lose exact control, lose abilty to make big changes
• pros: simplicity, stability
• solve with lagrange multipliers
• you might not get the absolute minimum – but any difference from the minimum has to be something that is required to meet the constraints

Some big question:

• is smallest change in the sliders the right objective?
• arguably yes (since the sliders are the UI)
• isn’t the “most natural pose” or anything like that
• other methods adapt the objective function to try to keep in the space of “natural” results
• unlikely – since model has lots of degrees of freedom
• nearly over-determines (pull apart two points that have to be next to each other)
• small movement in a point requires big movement of something far away
• damping helps a lot
• soft constraints help a lot (tradeoff – big changes in slider vs. meeting constraints)

### Example-Based Facial Rigging

• input
• set of sparse examples
• what blend values these approximately equate to
• an example of a good model
• output
• a blendable model of the example character
• expressions correspond to the original, but the examples are reproduced
• challenges
• make blendshapes match semantics (same deformation gradients)
• make blendshapes match examples
• Do everything in the deformation gradient space
• figure out how to deform the rest position mesh to the blendshapes
• blendshapes need to blend to make examples
• too few constraints
• blendshapes should have similar gradients as the target (where they are known)
• Bilinear alternation (find blend shapes given guess of weights, then updates weights)

Previous post:

Next post: