## The Approaches

### Hand Parameterized Models

- know what motions a face requires
- psychologists have figured this out
- FACS

### Blendshapes

- Data driven
- usually mesh interpolation – but sometimes internal “bones” as well
- often with a neutral pose and offsets (deltas)
- sometimes for part of the face (disjoint regions – but need to make sure the pieces fit together)
- good for manual control
- useful for automation
- choose key poses that you know are useful
- visemes
- important, often used expressions
- extremum

- or be opportunistic
- whatever poses that you can get

### Morphable Models (c.f. Blanz and Vetter)

- Very similar to blendshapes – just with automatically determined blendshapes
- Take a pile of data, do PCA to get “eigenfaces”
- similar in math to blendshapes, but different in practice
- blendshapes have no semantic meaning
- covers space of input
- need to use with automatic methods for control

- often used as a way of parameterizing a set of faces
- Blanz and Vetter: 100 (or so) German college students
- project another face as a linear combination of these faces
- Tom Hanks and Audrey Hepburn as a linear combination of University Students

- things you match can be low dimensional (match the images), combinations are of the data – so you get a 3D Tom Hanks from a picture of him

### High-Resolution Facial Capture

- can do 3D reconstruction and capture with high resolution

### Facial Capture

- large numbers of markers
- hard to get right
- hard to re-use
- MOVA solution

### Visual Speech

- Phonemes
- Visemes
- Dynamics
- Co-articulation
- Smoothing, look-ahead
- Cohen & Massaro model – dominance functions that decay

## The Papers

### Rigging Survey

- Rigs can be just about anything
- coming up with them is hard and important
- this paper is a remarkable grab bag of random stuff (says little about any particular thing – and much is meandering and not to the point)
- basic lesson: you can do anything – and people do

### Direct Manipulation Blendshapes

- blendshapes are cool since the parameters are interpretable
- high-end blend rigs may have hundreds of parameters
- may not know what parameters are necessary to change

directly manipulating points is nice

- result is a linear combination
- this is a REALLY easy IK problem

- need to choose which solution
- minimize how far the sliders are from the zero position (or rest position, or last position)
- damped minimization
- keep as close as possible to “rest” position (start of dragging movement)
- min ||w-w_0||
- this allows the user to slide sliders as well

- add in some amount of “keep w small” to avoid drift ||min w||

- keep as close as possible to “rest” position (start of dragging movement)
- linear constraints, quadratic objective function
- could do penalty method (they do)
- turn constraint into a tradeoff
- cons: lose exact control, lose abilty to make big changes
- pros: simplicity, stability

- solve with lagrange multipliers
- you might not get the absolute minimum – but any difference from the minimum has to be something that is required to meet the constraints

- could do penalty method (they do)

Some big question:

- is smallest change in the sliders the right objective?
- arguably yes (since the sliders are the UI)
- isn’t the “most natural pose” or anything like that
- other methods adapt the objective function to try to keep in the space of “natural” results

- what about over-determined cases
- unlikely – since model has lots of degrees of freedom
- nearly over-determines (pull apart two points that have to be next to each other)
- small movement in a point requires big movement of something far away
- damping helps a lot
- soft constraints help a lot (tradeoff – big changes in slider vs. meeting constraints)

### Example-Based Facial Rigging

- input
- set of sparse examples
- what blend values these approximately equate to

- an example of a good model

- set of sparse examples
- output
- a blendable model of the example character
- expressions correspond to the original, but the examples are reproduced

- challenges
- make blendshapes match semantics (same deformation gradients)
- make blendshapes match examples

- Do everything in the deformation gradient space
- figure out how to deform the rest position mesh to the blendshapes
- blendshapes need to blend to make examples
- too few constraints

- blendshapes should have similar gradients as the target (where they are known)

- Bilinear alternation (find blend shapes given guess of weights, then updates weights)