I will have your official grades posted. There was a little delay since some people seem to have vanished and I wanted to give them a chance to un-vanish.

It may take me a little longer to get you the feedback on where the grades come from (like, your project 2 grade). It’s all in a spreadsheet, but I need to write some scripts to turn them into an emailable form.

These are the parts of the final project.

Note that there are separate handin things for Moodle to turn in each part.

Note: as far as I know, everyone has requested (and was granted) the no-cost extension. Note: if you need to turn things in after the 5pm Tuesday May 14th no cost extension deadline you (1) must make arrangements with me via email and (2) may be penalized.

Note: each part must be done by each person. (unless it is explicitly a per-group component: showing the movie in class, providing the final movie, and the movie synopsis).

1. Presentations

On Friday, May 10th, everyone gets their 5 minutes in class to explain what they did. Feel free to bring materials to use in your “talk” (slides, videos). If you want to bring them on a USB stick and trust my laptop, you can. (this will minimize switching headaches).

Each of the movie groups will also get 5 minutes to show their current state (which may not be the final thing)

Grading: There will be an assessment of how well you present (but this will be a minor thing and won’t influence any grade much). It will also help me assess your content.

2. Reading List / Annotated Bibliography

On Friday May 10th (no cost extension deadline: 5pm, Tuesday, May 14th), you must turn in your reading list / annotated bibliography.

The minimum is 3 papers. I strongly recommend that you have more than the minimum.

For each reading that you did, provide the full citation information (including a web link), a short synopsis (at least a paragraph, maybe a few paragraphs), and a brief explanation of how it influenced your project. (it’s OK to say that it didn’t)

There is a moodle assignment for handing these in. Please upload either a PDF. (I much prefer a PDF to a word document – but if you really have to, other forms will be accepted).

Grading: These will be assessed on a qualitative scale, and contribute towards your project grade.

3. Project Writeup

On Friday May 10th (no cost extension deadline: 5pm, Tuesday, May 14th), you must turn in your project writeup.

This is the document where you describe what you did. Be sure to describe what problem you were trying to solve, the approach you chose (and maybe others that you considered), a discussion of you implementation, and a presentation and evaluation of your results (even a personal, qualitative assesment). Your writeup really needs to convey what you did, what you made, what the “thing” does, what you were able to do with it, …

One thing: if there is something that might not be obvious from looking at the results, be sure to explain it.

Use pictures as much as possible to help give me an idea of what systems look like, as well as to show the results.

I strongly prefer a PDF to any other format, but will accommodate other forms if that isn’t possible.

Grading: These will be assessed both for document quality, as well as for being the primary way I decide what you’ve done and how good it is. So they are a big part of how I will determine your project grade.

There is a separate moodle assignment just for handing these in.

4. Project Artifacts

On Friday May 10th (no cost extension deadline: 5pm, Tuesday, May 14th), you must turn in your project artifacts.

Please upload a single ZIP file to the moodle assignment (there is a specific moodle assignment for these). You zip file should include:

1. a README file listing all the files and explaining what they are
2. any examples you created (movie files, pictures, …) – be sure to describe them in the README (or have an additional caption file explaining the pictures beyond the readme)
3. your source code (please include instructions in the README on what is required to build it)
4. any data files you think are relevant (for example data required to make the examples you turned in).

Grading: these will be examined (especially the examples) to help figure out the magnitude and quality of your project. I may look at code to get a sense of how much you implemented, and how much effort went into building something decent, but I will not assess your code or try to build it. (if I feel the need to do this, I will contact you)

The moodle limit for a ZIP file is 200MB. You should not need anywhere near this much.

Self / Peer Assessment

Note: this may be turned in up to 12 hours after the other parts are due. (e.g. by Noon Wednesday, May 15th).

If you worked with a partner, please write up (for each partner):

1. How well did you work together (on the combined parts)?
2. How did you divide up the combined parts (who did what)?
3. How do you assess their individual technical component?
4. How do you assess their contribution to the final group project?

You can send these to me via email.

As far as your self-assessment, please answer the following questions.

• How well did you think it turned out?
• What went right?
• If you had another week, how might it have been better?
• What did you learn from the experience?
• For each member of your group (including yourself): what grade would you give for this project and why? The “why” is particularly important if you don’t give everyone the same grade.
• It’s a cliché to ask “what did you learn from this project.” It’s also a difficult question to answer (but good self-reflection). So you can answer it if you can think of it.
  An easier question (or pair): What advice would you give to someone starting this project? or What would you do the same/differently from what you did if you had to do it again?
• Optionally: what could we have done in class that would have made you better prepared to do this project? Did this project suggest a topic for class we didn’t cover? Is there some skill we should have taught? …

Please upload your self-evaluation to the moodle assignment. Note: I prefer a PDF, but you can actually just enter text into the textbox for this one.

Grading: I will give you an assessment of the thoroughness of your evaluations, but these will not be used to determine what you did for your project (except in unusual circumstances). So, its better to give a thorough and honest reflection, than to delude yourself and tersely say things were great if they weren’t.

The Spacetime “Philosophy”

• Motion synthesis as constrained optimization in space and time
• Specify “What” with constraints
• Specify “How” with objectives
• Obeying physics as a constraint

The “Philosophy” of Witkin and Kass in 1988

• Biological motions are energy minimal
• Animation principles try to capture this
• Our perception of motion is tuned to appreciate these qualities (smoothness, …)
• Physical optimal is a good starting point

My philosophy

• constraints are great for specifying what
• objectives offer the potential for “how” – if we can only figure out what they are
• grace, style, quality, efficiency, …

Mathematical Formulation

• Physics / Motion = Ordinary Differential Equation
• Two point boundary problems (know start and end)
• other unknowns (what forces)
• Variational Optimization (Variational Calculus)
• Minimum principle over a curve
• Looking for a curve as the answer
• trajectory
• control
• Optimal Control
• Do the right thing (as possible)
• local decision making vs. planning

Solution methods?

• Discretization
• Splines (discrete set of control points)
• enforces smoothness
• create more coupling between times (easier basis to solve in)
• can allow the times of the control points (knots) to change

Big, non-linear, constrained optimization

Examples

Particle

• F=ma
• force = jet engine, gravity
• Constraints
• where to begin and end
• initial velocities (otherwise could just coast)
• State = q(t)=x,v,f
• Set of discrete points
• q(0), q(1), …  (could be control points of a spline)
• could approximate v by fine differences
• could approximate a by finite differences
• f = ma may for may not be true
• make it a constraint
• really must hold at ALL times (variational constraint)
• approximate by making it hold at the sample times
• Constraints
• obey physics (f=ma)
• initial and final positions (and maybe places in between)
• get to the end at the right time
• or let time “float”
• Objective function
• use the engine as little as possible (sum=integral of F->work)
• smoothness, …
• big messy set of equations
• state for all time samples
• constraints for each time sample
• objective is sum over all time samples

Luxo Lamp

• They did it in 2D – 3D was just for picture making
• Luxo lamp has angles and 1 position (and different masses for different parts)
• heavy base demo
• Equations of motion
• f=ma is just for particles
• lagrangian dynamics (not lagrange multipliers)
• need to create the “equations of motion” for the model
• could have made simpler pieces and connected them with constraints
• Constraints and objective functions get very complicated, very quickly
• Non-linear optimization

Automatic Differentiation

• functions are modular
• function blocks
• function composition
• evaluation is easy
• derivatives by automatic differentiation (composition)  – compare with symbolic

Other Papers and Variants

Better Implementations of the same thing

• Liu, Gortler, Cohen, (1994) “Hierarchical Spacetime Constraints”
  • B-Spline Basis

Spacetime Controllers

• Controllers vs. motions
• van de Panne and Fiume (1990)
• Ngo & Marks (1993)
• Sims (1994)

Control of more complicated characters (real 3D)

• Popovic (1999, and follow ons)
  • human is too complicated
  • simplify (pogo stick, …)
  • do spacetime on simpler thing
  • motion edit to make changes to dynamics work on real motion
• Fang and Pollard (2003)
  • Taking derivatives of the dynamics equations can be expensive
  • Pick to optimize things that can be computed (and differentiated) efficiently
    • important since requires lots of evals to do optimization
    • can’t do torque minimization (since a torque can affect all bones, leading to costly things) – forces take linear time to compute their effects, and can be minimized

My Stuff

• Kinematic Spacetime (not my term)
• Kinematic constraints (on potentially many frames)
  • IK
  • non-overlap
• “Dynamics constraints” (on the movement paths)
  • smoothness (of changes)
  • minimalness (of changes – over all frames)
• To make fast, do B-Splines (to represent displacement maps)
  • also guarantees smoothness
• optimization problem over whole motion
  • non-linear constraints (kinematics on frames, frames from B-Splines)
• Editing (interactive)
• Retargeting

Beyond My Stuff…

• PFIK+F
  • Lee & Shin

The Approaches

Hand Parameterized Models

• know what motions a face requires
• psychologists have figured this out
• FACS

Blendshapes

• Data driven
• usually mesh interpolation – but sometimes internal “bones” as well
• often with a neutral pose and offsets (deltas)
• sometimes for part of the face (disjoint regions – but need to make sure the pieces fit together)
• good for manual control
• useful for automation
• choose key poses that you know are useful
  • visemes
  • important, often used expressions
  • extremum
• or be opportunistic
  • whatever poses that you can get

Morphable Models (c.f. Blanz and Vetter)

• Very similar to blendshapes – just with automatically determined blendshapes
• Take a pile of data, do PCA to get “eigenfaces”
• similar in math to blendshapes, but different in practice
  • blendshapes have no semantic meaning
  • covers space of input
  • need to use with automatic methods for control
• often used as a way of parameterizing a set of faces
  • Blanz and Vetter: 100 (or so) German college students
  • project another face as a linear combination of these faces
    • Tom Hanks and Audrey Hepburn as a linear combination of University Students
  • things you match can be low dimensional (match the images), combinations are of the data – so you get a 3D Tom Hanks from a picture of him

High-Resolution Facial Capture

• can do 3D reconstruction and capture with high resolution

Facial Capture

• large numbers of markers
• hard to get right
• hard to re-use
• MOVA solution

Visual Speech

• Phonemes
• Visemes
• Dynamics
• Co-articulation
• Smoothing, look-ahead
• Cohen & Massaro model – dominance functions that decay

The Papers

Rigging Survey

• Rigs can be just about anything
• coming up with them is hard and important
• this paper is a remarkable grab bag of random stuff (says little about any particular thing – and much is meandering and not to the point)
• basic lesson: you can do anything – and people do

Direct Manipulation Blendshapes

• blendshapes are cool since the parameters are interpretable
• high-end blend rigs may have hundreds of parameters
• may not know what parameters are necessary to change

directly manipulating points is nice

• result is a linear combination
  • this is a REALLY easy IK problem
• need to choose which solution
• minimize how far the sliders are from the zero position (or rest position, or last position)
• damped minimization
  • keep as close as possible to “rest” position (start of dragging movement)
    • min ||w-w_0||
    • this allows the user to slide sliders as well
  • add in some amount of “keep w small” to avoid drift ||min w||
• linear constraints, quadratic objective function
  • could do penalty method (they do)
    • turn constraint into a tradeoff
    • cons: lose exact control, lose abilty to make big changes
    • pros: simplicity, stability
  • solve with lagrange multipliers
  • you might not get the absolute minimum – but any difference from the minimum has to be something that is required to meet the constraints

Some big question:

• is smallest change in the sliders the right objective?
  • arguably yes (since the sliders are the UI)
  • isn’t the “most natural pose” or anything like that
  • other methods adapt the objective function to try to keep in the space of “natural” results
• what about over-determined cases
  • unlikely – since model has lots of degrees of freedom
  • nearly over-determines (pull apart two points that have to be next to each other)
    • small movement in a point requires big movement of something far away
    • damping helps a lot
    • soft constraints help a lot (tradeoff – big changes in slider vs. meeting constraints)

Example-Based Facial Rigging

• input
  • set of sparse examples
    • what blend values these approximately equate to
  • an example of a good model
• output
  • a blendable model of the example character
  • expressions correspond to the original, but the examples are reproduced
• challenges
  • make blendshapes match semantics (same deformation gradients)
  • make blendshapes match examples
• Do everything in the deformation gradient space
  • figure out how to deform the rest position mesh to the blendshapes
  • blendshapes need to blend to make examples
    • too few constraints
  • blendshapes should have similar gradients as the target (where they are known)
• Bilinear alternation (find blend shapes given guess of weights, then updates weights)

The last of the “project checkpoints” is due on Friday May 3rd, by 5pm.

Each person must send me an email.

Please send me an email that:

1. gives me an update on your technical and artistic (if you’re making a movie) progress. a few sentences is fine, but hopefully, I should get a sense of how well things are going (especially if there were issues in prior weeks)
2. an explanation of what you expect to show for your in-class demo on Friday, May 10th. note: informal is fine for this demo (a few minutes demo, and a brief explanation of what you did). we don’t have time for everyone to give a formal presentation
3. describes what form you expect to turn in your final result (do you want to give a final demo, turn in movie files showing off what you did, …). be explicit about what kinds of things you expect to turn in (e.g. source code to a simulator, a few movie files showing off how the simulator works, the final movie, and the 3ds max file and assets in a big zip file)
4. describes when you want to turn it in. (e.g. will you want the no cost extensions, …)

Note: for your final movies, I recommend that you post them on some video sharing service (Vimeo, YouTube) – you can make them private/password protected if you like (just let me know what the password is). However, I will also want a movie file for archiving, as well.

Officially, the final project is due at the end of classes, 5pm, May 10th.

Partially late assignments will be granted (subject the rules below). However, nothing will be accepted after noon on Thursday May 16th (so that I can get grades in on time). Note that in order to qualify for the extension, you need to follow some of the rules below.

What is required from final projects:

1. Each person must turn in an annotated bibliography.
2. Each person must turn in a writeup.
3. Each person must turn in their technical artifacts
4. Each group must turn in their artifact
5. Each group must give an informal demo (or show at least a draft of their movie) in class on May 10th.
6. Each person must turn in a self-evaluation and partner evaluations.

#5 is due in class on Friday, May 10th. If you cannot make this class, you must make an arrangement to show something in front of class beforehand (Tomislav has done this already).

All other parts are due by 5pm on May 10th. You may have a no-cost extension until Tuesday May 14th, 5pm if you request the extension as part of the May 3rd checkpoint. Note: you must present something in class even if you take the extensions. I was going to make going to the distinguished lecture a criterion for getting the extension, but I am not going to take attendance.

You may turn things in later (up until noon Thursday, May 16th) but I reserve the right to penalize you for being late.

Details for what you must do for each of these parts will be announced soon.

On Thursday, May 9th at 4pm in room 1240 CS there will be a distinguished lecture.

The speaker is Prof. James O’Brien from U.C. Berkeley.

James is a well-known animation researcher, who has focused on physical simulation (although he has done other things like motion capture-based animation, lighting, and computational geometry). Most recently he has been looking into perception issues as well.

I strongly recommend this talk, as it is quite relevant to class. 

I know you have a project due the next day, but you should be able to take a break for an hour, eat a cookie, and hear a (probably interesting) talk.

Distillation of notes Advice on Presentations. Keep it short to give Tomislav Time.

Framing:

• Lectures are not talks (so what you’ve seen hasn’t doesn’t apply)
• Talk vary – and you’ll get lots of varying advice.
• I may not be great at it – which is why I had to think about it

The Key Points:

1. It takes Effort
2. Your mileage will vary
3. Respect the Audience
4. Your materials are not your talk
5. Performance Art / Your voice is an instrument
6. Be Yourself and Have Fun
7. Learn from others (good and bad)
8. Organize and structure for the audience
9. Know your goals
10. Use fewer words
11. Onion model – and reconnect points
12. Talk to your audience not your slides
13. A recording of a presentation is not a presentation
14. Job Talks are different

Takes Effort (#1)

• Need to try to be a good speaker (Schewchuck)
• Need to put effort into preparing and planning a talk
• Looking “natural” often requires a lot of prep
• Over-rehearsal is possible
• Everyone has different issues
• No simple rules

Your Mileage Will Vary

• You – style, ability, experience
• non-native speaker, …
• Goals
• Venue
• Audience
• Material (e.g. dense math is different than …)
• The field
• humanities (reading) vs. …

Respect Your Audience

Talk to them at the right level – and

Be Yourself and Have Fun

Your enthusiasm is critical.

Don’t forget, for Checkpoint 2, every person must hand something in via moodle! (I have no idea where that Moodle link takes you – it takes me to the grading page where I can see how many people have turned things in).

Each group needs to send me email as well. (If you’re a one person group, the email is optional – you can put the info I may need into the moodle thing.

Re-Cap: What did we learn about skinning last time

• The skinning / rigging problem
• Hierarchical skeletal skinning
• Linear-Blend Skinning
• Example-Based fitting
• Pose-Space Deformations

What was wrong with linear blend skinning?

• Hard to come up with good weights (although, example-based approaches)
  • especially for local control
• Artifacts (candy-wrapper, elbow compression)
• Lack of control
  • What happens if your skeleton isn’t rigid?
  • Motion between co-ordinate systems
  • Scales? (vs. translations)
  • Add small amounts of detail

Dealing with Artifacts: Dual Quaternion Skinning

Math in paper is hard to fully understand – but they give you code, easy to use.

Prelude: the problem of artifacts

• linear interpolate matrices – not rotations
• lose rigidity

Why not interpolate rotations (e.g. quaternions)?

• different centers of rotation
• we’re not interpolating rotations! we’re intrpolating coordinate systems

Better interpolation of matrices?

• exponential maps (expensive, problem with centers)
  • allows for non-rigid things
• need a different representation of rigid transforms

Try to blend rotation – but choose a smart choice for the center

• Optimal center of rotation?
  • find the center such is a good center for all the transformations
  • SVD
  • must be done per-vertex
• Joint nearest the points (problem: discontinuities)
• Blend the centers
  • trans-rot is a pure rotation (just with a center somewhere else)
  • this is what exponentials do
  • this is what dual quaternions do

Dual Quaternions

• dual numbers are kindof like complex numbers
  • a + b e (e is the dual unit)
  • e^2 = 0 (as opposed to i^2=-1)
• dual quaternions a Q1 + Q2 e
  • not the same as an octonioan
• have a different algebra associated with them
• they can represent rigid transformations
• there is a way to define interpolation over them
• details in the paper

Cages

A different approach: rather than having an internal skin, have an external control cage.

• specific case of deformation control: based on outline / exterior boundary
• control space inside by manipulating boundary

Note: only the EXTERIOR is the cage – the internal segments are shown just for clarity – there is nothing special about those lines (we do not require interpolation at the internal lines

If we have a good way of deforming the inside of the cage we get:

• easy skinning (for free)
• easy addition of controls
• ways to deal with artifacts (add controls, or rely on “goodness” of deformation)
• ability to gracefully deal with non-rigid things

Several people noted that the “automation” and “connection to controls” issues. These are dealt with in another paper (re-usable templates).

Coordinates

Cartesian

• scale axes (independently)
• (in 2D) 2 scales + 2 translational degrees of freedom

note: once you know the coordinates, really easy to determine where things go based on the d.o.f.

Affine

• allow the vectors to change (arbitratily) – direction and scale
• 4 d.o.f. (2 vectors) + move center
• translation + linear

Barycentric

• express as a linear combination of the corner points
• center of mass if points were weights
• aA + bB + cC (for a triangle)
  • how graphics works
• unique for a non-degenerate triangle
• Note interpolation:
  • edges go to edges (corners to corners)
• only unique for triangles (simplicies – tetrahedra, …)

How to make barycentric coords for polygons?

• convex
• non-convex
  • Mean value coordinates – first really useful one
  • 
  • MVC can be negative (so places on the inside move in the opposite direction)
  • Positive – MVC (hard to compute)

Note: once you have the coordinates, “playback” is easy (since the result is just a linear combination)

Harmonic Coordinates

• laplacian is zero
• Mean value property (value of any point is average of circle around it)
• largest value in a region is bounded by values at the edge
• unique
• simple way to compute (by successive averaging) – but can be slow
  • but once you compute them, using them is fast
  • much faster ways to compute them exist
  • multi-grid, conjugate gradient
• can force to interpolate more points than just the edges to give additional control (Woody’s eyeball is a sphere)

Green’s Coordinates

Add additional controls – not just the points but the edges as well.

• gives additional info to make sure things work well
• mappings “conform” around the edges (derivatives across edges are preserved)
• local rotations (as defined by the changes in the edge)