1
|
- Michael Gleicher
- Associate Professor
- Lucas Kovar
- Post-Doctoral Associate
- University of Wisconsin- Madison
- www.cs.wisc.edu/~gleicher
- www.cs.wisc.edu/graphics
|
2
|
- Part 0 – Intro and Overview
- Part 1 – Basics of Motion
- Part 2 – Concatenative Synthesis
- Part 3 – Parametric Synthesis
- Part 4 – Motion Databases
- (the outline will be explained as part of part 0)
|
3
|
|
4
|
- We are interested in body motion
- Skelletal motion
- NOT faces, soft tissue, …
- We are considering motion capture
- Key-framed motion should work
- We aren’t game developers
|
5
|
- Need realism and expressiveness
- Need performance (interactive)
- Need controllability (interactive)
|
6
|
- People are good at watching people!
- Human appearance is very complex
- People do many things
- Subtlety matters
- Hard to describe movement
- “Normal” movements aren’t interesting
|
7
|
- “Gross” Body movement
- NOT:
- Appearance Models
- Facial animation
- Cloth, clothing, secondary movement
- Hands
|
8
|
- Create it by hand
- Expensive, but high quality
- Compute it (simulation or other algorithms)
- Not for complex human motions
- Not directable
- Capture it from a performer
- Animate by example
- Re-use existing motions
- Editing
- Synthesis by Example
|
9
|
- Use sensors to record a real person
- Get high-degree of realism
- Which may not be what you want...
- Actors are directable
- Motion capture provides recording
- Reliable and efficient if its done well…
|
10
|
- Use markers and special cameras
- Tracking + Math
- Processing is important
|
11
|
- Motion Capture can record and replay human motions
- Just a record of what happened
- Not controllable
- Can’t capture everything
- No continuous streams of motion
- Fine for a movie, not for a game
|
12
|
- MoCap (or keyframing) provide clips
- Combine examples into new motions
- Sequence to make longer streams
- Blend/Edit to get needed variations
- Create new motions as needed
- Make useful motions at author time
- Create in response to action at run time
|
13
|
- Human motion is rich, complex, varied, …
- Difficult to characterize mathematically or programatically
- The mathematic/algorithmic building blocks for working with motion are
low level
- How do we assemble these simple building blocks to do complex things?
|
14
|
- Basic building blocks are simple
- No excuse not to do them right!
- Basic building blocks combine to do useful things
- Basic building blocks can be extended to provide more versatility
- Automation saves more than time- it increases what you can actually do
|
15
|
- Part 0 – Intro and Overview
- Part 1 – Basics of Motion
- Part 2 – Concatenative Synthesis
- Part 3 – Parametric Synthesis
- Part 4 – Motion Databases
|
16
|
- Representing characters
- Rotations, hierarchies, ...
- Working with motions
- Signal processing intuitions and tools
- Applied signal processing
- Constraint-based approaches
|
17
|
- Basic blend/transition methods are limited – what can we do with
them?
- Answer: Use them to assemble fragments of motion into longer streams of
motion.
- Motion Graphs
- Search-based techniques
- Snap-Together Motion
|
18
|
- How do we enhance basic blending so its more general/useful?
- Registering motions so they can be blended – even if different
- Blending multiple motions
- Determining the blend amounts
|
19
|
- If we have a lot of motion, how do we find what we need?
- Database search via Match Webs
|
20
|
- Goal of this section is to review the basics of representing human
motion and working with motion
- Main insight:
- There are a small set of mathematical operations we can perform on
motions. We have to build everything else from these.
- (bear with me – Lucas has all the cool demos)
|
21
|
- Human Representation
- Motion Signal Processing
- Frequency Intuitions
- Applied Signal Processing Tools
- Constraints
|
22
|
|
23
|
- Need concise description of pose
- Goal:
- Summarize pose as a vector
- Motion is vector valued function
- Compact, yet flexible
- Make constraints implicit
|
24
|
|
25
|
|
26
|
- It depends!
- Generally, small numbers of degrees of freedom (50-60)
- Easier to animate/specify
- Don’t see the details from far away
- Better to use a few d.o.f.s well
|
27
|
- Small numbers of degrees of freedom for gross motion
- Articulated figures
- Rigid pieces
- Sometimes stretching allowed
- Kinematic joints
- Sometimes called a skeletal model
|
28
|
|
29
|
- A set of points that undergoes a rigid transformation
- Describe configuration by the rigid transformation
|
30
|
- Any rigid transformation can be de-composed into:
- A translation (all points same)
- A rotation (the interesting part)
|
31
|
- Mapping f : Rn->Rn
- Defined by properties:
- Has a zero
- Preserves distances
- Preserves handedness
- Is a linear mapping
|
32
|
- Goal: encode rotations in a vector
- Rn - > “set of rotations”
- Give “names” to members of the set of possible rotations
- Many ways to do this, all flawed
- No perfect method
- Use the best one for the job
|
33
|
- Compact
- (as few variables as possible)
- Complete
- Every rotation can be represented
- 1-to-1
- Every rotation has one value
- Every value has one rotation (stable)
- Singularity free
- “close” rotations are “close” in value
- Can compute with
- Compose, interpolate, apply, …
|
34
|
- Rotation Matrices
- Euler Angles
- Axis Angle formulation
- Unit Quaternions
- Exponential Coordinates
|
35
|
- We know the rotation is a linear function (e.g. Matrix)
- Use the matrix as the parameterization!
- Any rotation is represented by 1 matrix
- Must preserve distance
- Must preserve handedness
- Must preserve angles
- Positive, Orthonormal matrices
- Exactly the set of rotations
|
36
|
- Not compact
- Not all matrices are orthonormal
- Change 1 number, its not orthonormal
- Sensitive to numerical issues
- Can’t tell quickly
- Given a matrix, determine if orthonormal
- Can’t project quickly
- Given a matrix, find the “closest” orthonormal one
|
37
|
- Given two rotation matrices, M1 and M2
- Can you measure how different they are?
- Can you interpolate them?
- Fortunately, they are closed under multiplication
|
38
|
- 3x3 matrices – 9 parameters
- No intuitive meaning to parameters
- Only supports a few operations
- Apply to point
- Multiply (compose) – beware drift
- Use rotation matrices to apply rotations
- Use other methods to parameterize and manipulate them
|
39
|
- Any rotation can be represented by a single rotation about an arbitrary
axis
- Axis /
Angle Representation
- Any rotation can be represented by a sequence of 3 rotations around fixed
axes
- Euler
Angles
|
40
|
- Not compact (4 numbers, not 3)
- Each rotation represented by many groups of 4 numbers
- Can’t compute with
- Hard to compose
- Hard to compare
- Hard to interpolate
- Inefficient
|
41
|
- Pick 3 axes (XYZ, ZXZ, ZXY, …)
- Compact, Stable - Any 3 numbers is a rotation
- Every rotation has many values
- Singularities
- Not metric (close rotations->different numbers)
- Interpolations can be weird
- Can’t compose
- OK when 1 axis at a time
- Stability gives false sense of security in math
|
42
|
- Other parameterizations more recent in Computer Graphics
- Quaternions
- introduced to graphics 1985, popular recently
- Exponential co-ordinates
- introduced to graphics 1995, not yet popular
- Both methods are old
- Graphics just took a while to discover them
|
43
|
- Rotations in 2D aren’t too hard
- Examine them to see what happens in 3D (where it is much harder)
- Basic problems still occur
|
44
|
- Consider 1 point in 2D, center is the origin
- A rotation maps the point somewhere on the circle
|
45
|
- Not exactly…
- There’s the handedness thing
|
46
|
- No good mapping to the real line
- Real line goes on forever
- Circle wraps around
- Same problems as rotation!
- Note: circle (in 2D) is a 1D set
|
47
|
- Name point by x,y on circle
- Could be a complex number
|
48
|
- Good points
- Every point can be named
- Every point has a unique name
- Close points have similar names
- Bad points
- Not all points are on the circle
- Can’t manipuate vectors
- How to add? Takes you off the circle
|
49
|
- Extension of this idea to 3D rotation
- 4 dimensional complex number
- Real part, 3 imaginary axes (vector)
- Represent 3D rotation as a point on the unit 4-sphere
- Need to stay on sphere
|
50
|
- Multiplication is defined
- Interpolation is defined
- Special methods worked out
- Linear (1985), Cubic (1995)
- Relatively compact
- Singularity free
- “Nearly” 1-to-1
|
51
|
- Can’t add
- Can’t take linear combinations
- Can’t average
- Can’t linear filter
- (but SLERP works between 2)
- (but the hack works in practice)
- (and the “correct” methods aren’t that hard)
- Distance metric is trickier
|
52
|
- Its easy to get “back on the circle” via reprojection
- Pretend points are in 2D, then project back
- Example: averaging
|
53
|
- Gets the right answer for averaging
- Not for other linear combinations
- Works well when difference is small
- Small angle approximation
- Fails when opposite
- Useful since we can renormalize if computations have problems
|
54
|
- This hack works really well
- Better than any Euler Angle method
- In motion processing, often deal with smaller differences
- “Correct” methods exist
- SLERP is worthwhile
- Exponential maps worth knowing about
|
55
|
- Euler Angles
- Singularities
- Can’t interpolate
- Can’t compose
- Can’t compare
- Stable
- Quaternions
- No singularities
- Easy to interpolate
- Easy to compose
- Easy to compare
- Stability by re-normalization
|
56
|
- How far around circle?
- (unit radius makes things easier)
- Basically an angle
|
57
|
- Suppose the particle starts “at zero” and has a constant
velocity ω
- Where does it end up at the end of a unit of time?
|
58
|
- Velocity is tangent to circle – therefore it is initially upwards
- If circle is in the complex plane, the velocity is purely imaginary
|
59
|
- Velocity as “up” only works if we start at origin
- so always measure from origin
shift the start around
|
60
|
- It’s linear!
- Linearizes the circle around the origin
- Can operate on it
- Not perfect…
- Many different ways to get to any place
|
61
|
- Logarithmic map / Exponential map
- Good for describing the differences between orientations
- Good basis for performing linear operations on orientations
- Filtering
- Averaging
|
62
|
- Use quaternions to represent orientation
- Use tangent space (log map) to perform linearized computations
- Hack works, just as well in practice
- SLERP if differences are big
- Don’t tell anyone I said that!
|
63
|
- Humans too complex
- Some number of connected, rigid pieces
|
64
|
- Angle vs. positional data
- Global vs. relative
- Hierarchical vs. non-hierarchical
- Skeletal vs. Non-Skeletal
|
65
|
|
66
|
- Enforce key constraints
- Connected segments
- Rigid limbs
- Fewer Dof’s
- Only store angles between segments
- Easy for skinning
- Local coordinate systems defined
|
67
|
- Need 3D rotations
- Coupled parameters
- End effector controls require IK
- Forces rigidity
- Problems with reference
- Different ways of defining things
|
68
|
- There are systems based on points
- Diva (House of Moves) – one example
- Math on points is easy!
- Conversion to skeleton is hard
- Interpolation gives weird results
- Can’t blend dissimilar things
- Not compact, hard to draw, …
|
69
|
- Custom character setup (have right DOFs)
- Well chosen joint sets (placement and type) and controls (IK / FK)
- Good:
- make characters that animator can control
- Bad: no uniformity/standardization
- Important if motion from outside source
- Important if want to reuse motion
- Everybody has a different skeleton
|
70
|
- Obvious ways?
- Topology
- number of bones
- Connectivity of bones
- Joint Types
- Bone lengths
- Anatomical / skin relations
- Is spine in middle of body, or up the back?
|
71
|
- What to measure angles with respect to
- Doesn't matter, as long as we agree
- Poses (design of a skeleton)
- Zero Pose / Base Pose
- Dress or Binding pose
- Frankenstein Pose
- Da Vinci Pose
- Rest Pose (real pose of actor)
- Need to figure out how to get between these
|
72
|
|
73
|
|
74
|
- Motion data is relative to base pose
- Tells us how to interpret data
- Need binding pose to skin character
- Need reference poses for calibration
- Try to unify poses
- Base pose = Frankenstein?
- Base pose = Bind pose?
- Base pose = Rest pose?
- Animator’s T-Pose vs. Anatomical T-Pose?
|
75
|
- Represent human as hierarchical skeleton
- Vector with 1 position, 1 absolute orientation, many relative
orientations
- Vector really isn’t in RN
- Many different ways to do this
- Many things to be careful of
|
76
|
- What do operations really mean?
- Assume we get the basics right
- Can’t add or scalar multiply quaternions
- Just a notation thing
- A Å B “add” (really
compose)
- aA Å (1- a) B
(interpolate)
- aA Å βB Å γC Å
… (“linear” combine)
|
77
|
- With Quaternions “halfway” is halfway – and operation
make sense
- For the orientations!
- Not for the end effectors
- Not for the meaning of the pose!
|
78
|
- “Halfway” in math is not “halfway” in terms of
meaning
|
79
|
|
80
|
- Math doesn’t know about the meaning of poses
|
81
|
- Develop smarter pose operators
- REALLY hard
- Need to be general, directable, fast, …
- Consistent across similar poses
- Others are thinking about it
- Data centric methods
- Use simple operations
- Combine simple operations
- Use lots of data (examples!)
|
82
|
- Motion is a function of time
- Given time, provide a pose
- Represented as samples
- Sparse samples + interpolation
- Dense samples (at frames)
- How to manipulate sets of samples?
|
83
|
- Can’t just change 1 pose
- Introduces a discontinuity
- Need a vocabulary to discuss how things can/can’t change
|
84
|
- Need math to talk about how things change over time
- Not a cure all!
- No real connection between signal processing and high-level meaning
- Just like with poses
- Vocabulary for talking about motion
- And some mathematical tools
|
85
|
- Signals can be broken into frequencies
- Low frequencies = smooth parts
- High frequencies = abrupt changes
- Real signals contain lots of different frequencies
|
86
|
- Noise comes from errors in process
- Sensor errors
- Fitting errors
- Bad movements
- Noise is “data” that we don’t want
|
87
|
- Things in the world don't change that fast (have high freq)
- If there are high freqs, must be noise
- Get rid of high freqs (quick changes)
- Low-Pass Filter (LPF) easy (weighted average, FIR, ...)
|
88
|
- Getting Rid of High Frequencies does not eliminate noise
- Leaves a “soggy” look
|
89
|
- We want to remove the noise, to get back a signal that looks like
|
90
|
- Sometimes identification is easy:
- Clearly wrong (foot through floor)
- Marked wrong (missing data - gaps)
- More often, need to guess
- Might be a subtle twitch…
- Might be person shaking…
- Might be sensor errors…
|
91
|
- Use heuristics and rules of thumb to identify noise
- Use info about which body part as a discriminator
- Extremities are more likely to have sharp movement
- “Speed” of the movement affects how prevalent noise is
- Visual signal/noise ratio decreases as movement gets slower
|
92
|
- PROBLEM: High frequencies can be important!
- Getting rid of them makes motion look soggy
- ANSWER: Do not over-apply LPF
- How much is enough?
- Use a little LPF
|
93
|
- Small amounts of Low-Pass Filtering
- Noise modeling
- Adaptive filters
- Non-linear filters
- Hybrid solutions
|
94
|
- High Frequencies are Important!
- Don’t occur often
- Always significant
- Impact
- Rapid, sudden movement
- Emphasis
- Sensitivity of perception
|
95
|
- High Frequencies are important
- Can’t remove
- Can’t add
- Don’t want to put something in
|
96
|
- Can’t introduce pops (high-freqs)
- Removing them is a nightmare!
- Must change poses together
- Two tools:
- Motion Displacement Maps
- Motion Blending
|
97
|
- Motion Displacement Maps
- Spread changes over time
|
98
|
- Think of the change as a motion (signal)
- It can’t be discontinuous
- In fact, we want it to be smooth
- Make a spline that interpolates goals
|
99
|
- “Add” in another motion
- m(t) = m0(t) Å d(t)
- Pick other motion so that it doesn’t stick out (no high
frequencies)
- Changes are low frequency
|
100
|
- High frequencies are important
- Eye is sensitive to them
- Always signifies important events
- Avoid high frequency changes
- Preserve existing high-frequencies
- Avoid adding new ones
- Band limit the changes
- Not the resulting motions
|
101
|
- Intuition: ease in, ease out
- Signal Processing – Superposition
- Adding signals does not create frequency content
- MD Maps don’t add bad high frequencies
- That doesn’t mean they work all the time!
|
102
|
- “Add” two motions together
- m(t) = a m0(t) Å (1- a) m1(t)
- Note: this is a per-frame operation
- Interpolate corresponding poses!
- No new frequencies generated
|
103
|
- No!
- Doesn’t create HF, but doesn’t mean anything
- Yes – but only if
- Individual pose blends work out ok
- New sequence of poses makes sense
- Or… it happens so fast that no one sees
|
104
|
- Blending only works when corresponding poses are similar
|
105
|
- Interpolate similar motions
- Need to know when is similar (Part 2)
- Need to make motions similar (Part 3)
- Transition between motions
- Time varying blend (a=0
-> 1)
- Over a short period of time
- A bad pose isn’t such a big deal
- Avoids discontinuities
- m(t) = a(t) m0(t) + (1-a(t)) m1(t)
|
106
|
- Very useful!
- Often get small pieces of motion
- Need to connect
- Easy if motions are similar
- Hard if motions are not similar
|
107
|
- Cut transitions
- Gap-Filling
- Motion Displacements + Cut Transitions
- Blend Transitions
|
108
|
- Just put one motion after the next
- Works if the end of one is the same as the begining of the next
- This is unlikely to happen unless you your motions are made special
- Or, if you don’t care about the pop
|
109
|
- Make a new motion that starts at the end of M0 and ends at M1
- Can use interpolation
- Only works if interpolation works
- Only works if timing isn’t important
- (generally, it doesn’t work – it’s a bad idea)
|
110
|
- Change the end of M0 to be the same as the beginning of M1
- Change both to meet halfway
|
111
|
|
112
|
- Blends require overlap
- Blends fade the changes in
|
113
|
|
114
|
- Faster is better (get it over with!)
- Better control
- Less likely to see a bad pose
- Slower is better (don’t add HF)
- Less inducement of rapid changes
- Depends on how similar motions are
- Exact match? Zero length is OK.
|
115
|
- Signal Processing gives tools
- Blending
- Displacement Maps
- Tools can do useful things
- Only work when poses are similar
|
116
|
- Signal Processing considers temporal aspects of parameters
- How about detailed requirements of end-effectors?
- For one frame this is the IK problem
|
117
|
- Blending poses may not blend end-effector positions
- See problems in blended motions
- Feet slide on floor
- Hands slip away from goals
- Even evident in transitions
|
118
|
- Find joint angles to meet goals
- Variety of methods
- Numerical
- General but Slow, Ill-conditioned
- Geometric (closed-form)
- Fast, simple – but only for special cases
|
119
|
- Can’t change each frame independently
- Special IK Solvers for continuity
- Can do a Motion-Displacement Map
- Special variants of constraint solvers to solve special motion problems
- Footskate
- (but it can be any IK goal over time)
|
120
|
- Known constraints
- Clean footskate
- Follow an IK goal
|
121
|
|
122
|
|
123
|
- Special case: human limbs
- Documented in a paper
- Open source implementation
- www.cs.wisc.edu/graphics/Gallery/FootskateSolver
- Really important for blended motion
- Used in everything Lucas will show
|
124
|
|
125
|
- Assume specific form
- There are closed form solutions
- Simple, fast, guaranteed continuity
|
126
|
|
127
|
|
128
|
|
129
|
|
130
|
|
131
|
|
132
|
- Changes to any frame are blended into the neighboring frames as well
- Requires buffering if on-line
|
133
|
- Yes, it works for arms
- No, I haven’t given you details
- Read the paper
- Use the Open Source implementation!
- www.cs.wisc.edu/graphics/Gallery/FootskateSolver
|
134
|
- Review human animation basics
- Signal Processing Building Blocks
- Constraints
- Next stop: Animation by Example!
|