Physical Facial Modeling
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Physically-based
Facial Modeling
COMP 259
Spring 2006
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Overview
Motivation
Facial Anatomy
Historical view
Techniques
Traditional animation
Muscle-vector techniques
Mass-spring + muscles
Finite-element + muscles
An aside: speech
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Motivation
Why a talking head?
Enhanced communication for people with disabilities
Training scenario software
Entertainment: Games and Movies
Why physically based?
Unburdens animators
Provides more realistic looking simulations
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Anatomy of the face
There are 268 voluntary muscles that contribute to your expression!
Three main types:
Linear muscles (share a common anchor)
Sheet muscles (run parallel, activated together)
Sphincter muscles (contract to a center point)
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Muscles
Bundles of thousands of individual fibers
Thankfully, can be modeled as these bundles
When activated, all of the fibers contract
Contraction only
Most parts of the body use opposing pairs of muscles, but the face relies on the skin
Bulging
Occurs due to volume preservation
Thicker on contraction, thinner on elongation
Important for realistic faces (e.g. pouting lips)
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Skin
Epidermis
Thin, stiff layer of dead skin
Dermis
Primary mechanical layer
Collagen and Elastin fibers
Subcutaneous or Fatty tissue
Allows skin to slide over muscle bundles
Varies in thickness
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Modeling viscoelastic skin
Collagen fibers – low strain for low extensions
Near maximum expansion, strain rises quickly
When allowed to, elastin fibers return system to rest state quickly
Biphasic model:
Two piecewise linear modes
Threshold extension to pick spring constant
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
The skull
Unlike most of the body, the face only has a single joint
All other expression is due to computer-unfriendly soft tissues
Can be treated as a rigid body
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Facial Action Coding System (FACS)
Proposed by Ekman and Friesan in 1978.
Describes facial movement in terms of the muscles involved
Purposely ignores invisible and non-movement changes (such as blushing)
Defines 46 action units pertaining to expression-related muscles
Additional 20 action units for gross head movement and eye gaze.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Traditional techniques
Key-framing
Extremely fast
Extremely hard to model appropriately
Large storage footprint
Basically never used to edit faces, but works as a final format, especially for games
MPEG-4 approach
Defines 84 feature points with position and zone of influence on a few basis keyframes of a standard 3D mesh
Defines animation independently of the visual rep.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
MPEG-4 Facial Animation
68 facial action parameters (FAPs), defined in terms of face independent FAP units (FAPUs)
Most define a rotation or translation of one or more feature points, with a few selecting entirely new key frames (e.g. an emotion basis)
Same animation can be used on different model, provided the model is properly annotated
Some MPEG-4 feature points
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Muscle vectors
Muscle vector properties
Attachment point (to bone)
Insertion point (to skin)
Influences nearby skin vertices, more strongly along the direction vector and close to the muscle.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Muscle vectors (2)
Advantages
Fast
Compact, easily controlled
Disadvantages
Treats the skin like a 2D surface, no concept of curvature
Artifacts when vertices are within two influences
For more information, see Jason Jerald’s slides from 2004 (on course website)
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Mass-spring models
Model the skin (and sometimes muscle and bones) as a number of point masses connected by springs, like a cloth
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Terzopoulos and Waters
Terzopoulos90 models the entire face as a three-layer mass-spring system
Horizontal layers and interconnects:
Epidermis
Fatty tissue
Underlying bone.
Vertical interconnects:
Top-to-middle springs correspond to the dermis
Middle-to-bottom springs provide the simulation of muscle fibers.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Terzopoulos and Waters (cont)
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Terzopoulos and Waters (cont)
Simplifies implementation: everything is handled in a single system
Fast: interactive rates in 1990 (not on a desktop PC)
Provides some wrinkle effects
Unrealistic model of muscles and bone
Cannot control via muscle activations
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Kähler, et al.
Model the muscles as ellipsoids
Long or curved muscles are broken into piecewise linear segments
Scale the diameter as length changes to implement bulging in a nearly volume-preserving manner.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Kähler, et al.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Kähler, et al.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Kähler, et al. – Editor
Also present an easy-to-use editor to define muscles
Provided a skin model, automatically creates skull
Users sketch sheets of muscles and they are iteratively subdivided into individual muscle chains of ellipsoids
Automatic fitting process to place the ellipsoids underneath the skin.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
‘Preservation’ springs
To prevent interpenetrations, Kähler use preservation springs.
Each skin-muscle and skin-bone attachment point gets a mirrored phantom preservation spring acting on it.
Similar to penalty based approaches
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Finite-element models
Break the system down into a regular discretized representation (e.g. tetrahedrons)
Comparison to mass-spring
More accurate
More stable
Far more expensive
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Finite-element skin
Beautiful results
8 minutes per frame*
Creepy video demo
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
An aside: Speech
Phones and phonemes: Unit of sound versus unit of perception
English is considered to have 44 phonemes: 20 vowels and 24 consonants, less per dialect
Distinguishing factors:
Place of articulation (teeth, lips, etc…)
Manner of articulation (flow rate, sort of)
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
An aside: What is speech?
From top to bottom: Amplitude, spectrogram, timeline, and pitch contour, for the word “Welcome” (W EH L – K AH M)
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Parts of speech
Not all changes are visible
Try saying ‘b’, ‘p’, ‘t’
Concept of Visemes
Speech readers say 18
Disney says 12
Some games use 6
Coarticulation
Or, why we don’t have good speech interfaces yet
Vowels
Consonants
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Paper References
E. Sifakis, I. Neverov, R. Fedkiw, Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data, 2005
D. Terzopoulos, Waters, K., Physically-Based Facial Modeling, Analysis, and Animation, The Journal of Visualization and Computer Animation, 1990
K. Waters, A muscle model for animating three-dimensional facial expressions, SIGGRAPH’87
K. Kahler, J. Haber, H.-P. Seidel, Geometry-based muscle modeling for facial animation, Proceedings Graphics Interface 2001
MPEG-4 standard
[Cohen93] M. M. Cohen, D.W. Massaro. Modeling coarticulation in synthetic visual speech, Computer Animation ’93. Springer-Verlag, 1993.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Michael Noland
Video References
http://graphics.stanford.edu/~fedkiw/