It's Tricky, Grafting
Brando's Sneer to Bogart's Shrug
By ERIC A. TAUB
LOS ANGELES
IT pales in importance when compared with the cloning of Dolly
the sheep or the mapping of the human genome, but researchers from the
University of Southern California are trying to deconstruct the basis of what
makes humans look human. And this time Hollywood directors, not just
scientists, care about the results.
Once filmmakers can understand what makes people look real and unique to one another, they will be able to recreate reality with ease, inexpensively populating movies with virtual characters - rampaging Mongol hordes, clones of movie stars performing physically impossible feats - whose appearance and actions are as lifelike as a next-door neighbor.
In fact, recreating a celluloid duplicate of Humphrey
Bogart or Marilyn Monroe will soon be possible, although there is little reason
to fear that some cinematic Frankenstein will produce a clone of Elvis that can
convince the world he never died.
"When we finally understand what makes Humphrey Bogart
look like the character that he is, we won't want to see new Bogart
movies," said Dr. Ulrich Neumann, director of the Integrated Media Systems
Center at U.S.C. "Rather, we will want to import his unique
characteristics into other virtual humans." Filmmakers would then be able
to, say, create an artificial actor with Bogart's tough-guy persona and have it
do whatever they need.
Dr. Neumann heads a staff of scientists and graduate
students seeking to identify the essence of what makes people human. By
breaking down movement and speech into a comprehensible and interlocking set of
patterns, he said, programmers will eventually create software that can easily
reproduce the extraordinary complexity of skin, muscle, eye and hair movements
that convey emotion.
The work is still in its infancy, and visual tricks are
often used to cover up technological shortcomings. For example, the virtual
people that strolled across the deck in "Titanic" were deliberately
kept small to mask their lack of sophisticated movement. More recently, the
Agent Smith clones doing battle with the Neo character in "The Matrix Reloaded"
looked real because they wore sunglasses.
Current motion-capture techniques are not sophisticated
enough to create virtual humans that seem real because humans communicate using
a variety of expressions. "A human look is much more than the movement of
hundreds of points of skin on the face," Dr. Neumann said.
For example, real skin does not move smoothly; it bulges
and creases in many combinations. As people converse, they move their hands in
ways specific to their culture and personality, simultaneously wrinkling facial
muscles in unique patterns. When people speak, their mouths move not just to
form words but also in relation to what was and will be expressed.
Even a tiny error in the representation of a virtual
character can cause viewers to sense that they are looking at a contrived
figure. "There is such intricacy and detail and proper timing involved in
the science of human expressiveness that when something is not right we know
it, but we can't explain it," Dr. Neumann said.
Researchers at the Integrated Media Systems Center have
devised a number of studies to classify and quantify characteristic facial
combinations. Once the movement that makes up a person's emotional state is
defined, filmmakers will be able to apply an entire set of actions with one keystroke
rather than arching a character's eyebrow or flaring a nostril individually.
The idea is that if the essential facial and body movements
that make up Arnold Schwarzenegger can be distilled, a virtual actor can be
given the same attributes.
The center has created several projects to help identify
those physical traits. In one, researchers use photos to measure the distance
between points on a face. By overlaying the distances found on one face onto
another, the computer can create a caricature of the second person that
incorporates the features of the first. For example, Michael Jackson's face
could be overlaid with the essence of the filmmaker Michael Moore's, creating a
Mooreish-looking Jackson.
Although testing procedures are still being defined,
researchers plan to present such photos to test subjects and ask them whom the
person pictured resembles. if enough subjects report that the photograph indeed
looks like Mr. Moore, the researchers know that they have captured the
attributes that help define a unique look.
But a face is defined by movement as well as proportion. To
understand how emotions are portrayed, real people were filmed expressing
feelings like happiness and surprise. Their faces were divided into nine
regions, and researchers plotted the movement of three muscles within each in
terms of the distance and length of time each muscle moved.
With that data, researchers were able to create a formula
that described, for each expression, the movement of a combination of muscles
across time. A person's emotional state could then be duplicated by
manipulating the muscles of a virtual actor according to the formula. Depending
on which formula was followed, the actor could have the characteristics of Bill
Clinton, George Bush or anyone else whose muscle movements for a range of
expressions had been plotted.
The perfected facial movements would next be combined with
eye and mouth movements. The virtual eyes would need to swivel and the eyelids
blink unpredictably, which can be accomplished with a technique called texture
synthesis. With that, a computer can create an array of eye movements that show
no discernible pattern.
Finally, the computer-generated character's entire mouth
and face would need to move not just in sync with what is being said, but also
in a way that reflects the words just spoken and about to be spoken. Because
speech typically consists of one word following another, a virtual or real
speaker must move his or her tongue from the proper place for one word to the
proper place for the next.
"I don't have any illusions about figuring this all
out in one year," Dr. Neumann said. But he said he was convinced that once
researchers completed the mapping of the human face and emotions, commodity
software would be developed that made the creation of virtual humans a simple
task.
Eventually, computer animators will be able to select
"Edward G. Robinson anger" from a program's drop-down menu, and
create a virtual actor that can sneer in a way reminiscent of Little Caesar.
"You can count on that happening," Dr. Neumann said.