How to create a Digital movie star [ PAGE -02 ]
A different kind of "Making of" (by - Jean Lafleur)
Creating the facial expressions of a movie star
I am constantly being told that there are many ways to create facial expressions - bones, clusters, lattices, etc. But I love blendshapes. They are fantastic and creative tools. Yes they are linear, but if we combine two of them, they don't have to be. I must also say that I believe that we'll never be able to create realistic expressive characters without relying on the performance of real actors. They are an essential element in the mix of talents that we must put together to succeed.
Facial expressions are, of course, the most important aspects of a character. They work with the voice performance and mocap performance to define our character's psychology and personality.
Facial expressions are the result of the skin and facial fat being pulled by muscles over the underlying skull. This transforms the face and, whether we want it or not, create wrinkles... Yes, even the youngest, most beautiful woman will have expression wrinkles. I know that the cosmetic industry spends billions trying to convince us that it is not true, but there is very limited expression without wrinkles. Children have expression wrinkles. Even babies have expression wrinkles. Look at a baby crying. Unless we want a botox-paralysed character, we need to embrace wrinkles and use them wisely and often.
Naturally the aim is to give a fully functioning tool to the animator. To create expression blendshapes that work, we must be totally familiar with all the muscles of the face and their effects on expressions. Words like "corrugator", "frontalis", and "zygomatic" start appearing in our vocabulary. Each muscle, alone and in combination with others, shapes the face in a specific expressive way. It is not enough to create believable expressions; it must be done in a way that reveal the character's personality. The face must be ever changing, yet remain recognisable as the same character. This, believe me, is one of the most challenging task in creating a believable and compelling character. The forehead, nose, ears, and neck all need expression blendshapes, but where it gets interesting is with the eyes and the mouth.
Blendshapes and the eyelids
The eyelids are made of very thin, flexible skin. Minute muscles pull at the skin causing it to wrinkle, both on the lids and around the eyes. These movements and these wrinkles contribute immensely to the reality, to the life, and to the expressiveness of our movie star.To exploit the expressiveness of the eyes, control all the small muscle contractions, and manage the telltale micro wrinkles, we need an awful lot of very precise and harmoniously sculpted blendshapes.
Blendshapes and the mouth
Of all the elements of the face, the mouth is the most difficult to manage. It is extremely flexible and is pulled in all directions by muscles that not only act alone but in combination. Often the mouth is simultaneously fulfilling many functions: talking, chewing, spitting, breathing, mugging, and expressing emotions. To achieve that, we must create an adequate number of blendshapes and design and plan them carefully. Observation of real life examples is the only way we can do that successfully. Many think that it will diminish their creativity to rely on real life observation, but it work and there is a long artistic tradition to support it.
To complicate things, the mouth also interacts with the teeth and tongue. More than any other facial movements, those of the mouth are accompanied by tugs, stress, and vibrations throughout the face and the neck area. The blendshapes must be designed to reflect all these secondary tugs and bulgings.
Skin and texture
Again, if we want to go real and not plastic, we need to learn to love natural skin texture: pores, small wrinkles, blemishes, scars and all the elements that make the skin look real. These give character, reality, and a "personal history" to our character - somewhat like the subtext that actors use to flesh out a character's psychology.
Here is an area where the tools are not quite adequate. I confess that I yearned for instant feedback as we were creating Kyra's skin texture. The idea of making a flat texture in photoshop and then applying to an object with volume is incredibly impractical. It leads to all kind of stretching and moving around. Of course the people that have done it for years tell us that it works fine. But the point of view of this director is that we need a tool where we can apply the texture to the volume itself... and see it in real time. I hear that ways of doing that are coming down the road. Great!
This subject includes hair, beards, eyebrows, and fuzz. This is all a question of finding a proper look for your character and tweaking all the parameters to fit that look. It's a lot of work to find just the proper balance. How much movement must the hair have is another delicate decision; it cannot be totally rigid or, on the other hand, look like a slow-motion shampoo commercial. If you want a great example, look at Beowulf. It was great. The skin was pretty good also. Eyelashes are different. They are usually made of geometry and animated to follow the lids. Here again, there are a number of choices. We have to decide how long, how many, how much curve, how thick, how much clumping, etc. We need to strike the proper balance.
Mocap and expressions
Casting for mocap and facial expressions
It is assumed that the actor that does the voice - usually a well-known name - does the MOCAP and serves as a guide for the expressions. Judging from the results, it is not always the case, especially for the mocap. The casting for mocap and facial expressions is the most important choice that a director is called upon to make. Putting anybody in a mocap suit will not work. Body language is a subtle and powerful means of secondary communication and an essential component of a good movie. One important advantage of a virtual character is that it could be played my many actors: one to record the voice, one as a guide for facial expressions, one to do the physical mocap, and as many as you need to do the various stunts, etc. A daring concept to be sure but one that, if done right, might give interesting results. Not all actors are equally qualified to produce a good digital performance. While all the usual requirements for a talented actor still prevail - sincerity, intensity, and creativity - a face that is truly expressive (and I do not mean "grimacing") will enhance the performance considerably. It will help the animator to orient his choices and give more life to the performance.
The technical aspect of mocap
I know I will offend a lot of people, but the way we do mocap with small reflective balls and zillions of cameras is truly not very efficient. It is slowly becoming more interactive and this is a great step in the right direction (this was not available on Kyra; I was flying blind). But I believe we need a better technique. Now some brilliant techno should be able to create a miniature emitter that could be attached to the actor - small enough not to be intrusive - and that would send to three stationary sensors/recorders its position in space and its identity. The result would be a great improvement: no more obstructions, positive identification, fewer long sessions spent cleaning data, less data - making the real time easier. I am by no means an expert in electronics but I cannot believe that it is impossible. We urgently need help here!!!
We did our facial animation from a recording of the actress' performance and it worked well. It is sometimes called "rotoscoping" but is not an accurate term. It could not be a perfect copy since our actress did not look anything like our character. As a matter of fact, the actress was cast after the character had been pretty much modelled. I do not believe that this is a bad technique. Taking what is best about the actor's performance and adapting it to the character's personality is a creative process that will go a long way in creating a compelling character and a real movie star. I also selected bits and pieces from various takes and our animator blended them seamlessly into one flowing shot. That was great.
Because of circumstances and availabilities of equipment, in Kyra, we did our facial animation first. I am now convinced that the performance would have been more subtle and stronger if it had been done at the end of the process. I firmly believe that facial animation should be the last thing that we do. It would be improve the result considerably to animate from the proper angle, with the mocap integrated, with the texture applied, and with the real lighting. Naturally there are technical implications to this and it requires a certain amount of computing power, but I think it would be worth it.
I do not think that we can even guess yet at all the potential of this new medium. Kyra was only a short prototype and I ran out of money before getting to the end of my goal, and yet the process was incredibly exciting. I was dazzled by the degree of control and the possibilities for creativity that realistic digital movies offer.
felt, as I was making Kyra, that strangely enough the greatest challenge - creating a virtual movie star - might turn out to be the factor that will make this medium unique and powerful. It is not a simple or quick process but the very fact that we have to reproduce and control every little detail in creating the characters might just turnout to be what will set the medium apart from conventional cinema. Would it not be ironic if it were the realistic characters, and not the spectacular effects or the weird monsters, that contributed the most to making digital movies special?
There is no doubt in my mind that digital movies will not take off until we have a big commercial hit. Unfortunately, the medium alone will not do it. Many other factors come into play for a commercial success. If we make a movie with the whole package: create movie stars that are real, expressive and compelling, go with a good concept, write a well-structured script, and give it an interesting and appropriate look, we will make a hit. So we must try to convince more producers to get involved in making digital movies.
In ending, if I may be allowed a few personal comments. First I want to thank my technical director, Daniel Cantin, and the rest of the team for their infinite patience with my questioning everything and for sharing so graciously their knowledge with me.
Kyra was strictly a beg, borrow, and steal production with a part-time staff, and I now have run out of financial resources, but I am ready, willing, and able to participate in a venture that would further the cause. With proper funds, staff, and support, I am convinced that I can use what I have learned, working in the digital world for the past three years, to go further and make a definite push toward the creation of realistic digital movie stars and a realistic digitally created movie that will be an artistic and commercial success.
I hope you will find some of the things that I wrote useful and thank you for you attention. There is much more that I would like to share with you, but I believe that I have gone on too long already. However, if you are interested in this new medium and feel like having an exchange of ideas, do not hesitate to contact me.
Author: Jean Lafleur
<<<<...GO Back to HOMEPAGE!