Animation Pipeline

Here's an overview of the process I've used to animate most shots in my film. After much trial and error, I came up with a hybrid approach that combines keyframe animation, motion capture and crowd simulation.

Step 1: 'Hero' characters - keyframed by hand in Maya



Step 2: Add in background crowd (authored in Massive, then imported into Maya using a custom script)

Step 3: Add in middleground crowd (authored in Massive, then imported into Maya using a custom script) *

Step 4: Add in foreground crowd (motion-captured and solved using Vicon IQ and Motion Builder, then imported into Maya)

Final Shot:

* In my pipeline, the middleground crowd could be either Massive or custom mocap. The tighter the shot, the more I used mocap characters that were placed individually in the scene (which takes ages)

Details:

Three layers of crowd (Foreground, Middleground, Background) seemed to be the most effective at creating believable scenes with depth. I also had to spend tons of time randomizing models, shaders and movement so that the crowd was convincing. Check out this test and this other test to see my earlier attempts.

Crowd simulations generated out of Massive seemed to work best for background characters. The problem with building an entire scene with Massive was that it 'felt' like a simulation (no doubt due to my own inexperience with the software). As a result, I resorted to individually placed motion-captured characters for the foreground and middleground crowd layers which really helped break up the crowd motion.

The other advantage of using custom characters in the FG and MG was that I could art direct the crowd and ensure that the crowd parted at just the right time or that the crowd characters helped direct the viewer's eye to the action.

Although all the keyframes were created in different software, Maya became my central application to combine all the animation elements. This approach deviates from the standard industry practice of rendering crowds straight out of Massive using PRMan, but numerous technical challenges (unavailability of RenderMan Pro Server/PRMan at my school, problems importing Maya cameras into Massive, etc) necessitated changes to my workflow. Therefore, I hacked together a MEL script that imported characters from Massive into Maya, and then rendered everything out with RenderMan for Maya. It wasn't pretty, but it worked.

My, What Long Arms You Have!

As I get closer to finishing, I cheat more and more. In this case, I wanted to exaggerate perspective and scale, and had to stretch out Sam's arm to make the shot work.

View from Shot Cam:

View from a different angle:

Making it feel like a 'Movie'

Sometimes, a little bit of editing/camera work and different character models can make a world of difference to the same bit of animation.

1. For instance, here's a fight scene I shot in an introductory motion capture class applied to generic off-the-shelf characters (the old man is actor Kevin J. Ryan and the alien is classmate John Helton):

2. Next, I created some really basic crowd character models and used a segment of the same motion on them, but changed the camera angle:

(This step is important because it verifies that my models have been rigged correctly)

3. That looked ok, so I spent some more time on the camera angles to previz the sequence as it might take place in the film:

4. Hmm, what about with tighter editing, better looking characters and some lighting?

5. And here's the final sequence:

All through this process, the actual animation on the characters was unchanged.

Confrontation Revisited

The confrontation between Maqbool and the goons has been bothering me for some time now. It feels a little sluggish and lacks impact. With input from colleague Joanna Griebel, I reedited the sequence.

Old:

New:

The new cut has an added line of dialogue for Maqbool ("He dropped his cap"). The old edit seemed unclear to a lot of viewers, and I think adding this line back in is an improvement. (The line was in the original script, but I had taken it out for some reason that I can no longer remember. Perhaps I felt it was redundant at the time.). Another reason I like the new line is that it is a reference to Bir's "Sir, you dropped your cap" earlier in the film.

Outtake

Sometimes a crowd character tries to steal the limelight:

First Cut Part 2

I finally managed to upload the file, so here it is!

First Cut

After exactly 7 months of animating, I have my first cut of the film.

Naturally, since this is an important milestone, Blogger won't let me upload the video. But I want to mark the occasion with a post anyway.

While there's still a lot of work to be done on the crowd/animation fixes/lighting/rendering/etc, I'm at a point at which I can get feedback on the film as a whole.

Video Reference

Video reference is pretty useful, especially when animating scenes where there is a lot of complicated body movement. For example:

Reference:
(with classmate Paul Shepherd)


Rough Animation:



Final animation:



Another Example:



A lot of things change during the animation, but the reference is a great way to avoid starting with a blank piece of paper.

(I also feel that reference video footage will soon be replaced by low-cost motion capture systems. Same idea, but the reference will automatically be available as 3D information in the computer, and then you can either layer keyframe animation on top, or use it purely as a traditional reference.)

Yet Another Example:

Reference (starring myself and classmate Joanna Griebel):

Animation:

Motion Capture Performance Final Project

Should've posted this a while ago - my final from the Eric Furie/Robert Zemeckis Motion Capture Performance class which includes more crowd work. (I also shot some fight sequences that were not yet reconstructed in time for this video but can be seen here.)



The foreground and middle ground crowd elements are motion captured (the performers are my classmates John and Nahomi, myself and an actor, Kevin), and the background crowd elements are generated out of Massive. This is my new approach to the crowd because I can still art direct the crowd manually and then just give it additional volume using Massive.

I liked where this is going, but my advisor, Christine Panushka, wants to see A LOT more people and some children so I need to work on that. RenderMan for Maya 2.0 supports instancing, so my plan is to create instances of the FG and MG crowd individuals that I already have, giving me a lot more volume closer to the camera without a substantial performance hit (I hope).

Other than that I got good feedback on the look, the design of the crowd, etc. from the faculty.

Ollie Johnston's Notes on Animation

These are posted on an old Siggraph site, and John Lasseter got them from Glen Keane who got them from Ollie.

1. Don’t illustrate words or mechanical movements. Illustrate ideas or thoughts, with the attitudes and actions.
2. Squash and stretch entire body for attitudes.
3. If possible, make definite changes from one attitude to another in timing and expression.
4. What is the character thinking?
5. It is the thought and circumstances behind the action that will make the action interesting. Example: A man walks up to a mailbox, drops in his letter and walks away OR A man desperately in love with a girl far away carefully mails a letter in which he has poured his heart out.
6. When drawing dialogue, go for phrasing. (Simplify the dialogue into pictures of the dominating vowel and consonant sounds, especially in fast dialogue.
7. Lift the body attitude 4 frames before dialogue modulation (but use identical timing on mouth as on X sheet).
8. Change of expression and major dialogue sounds are a point of interest. Do them, if at all possible, within a pose. If the head moves too much you won’t see the changes.
9. Don’t move anything unless it’s for a purpose.
10. Concentrate on drawing clear, not clean.
11. Don’t be careless.
12. Everything has a function. Don’t draw without knowing why.
13. Let the body attitude echo the facial.
14. Get the best picture in your drawing by thumbnails and exploring all avenues.
15. Analyze a character in a specific pose for the best areas to show stretch and squash. Keep these areas simple.
16. Picture in your head what it is you’re drawing.
17. Think in terms of drawing the whole character, not just the head or eyes, etc. Keep a balanced relation of one part of the drawing to the other.
18. Stage for most effective drawing.
19. Draw a profile of the drawing you’re working on every once in a while. A profile is easier on which to show the proper proportions of the face.
20. Usually the break in the eyebrow relates to the highpoint of the eye.
21. The eye is pulled by the eyebrow muscles.
22. Get a plastic quality in face — cheeks, mouth and eyes.
23. Attain a flow thru the body rhythm in your drawing.
24. Simple animated shapes.
25. The audience has a difficult time reading the first 6-8 frames in a scene.
26. Does the added action in a scene contribute to the main idea in that scene? Will it help sell it or confuse it?
27. Don’t animate for the sake of animation but think what the character is thinking and what the scene needs to fit into the sequence.
28. Actions can be eliminated and staging "cheated" if it simplifies the picture you are trying to show and is not disturbing to the audience.
29. Spend half your time planning your scene and the other half animating.
30. How to animate a scene of a four-legged character acting and walking: Work out the acting patterns first with the stretch and squash in the body, neck and head; then go back in and animate the legs. Finally, adjust the up and down motion on the body according to the legs.

I pasted them here in case the source site disappears.

Music

I can't listen to music when I'm doing the thinking part of the animation e.g. poses, but I do listen to it when I'm watching the edit. This way I have some internal rhythm to edit to. Of course, this doesn't have to be the final music but just something to set the mood in my brain. The music I listen to varies widely depending on the scene and its emotional intensity/tone. For the more dramatic slower scenes that I've been working on recently, I listen to Zakir Hussain's Kalpana a lot. Not that I listen to a lot of classical music but this track is one of my favorite pieces of music. I would've liked to use it in the final music but it may be too meditative for the film.

Rope a dope

Often, my scenes have funny intermediate states. Sam's hand grab is particularly good when looped.

Thoughts on Animation

How has my approach to character animation changed over the course of this film?

I plan things out a lot more. Probably a third or half the time on each shot is spent planning out the camera and poses. Earlier, I'd rush through the scene and go straight to subtle stuff like the fingers, but now I spend the bulk of my time trying to iron out the timing of the main actions.

I also step the keys a lot more, which is making 3d animation seem more and more like 2d animation since I feel like I'm animating keys and holding them on 3s, 4s or 6s before doing in-betweens. It's all perfectly obvious now, but it's taken me a while to start thinking like that.

I go a lot broader on facial animation and close-ups. A lot of my earlier scenes are quite stiff because I was being too subtle. Some of the faces were so rigid that I went in and reanimated them, particularly with Bir.

On the other hand, my body animation is a lot less broad than the usual manic stuff I churn out. Part of the reason is that Looney Tunes animation wouldn't really fit the content and style of this film, but I also try to plan each scene around fewer big poses than I used to. That way, I only emphasize what I want to in each scene by balancing big and small, fast and slow gestures. So if there's enough slow and medium paced action, fast action stands out more, and vice versa.

Similarly, less is often more. By resisting the temptation to do a zany wrist snap or an overexaggerated take, I am trying not to distract the audience in order to give them more time to digest a shot and understand its main point.

Of course, my animation might still not be any better, but I like to think that it's clearer.

Anatomy of a Shot

I picked a random shot to illustrate the steps that I go through. I'll keep updating this post until the final render.

Shot 53: Goon moves toward Bir
Animatic Duration: 2 sec

Stage 1: Storyboard/Animatic - mid-December 2007



Stage 2: Layout Create Maya scene and import characters. Rough out camera angle/scene composition. - July 2, 2008

Notice that the screen direction is flipped horizontally compared to the storyboards as a side-effect of some story edits that I made during the animation phase of the film.

The character design has also changed significantly. I didn't feel that this character (Sam) needed a beard because the hat is the point of the story. Also, Maqbool already has a beard which will contrast well with Sam's lack of one during the final confrontation scene.

Angle 1:


Angle 2: Roof lines up better with his eyes


Stage 3: Animation July, 2 2008

1st pass blocking/rough animation



More blocking - the hand wipe was really slowing down the momentum of the scene, and felt out of character/confusing i.e. is poor Sam sweating and tired after a long day of killing people? I considered other options such as having Sam wipe his mouth instead or maybe spit, but none of those actions seemed to fit either. So now he's just going to walk towards Bir. By framing the shot wider, Sam doesn't need to raise the knife as high, again making the shot less over-the-top as compared to the storyboards.



Several hours later, this is the final animation. I also adjusted the camera movement.



Since I use RenderMan to smooth the models at render time, I usually do a quick render to preview the animation without paying much attention to lighting.



Sure enough, I ended up tweaking the camera angle based on the preview render - I widened the shot by changing the lens so you can see Sam's leg stepping over the dead body. No one will probably notice, but I like having it in there because it also breaks up the walk cycle a little bit.

Stage 4 - Crowd - November 25, 2008

I usually add in the crowd right before I render, because my pipeline involves getting all the crowd animation into Maya, and this makes the scenes really heavy. At this stage, it takes about 5 minutes just to open the Maya file for the shot.

Stage 5 - Lighting/Rendering - November 27, 2008

I'm not the best lighter :) so everything comes out looking pretty flat usually.

Stage 6 - Compositing/Final Shot: March 8, 2009

Here's where I add dust, depth of field and color correction.

Phew.

Fighting

This the first pass on the beginning of the riot. There will be more characters added, but I'm trying to get the blocking/edit down.

V1:



V2: Tighter shot on the two characters fighting

Animation!

Even though it might seem that all I do for my thesis is collect references, I have been known to animate on occasion. I have a nice long segment done, there are bugs in it but I think it's mostly there.



(As for the titlecards, I've never really liked them and I am playing around with the idea of replacing them with some narration from the boy - same information, but instead of saying "1 million people" maybe the boy can say "lots of people". This will move the intro more towards the boy's POV. The problem is that I already have some dialogue in the film where the mom tells the boy that India has been partitioned. So the boy's narration in the beginning shouldn't make this dialogue redundant since I've already animated it).

Huge update

It pains me to leave this scene looking so crude, but I'm shelving it for now. I'll be back, with better models, more foreground characters and some custom mocap walks. At least I know that I can pump shots through this pipeline, and for the sake of my sanity, I need a break.

Gargantuan

After a week of hacking on Massive, I was finally able to get something out of it.

A couple of highlights:
1. Gave up on RenderMan Pro Server - I haven't been able to get a hold of this software for months now, and I decided to just render with Massive's native hardware renderer and make the most of it. Also, the integration between the Windows version of Massive and RenderMan might present additional problems.

2. Camera moves - you're supposed to be able to import a Maya camera straight into Massive, which is really nice if you want to choreograph the crowd to an existing camera move. But I couldn't get this feature doesn't work, and after two and a half days of segmentation faults and crashes, I gave up. So now I am manually copying the camera keyframes from Maya to Massive, one channel at a time. This process is not as simple as it sounds because Maya and Massive deal with camera aperture in completely different ways. As a result, I had to figure out the conversion between Maya's aperture/film aspect values and Massive's film back to make sure that the camera angles matched. Totally fun.

The first video is what I got when I aligned the camera position but the aperture/filmback didn't match.



The second is a much more accurate alignment after I figured out how to convert between the cameras (the color correction is just something I threw on in Premiere).




So now I have to get the dudes to stop walking back into the train. This is harder than it sounds because I am working with a library agent that Massive provides which is tricky to customize. Still, at this point I'm not averse to manually painting them out if I have to.

Also, I'll work on replacing the models with my own, or at least switch their clothing. I also need to integrate my own motion capture shoot data into this scene. I may just do that as a separate pass at this point.

I'm still not entirely convinced that trying to render in Massive is better than my earlier hacked Maya workflow in which I exported the animation curves out of Massive and then attached them to characters I had created in Maya. The advantage to using Massive is that it's very efficient at rendering (so far) and it's easy to move the agents around, change their outfits, etc. The disadvantages are that the Windows version of Massive isn't very stable, and that I need industry-grade RenderMan Pro Server (to match the rest of my elements that are rendered with RenderMan for Maya) which is not available at my school.

Character facial references

Now that I'm animating, I turned to some of my favourite films for ideas on facial expressions and acting. These really help get me thinking about how far I can push my characters, as well as how to express certain emotions.

Bir
Dash is a really great character study because he's a manipulative brat and cute at the same time - studying him has helped me loosen up Bir's animation so that he's not so angelic all the time.




Santosh
Looking at Elastigirl/Helen Parr helped me develop the mom into more of a character as well, instead of the stiff person I had in my storyboards. However, I've always found the animation on Elastigirl to be just a little bit awkward, so studying the animation also helped me dial into the elegance that I wanted in my character (of course, whether I'll succeed is another issue altogether).

Opening sequence