Animation Pipeline

Here's an overview of the process I've used to animate most shots in my film. After much trial and error, I came up with a hybrid approach that combines keyframe animation, motion capture and crowd simulation.

Step 1: 'Hero' characters - keyframed by hand in Maya



Step 2: Add in background crowd (authored in Massive, then imported into Maya using a custom script)

Step 3: Add in middleground crowd (authored in Massive, then imported into Maya using a custom script) *

Step 4: Add in foreground crowd (motion-captured and solved using Vicon IQ and Motion Builder, then imported into Maya)

Final Shot:

* In my pipeline, the middleground crowd could be either Massive or custom mocap. The tighter the shot, the more I used mocap characters that were placed individually in the scene (which takes ages)

Details:

Three layers of crowd (Foreground, Middleground, Background) seemed to be the most effective at creating believable scenes with depth. I also had to spend tons of time randomizing models, shaders and movement so that the crowd was convincing. Check out this test and this other test to see my earlier attempts.

Crowd simulations generated out of Massive seemed to work best for background characters. The problem with building an entire scene with Massive was that it 'felt' like a simulation (no doubt due to my own inexperience with the software). As a result, I resorted to individually placed motion-captured characters for the foreground and middleground crowd layers which really helped break up the crowd motion.

The other advantage of using custom characters in the FG and MG was that I could art direct the crowd and ensure that the crowd parted at just the right time or that the crowd characters helped direct the viewer's eye to the action.

Although all the keyframes were created in different software, Maya became my central application to combine all the animation elements. This approach deviates from the standard industry practice of rendering crowds straight out of Massive using PRMan, but numerous technical challenges (unavailability of RenderMan Pro Server/PRMan at my school, problems importing Maya cameras into Massive, etc) necessitated changes to my workflow. Therefore, I hacked together a MEL script that imported characters from Massive into Maya, and then rendered everything out with RenderMan for Maya. It wasn't pretty, but it worked.

My, What Long Arms You Have!

As I get closer to finishing, I cheat more and more. In this case, I wanted to exaggerate perspective and scale, and had to stretch out Sam's arm to make the shot work.

View from Shot Cam:

View from a different angle:

Topi featured in 'Introducing Maya 2009'

A couple of stills from 'Topi' have been included in Dariush Derakhshani's new book, 'Introducing Maya 2009'.

The still of Bir in the book is actually an early version. The final composite looks more like this:

Needless to say, I'm super-stoked that the images were published.

Making it feel like a 'Movie'

Sometimes, a little bit of editing/camera work and different character models can make a world of difference to the same bit of animation.

1. For instance, here's a fight scene I shot in an introductory motion capture class applied to generic off-the-shelf characters (the old man is actor Kevin J. Ryan and the alien is classmate John Helton):

2. Next, I created some really basic crowd character models and used a segment of the same motion on them, but changed the camera angle:

(This step is important because it verifies that my models have been rigged correctly)

3. That looked ok, so I spent some more time on the camera angles to previz the sequence as it might take place in the film:

4. Hmm, what about with tighter editing, better looking characters and some lighting?

5. And here's the final sequence:

All through this process, the actual animation on the characters was unchanged.

Composite Breakdown

I started off compositing in After Effects, but have since shifted over to Nuke. I was reluctant to learn another piece of software at first because I had already had to learn how to use Massive, RenderMan for Maya, IQ, Motion Builder and others for this film. But I'm glad I gave Nuke a shot, because it has turned out to be a lightweight, fast and intuitive package.

I like rendering in layers, it gives me a lot more control in the composite.

For example:

Render Layers:

• Character (from multipass render):
  

• Middleground Crowd:
  

• Background Crowd:
  

• Ground:
  

• Platform:
  

• Sky:
  

Result of combining the layers with no compositing:

Final composite (after color correction and addition of atmospheric elements):

Working in 16-bit color is also very useful, as it expands the range within which you can color correct the image without creating artifacts.

Render Workflow Overview

The main reasons for using RenderMan for Maya as my renderer are:

1. Speed, especially when scenes are not raytraced
2. Fast motion blur, which is important in my film
   
3. Good memory management, which means fewer crashes (my Maya files contain anywhere between 2 and 200 characters, and can get as large as .5 GB)
4. Render-time smoothing (i.e. subdivision scheme) enables me to avoid heavy polygon meshes in my already huge scenes
   
5. Integration with Massive - initially, I was going to use Massive's direct integration with RenderMan, but have since abandoned this approach
6. I'm a geek - I've always wanted to learn more about RenderMan because of its role in the history of CG
   

Due to time and resource constraints, I can't render everything with all the bells and whistles I like. I've avoided ray tracing because the size of my scenes would probably cause all sorts of memory issues and require heavy optimization. As a substitute, I've relied on RenderMan for Maya's point cloud occlusion, which is a non-raytraced solution. Even with this feature, however, I can't render entire scenes in a reasonable time frame. (i.e. less than 10 minutes per frame as I only have access to 4-6 render nodes at a time).

Therefore, I only use occlusion on the 'hero' characters such as Bir, his mother, the old man, the goons, etc. For the rest of the characters and the set, I baked the occlusion into the respective file textures using Mental Ray. While baked occlusion doesn't look as good as the real thing, the rendering speedup makes it a worthwhile compromise.

So for example, a multipass exr might contain:

DiffuseDirect:

DiffuseIndirect (i.e. Shadowing):

DiffuseEnvironment:

OcclusionIndirect (i.e. Ambient Occlusion):

alpha:

Put the above layers together*, and you get:

Beauty:

Initially, I was rendering out all my render layers (such as the background, sky, etc) into multipass EXRs, but found that it was overkill and I really just needed that extra level of control on the hero characters.

* The actual compositing operations are:

(DiffuseDirect - DiffuseDirectShadow + DiffuseEnvironment) * OcclusionIndirect

** I left out the Specular components for simplicity, but the math is the same as for Diffuse

Confrontation Revisited

The confrontation between Maqbool and the goons has been bothering me for some time now. It feels a little sluggish and lacks impact. With input from colleague Joanna Griebel, I reedited the sequence.

Old:

New:

The new cut has an added line of dialogue for Maqbool ("He dropped his cap"). The old edit seemed unclear to a lot of viewers, and I think adding this line back in is an improvement. (The line was in the original script, but I had taken it out for some reason that I can no longer remember. Perhaps I felt it was redundant at the time.). Another reason I like the new line is that it is a reference to Bir's "Sir, you dropped your cap" earlier in the film.