Animation Pipeline

Here's an overview of the process I've used to animate most shots in my film. After much trial and error, I came up with a hybrid approach that combines keyframe animation, motion capture and crowd simulation.

Step 1: 'Hero' characters - keyframed by hand in Maya



Step 2: Add in background crowd (authored in Massive, then imported into Maya using a custom script)

Step 3: Add in middleground crowd (authored in Massive, then imported into Maya using a custom script) *

Step 4: Add in foreground crowd (motion-captured and solved using Vicon IQ and Motion Builder, then imported into Maya)

Final Shot:

* In my pipeline, the middleground crowd could be either Massive or custom mocap. The tighter the shot, the more I used mocap characters that were placed individually in the scene (which takes ages)

Details:

Three layers of crowd (Foreground, Middleground, Background) seemed to be the most effective at creating believable scenes with depth. I also had to spend tons of time randomizing models, shaders and movement so that the crowd was convincing. Check out this test and this other test to see my earlier attempts.

Crowd simulations generated out of Massive seemed to work best for background characters. The problem with building an entire scene with Massive was that it 'felt' like a simulation (no doubt due to my own inexperience with the software). As a result, I resorted to individually placed motion-captured characters for the foreground and middleground crowd layers which really helped break up the crowd motion.

The other advantage of using custom characters in the FG and MG was that I could art direct the crowd and ensure that the crowd parted at just the right time or that the crowd characters helped direct the viewer's eye to the action.

Although all the keyframes were created in different software, Maya became my central application to combine all the animation elements. This approach deviates from the standard industry practice of rendering crowds straight out of Massive using PRMan, but numerous technical challenges (unavailability of RenderMan Pro Server/PRMan at my school, problems importing Maya cameras into Massive, etc) necessitated changes to my workflow. Therefore, I hacked together a MEL script that imported characters from Massive into Maya, and then rendered everything out with RenderMan for Maya. It wasn't pretty, but it worked.

Making it feel like a 'Movie'

Sometimes, a little bit of editing/camera work and different character models can make a world of difference to the same bit of animation.

1. For instance, here's a fight scene I shot in an introductory motion capture class applied to generic off-the-shelf characters (the old man is actor Kevin J. Ryan and the alien is classmate John Helton):

2. Next, I created some really basic crowd character models and used a segment of the same motion on them, but changed the camera angle:

(This step is important because it verifies that my models have been rigged correctly)

3. That looked ok, so I spent some more time on the camera angles to previz the sequence as it might take place in the film:

4. Hmm, what about with tighter editing, better looking characters and some lighting?

5. And here's the final sequence:

All through this process, the actual animation on the characters was unchanged.

'Topi' featured on VFXWorld + Massive Student Gallery

Some crowd tests that I did for my thesis are now featured on Massive Software's student gallery. There's a lot of other interesting projects that have been posted there as well.

VFXWorld also did a write-up on student work that features Massive, and included an image from my film - check out http://vfxworld.com/?atype=articles&id=3602&page=2

Outtake

Sometimes a crowd character tries to steal the limelight:

Mocap setup

This is an image of an actor, Kevin J. Ryan, that I motion captured performing some fight sequences (with my classmate, John Helton) in the fall semester. At the time, the footage was shot as a test, but I now plan to include it in the final film (it's the fight between Sam and Manoj, my secondary characters).



The markers are very reflective and appear to glow because of the camera flash. Note that the fingers are not mocapped and need to be animated by hand.

(Btw, Kevin's a great actor to work with, and I highly recommend him for both mocap and live-action shoots).

Motion Capture Performance Final Project

Should've posted this a while ago - my final from the Eric Furie/Robert Zemeckis Motion Capture Performance class which includes more crowd work. (I also shot some fight sequences that were not yet reconstructed in time for this video but can be seen here.)



The foreground and middle ground crowd elements are motion captured (the performers are my classmates John and Nahomi, myself and an actor, Kevin), and the background crowd elements are generated out of Massive. This is my new approach to the crowd because I can still art direct the crowd manually and then just give it additional volume using Massive.

I liked where this is going, but my advisor, Christine Panushka, wants to see A LOT more people and some children so I need to work on that. RenderMan for Maya 2.0 supports instancing, so my plan is to create instances of the FG and MG crowd individuals that I already have, giving me a lot more volume closer to the camera without a substantial performance hit (I hope).

Other than that I got good feedback on the look, the design of the crowd, etc. from the faculty.

Fighting

This the first pass on the beginning of the riot. There will be more characters added, but I'm trying to get the blocking/edit down.

V1:



V2: Tighter shot on the two characters fighting

Look Test

First pass render/composite:

Massive woes

I've been unable to import mocap data into Massive without crashing. After some more debugging, I think that the problem comes down to the fact that I am using Maya 2008, whereas Massive only appears to work with Maya 8.5. This incompatibility is probably also what caused all my camera headaches a couple of weeks ago.

For example, when I save an animated camera using Maya 8.5, the end of the file has the following snippet:

connectAttr "persp1_visibility.o" "persp1.v";
connectAttr "persp1_translateX.o" "persp1.tx";
connectAttr "persp1_translateY.o" "persp1.ty";
connectAttr "persp1_translateZ.o" "persp1.tz";
connectAttr "persp1_rotateX.o" "persp1.rx";
connectAttr "persp1_rotateY.o" "persp1.ry";
connectAttr "persp1_rotateZ.o" "persp1.rz";
connectAttr "persp1_scaleX.o" "persp1.sx";
connectAttr "persp1_scaleY.o" "persp1.sy";
connectAttr "persp1_scaleZ.o" "persp1.sz";

On the other hand, in Maya 2008 the same section looks like this:

connectAttr "persp1_visibility.output" "persp1.visibility";
connectAttr "persp1_translateX.output" "persp1.translateX";
connectAttr "persp1_translateY.output" "persp1.translateY";
connectAttr "persp1_translateZ.output" "persp1.translateZ";
connectAttr "persp1_rotateX.output" "persp1.rotateX";
connectAttr "persp1_rotateY.output" "persp1.rotateY";
connectAttr "persp1_rotateZ.output" "persp1.rotateZ";
connectAttr "persp1_scaleX.output" "persp1.scaleX";
connectAttr "persp1_scaleY.output" "persp1.scaleY";
connectAttr "persp1_scaleZ.output" "persp1.scaleZ";

Even a non-programmer can tell that these two code blocks are semantically identical, Maya 2008 is just more verbose than Maya 8.5. Still, that's all it took - when I copy-paste the same code block from my Maya 8.5 file to the Maya 2008 file, everything works like a charm.

More crowd stuff - Salaam Bombay

Salaam Bombay is one of my favorite Indian films (everyone should watch it), and there's a good crowd scene at the end. In fact, it's not too hard to find a couple of shots from this clip in my film :)

Mocap shoot 2

Speaking of mocap, I just got my latest shoot approved by the head of physical production. I have two actors, Lester Reynolds and John Michael Herndon, coming in on Thursday and I'm going to capture some fight sequences and some falls. Since the actors are going to be scuffling, technically I am conducting a hazardous shoot, which meant going through some paperwork and the production division in order to make sure everyone was insured. So I'm all clear and now I've got some planning to do to make sure things go smoothly. I don't have any experience choreographing a fight, but I need to make sure I get what I need. Previz, previz, previz.

I'm planning to buy some foam props and mesh clothing to simulate swords, lathis (staffs) and kurtas. I was inspired by a class visit to the set of the new Zemeckis film ("A Christmas Carol") last week; these little tactile touches do a lot to help an actor get into character.

Huge update

It pains me to leave this scene looking so crude, but I'm shelving it for now. I'll be back, with better models, more foreground characters and some custom mocap walks. At least I know that I can pump shots through this pipeline, and for the sake of my sanity, I need a break.

Gargantuan

After a week of hacking on Massive, I was finally able to get something out of it.

A couple of highlights:
1. Gave up on RenderMan Pro Server - I haven't been able to get a hold of this software for months now, and I decided to just render with Massive's native hardware renderer and make the most of it. Also, the integration between the Windows version of Massive and RenderMan might present additional problems.

2. Camera moves - you're supposed to be able to import a Maya camera straight into Massive, which is really nice if you want to choreograph the crowd to an existing camera move. But I couldn't get this feature doesn't work, and after two and a half days of segmentation faults and crashes, I gave up. So now I am manually copying the camera keyframes from Maya to Massive, one channel at a time. This process is not as simple as it sounds because Maya and Massive deal with camera aperture in completely different ways. As a result, I had to figure out the conversion between Maya's aperture/film aspect values and Massive's film back to make sure that the camera angles matched. Totally fun.

The first video is what I got when I aligned the camera position but the aperture/filmback didn't match.



The second is a much more accurate alignment after I figured out how to convert between the cameras (the color correction is just something I threw on in Premiere).




So now I have to get the dudes to stop walking back into the train. This is harder than it sounds because I am working with a library agent that Massive provides which is tricky to customize. Still, at this point I'm not averse to manually painting them out if I have to.

Also, I'll work on replacing the models with my own, or at least switch their clothing. I also need to integrate my own motion capture shoot data into this scene. I may just do that as a separate pass at this point.

I'm still not entirely convinced that trying to render in Massive is better than my earlier hacked Maya workflow in which I exported the animation curves out of Massive and then attached them to characters I had created in Maya. The advantage to using Massive is that it's very efficient at rendering (so far) and it's easy to move the agents around, change their outfits, etc. The disadvantages are that the Windows version of Massive isn't very stable, and that I need industry-grade RenderMan Pro Server (to match the rest of my elements that are rendered with RenderMan for Maya) which is not available at my school.

Crowd reel

After much procrastination, I am starting mocap shoots for the crowd scenes in the film. My goal will be to capture small segments of animation (e.g. walks, characters sitting, etc) that I can blend together to create different crowd people. I hope to use Massive to achieve this blending, but given my lack of progress at learning the software, I may just have to do all of this in Motion Builder and Maya (i.e. create the scene by hand, instead of relying on Massive's AI).

I compiled a reel of crowd shots from 'Gandhi' and 'Water' to help me figure out what I need.

Second crowd test

For this test, I started to explore what the crowd might look like before the riot starts. I still want to stay with the idea of silhouettes, but they don't need to be black.

I like what I came up with because it's getting away from the idea of distinct 3D characters in the crowd and starting to resemble a moving painting. I've still got a lot of work to do - for starters, I need to get some more dust and atmosphere in. I also need to integrate Bir and Maqbool so that I can see if the crowd design is too distracting.




**** Update
Christine agreed that the projected texture looks weird when the characters are really close to camera. So I need to try to give each character a unique texture, perhaps by projecting the textures in 3D.

Motion Tests

Based on my meetings with Mike Patterson and Christine Panushka last week, I decided to try some motion tests to see if my ideas for the crowd were even feasible.

Massive (the crowd simulation software) doesn't export geometry to Maya, so I had to hack together a script to get my Massive animation to drive Maya characters. These tests use just one character although the final version of the film will have more variety.

Here's the 3D render:



The composite with color correction, temporary smoke and a background:
***Update: I also added Bir to this shot



I think the next step is to get some more handdrawn texture onto the crowd and Bir. Also, the white caps are not really that visible right now.