My current animation workflow relies a lot on reference footage and what I can bring into my animations from my own acting. It works great and produces good results. So problem solved? Animation workflow mastered and ready to take on any studio or supervisor? NOPE, not even if I work in two different situations that both require reference footage.
What is a reference footage workflow for animation?
In short: Reference footage are videos an animator uses as a guide for their animation.
Before we get into the problems, let’s discuss very briefly the advantages of reference footage, which I described more fully in this article. For new animators out there or people who aren’t yet animators, reference footage is used to inspire and guide an animation. Us animators often record the reference footage we use ourselves and other times we find videos. Some animators use reference more closely than others but for realistic styles like mine, it is used extensively. Like me, most animators will have their video reference open and by the side of wherever they’re working. We don’t trace and it isn’t the same as motion capture, it’s simply there for us to look at and study.
Why does using reference footage in animation cause problems?
In short: Reference footage can cause problems because animations aren’t meant to move exactly as they do in real life and look weird if they do.
First off, reference footage solves more problems than it causes. The advantages of reference footage are that acting and movement looks more believable in the finished animation. The problems with using video reference to animate are usually due to the fact that animated characters rarely move like actual human beings. Just as an illustration or cartoon of someone doesn’t look like their photo, an animation of someone moves in a different way to a live action video of them. In fact, animations have to move differently to live action because otherwise it looks really weird.
Another reason reference footage causes problems is because the people in the live action video often don’t look like the animated characters and often there are parts of the video reference that will need to be changed for various reasons.
Uncanny valley animations?
You’ve probably heard of uncanny valley before but in case you haven’t, uncanny valley is the unease we humans feel when we see an artificial representation of something look too realistic. From robots to clay sculptures, if a representation of an organic being has too many realistic details, it looks weird. Even dead stuffed animals (taxidermy) results in weird looking creatures. The reason all those things look weird is because as realistic as they are, we can tell that something is off about them and that disturbs our senses.
To be fair, uncanny valley is usually a problem with character design and models rather than the actual movement they make. So we don’t say an animation is in uncanny valley if the animator followed reference too much. However, sometimes real life movements on animated characters can look floaty, slow or linear. On occasion, real life movements even look like animation mistakes on animated characters.
Using reference footage in animation is a balancing act
I hope that quick overview of reference footage has helped any less experienced or non animators out there understand why problems arise. Animators have to do dose their animation with the right amount of realistic details. Too few realistic details and the animation won’t look believable, too many details and it’ll look weird.
Problems that reference footage can cause animators
Now that we went through what reference footage is and why it can cause problems for animators, let’s get into specific details. These are some of the things that can go wrong and how to put them right. I’ve written this from my own experience and intend it to help any other animators who use reference footage and also other animators tried and had trouble animating from reference. Some of this is obvious however I’ve always found it useful to remind myself.
Weak poses from reference video
In short: The poses you or others make in videos are often going to need perfecting for artistic flair and clarity.
The planning workflow I have right now works well for what I do because I need to represent lifelike characters. My reference footage gives these characters the smallest subtleties and makes them feel very human. I’m still an animator however and the animations I make are still stylised. One frequent weakness of reference footage is that the poses I might make in my video don’t look as good as they could, so will need to be altered during the animation process.
Posing issues in reference footage need to be solved during animation. Each animator will choose to solve weak reference poses at different stages in the animation process. Different supervisors or directors might also ask you to address the weaknesses of reference footage sooner or later according to their preference and your skill level (changing posing sooner requires more skill).
Example of weak poses I had to change in animation
This is a still from an old personal animation I did in 2017. There are several things in it that I dislike and would do better now. The reason I’m using this example rather than stuff from Arcane or my other professional projects is because I prefer using personal examples while I’m critiquing my own work online. If I hadn’t changed both of the poses in this example, the resulting animation would have been much weaker.
You should be able to see the pose differences between the animation and the reference video. While I now dislike some parts of the pose on the guy, his pose is still far more clear than the one I held in my reference footage. In the animation, he’s holding his own neck and looks extremely anxious. The whole shot is about how awkward the guy makes the situation, so making him even more anxious is a major storytelling decision.
The woman’s pose is closer to the reference but there are subtle differences. First of all, I pulled her hand out from the silhouette of her body and I placed her other hand on her knee to keep it from hiding her face. The woman in my animation is also pointing slightly more away from the guy than in my reference, because I wanted her change of direction to be bigger.
Option 1: Changing reference footage poses immediately during blocking
One way of dealing with reference footage is to make artistic choices as I’m starting to pose my character for the first keyframes. In this case I’ll choose a pose I want from the reference video and then pose my character while changing limb placement etc to improve things like the silhouette and general appeal of the character. The idea in this case would be that I avoid making destructive pose changes later on in the animation process.
This approach to solving posing issues is based on initiative and anticipation of what I’ll later need to change. It can work very well to make these choices early on, however it isn’t always the best choice. Sometimes I can change poses too much and lose the essence of what I liked in my reference footage. In that case I will have to go back to my reference footage and copy it more closely onto the character in order to recapture what it was that I wanted.
Option 2: Changing poses during blocking plus or spline
Another way of dealing with weak poses from reference footage is to change them later on in the animation process. In this case I’ll choose a pose I want from reference and copy it as closely as possible onto my character. This means that the silhouette may not be ideal and the appeal of the character might not be as strong initially. What I do know by using this method is that I’ll capture a performance very close to what I have in the reference video.
The shortcoming of changing poses later on in the animation process is that it can break motion that I already finished detailing. Changing major poses in an animation can mean deleting breakdowns and in-betweens, then putting them back in after changing the poses.
Which choice is better? Fix poses sooner or later?
There are strengths and weaknesses to each solution and when working with reference footage, I need to choose one or the other. Depending on the circumstances, the best solution can change. In the event that I understand the motion completely or have experience animating that kind of thing before, I’ll usually adapt poses immediately because it is most likely going to save me time. If I don’t have a lot of experience animating a motion or it seems complicated, I may well choose to adapt posing later on after I’m already sure I’ve gotten what I needed.
What about the preferences of supervisors or directors? In the case of posing, it matters. Some directors don’t feel comfortable imagining what I might do with a pose and so prefer me to show them all the adaptions I plan to make from the start. Other directors feel they could be losing detail if they allow me to deviate from my reference footage, so may ask me not to change the posing too much initially.
Interpreting timing and spacing in reference footage for animation
In short: Real life video can move in a linear and rhythmically boring way. Animators need to tweak movement to give it rhythm.
Another weakness of reference footage can be the timing and spacing it presents. Like the problem solving I described with posing from reference footage, timing and spacing require a similar choice. It can either be changed during blocking early on or adjusted later in blocking plus, spline or polish. Like with posing, some supervisors and directors require it to be done sooner or later in the process.
While I prefer changing poses early on in the animation process, I prefer changing timing and spacing later on, especially for acting. I’m more likely to miss performance details if I mess with the timing early on in my animation process, particularly if I’m working to a realistic style. There are exceptions though, for example with simple body mechanics like an arm falling down to a table; in such a case I’d probably design the spacing of poses on the arm early on.
Example of an animation where I changed the timing
This animation is the same when I used above to describe posing changes. Now I invite you to watch the animation and take note of the significant timing differences between how the characters move in my animation vs how they moved in my reference footage. I’m almost constantly moving in my reference footage and also distract from the performance of my wife. In order to render the female character’s performance more clear, I had to hold poses on the male character so that the audience doesn’t need to look at him while she’s moving. The result is that the man moves less often but quicker. It improves the back and forth interaction within the shot but also the rhythm of the animation. Keeping the timing I had in my reference footage would have resulted in weak animation.
The animated character’s body type is different to the person in the reference footage
Back when I was a junior animator I remember struggling when I had to adapt what I saw in my reference footage to a character with different sized limbs. Adapting a pose for a character with much longer arms and legs can sometimes be tricky, particularly to maintain negative space and a good silhouette. Likewise, adapting a reference pose from someone who’s very slim to a character who has a lot of muscle or fat requires creative juice.
Unlike with the other earlier things I described, adapting a reference pose to the animated character’s body type needs to be done immediately. Poses need to be adjusted and tweaked and it isn’t possible to simple “copy and paste” what is seen in the reference footage to such a character. Sometimes the animated character’s limbs etc can be scaled but there is usually a small margin of what is allowed before the character is considered off-model.
The animated character’s personality is too different to yours
Occasionally it’s best to get someone else to do my reference footage. When working at studios, my colleagues usually don’t mind trying to capture something for me if they’re physically a better match for the character I’m working on. Sometimes it can even be a case of personality rather than body type, where a colleague’s personality is more suitable for a character than mine.
Most animators act better in reference footage if the character they’re trying to perform has a personality and temperament that’s closer to their own. We have some acting skills and can perform wider ranges of personalities if we put a lot of effort in, however in reality we’re often changing which character we’re working on too frequently to put ourselves in the zone of a personality completely different to our own every time. Unlike an actor who will focus on getting into one character’s head, an animator can work on one character for a 10 second shot, then move on to several other characters afterwards, one after the other.
An example of this would be if I had to record reference footage for an animated character who is deliriously happy about ice cream. The problem is that I’d almost never act that way in real life, so unless I spent a lot of time rehearsing, me behaving like that would look contrived or too much like a cartoon. When I have a quota to take care of and a deadline to meet, I’m not going to waste four hours rerecording myself attempting to be excited about ice-cream when I know that my happy-go-lucky, cheerful colleague could get it done in 20 minutes. Likewise, my French colleagues occasionally ask me to record myself saying lines of dialogue if their character has a British accent, because I’m British and it’s easier for me to talk like that naturally.
Your reference footage actors are trying not to hurt themselves
Action scenes can be violent and dangerous. Sometimes animators will look at actual videos of fights and stunts in order to animate a scene but often they will try to record an action to get it closer to what they need. The problem with recording reference footage for fights or extreme action is that animators have to avoid injuring themselves. We aren’t trained stunt artists and sometimes don’t have the appropriate equipment, so when we mimic fights in front of a camera, our hesitation shows and our attacks are slow and controlled.
If you follow your reference footage exactly for such extreme scenes, you risk translating the hesitation and lack of force from your reference footage into your animation. You need to look at external videos of real fights and stunts. A lot of the time, you’ll need to animate largely from imagination and heavily diverge from the posing and spacing. Usually extreme shots like this are better done by very experienced animators, because it’s a little like animating without a safety net (the safety net being the reference footage).
Your director liked your reference footage but dislikes part of the resulting animation
In an ideal world, animators would get their reference footage validated, then their blocking validated, then their spline and live happily ever after. In reality, animators are lucky if nothing changes from their original reference footage. It’s very common for animators to have to reanimate parts of their animation because the director or supervisor found that it didn’t work.
As a young animator it can be very frustrating when directors change direction suddenly and even experienced animators get tired of it. It happens though and it’s best to be prepared for it. To understand a director’s point of view often helps. They don’t always have all of the information they need when an animator first starts working on an animation. For instance they may validate one piece of reference footage before seeing everyone else’s, because different animators will need to start animating a sequence at different times. Sometimes things pop up later that weren’t obvious at the start. Other times, directors don’t know how an animator intends to interpret their reference. Directors are just human and so sometimes just get things wrong as well… they’re looking at so many different things at the same time that it’s easy to overlook a weak part of an animator’s reference footage.
Referencing my own animation that I put above, you’ll notice that there is a second piece of reference footage that briefly appears. This is because my mentor at the time recommended that I give more time to the female’s character before the male character starts moving again. In order to give the female character more time, I had to do the pointing backwards gesture that I originally did in my reference while sitting within my walking forwards movement. This was the reason a reference reshoot was necessary in that case.
