In our previous blog, “Debunking the Pre-Testing Vs. Creativity Myth”, System1 co-founder Rod Connors challenged the belief that idea testing in early stage creative development destroys great ideas. Now I’ll tackle a related question – what’s the most effective form of stimulus for ad testing? – and the belief that animatics can’t be a good indicator of how a final film performs in market. Really?

It’s true there was a time when animatics made characters look like robots, numbing their all-important emotions in the process. Likewise, storyboards often relied heavily on voiceover and lengthy explanations… and we all know what it feels like when a joke needs explaining.

But it needn’t be so. There’s all the kit and talent available to create great animatics these days. “Great” as in research stimulus that actually makes you feel something – which is what matters most. So then, is this yet another case of challenging our beliefs or, even better, just trying out something new?

System1 Agency’s process relies on what we call ‘evidence-based creativity’ i.e. we quantitatively test and retest the ideas we develop with our creative network until they reach 3, 4 or 5 Stars. That’s on a 1 to 5 scale where 50% of ads score just 1 or 2 Stars and only 4% go all the way to 5-Stars. For examples of 5-Star ads, take a look at BrainJuicer’s FeelMore50™ ranking of the world’s most emotional ads.

Hitting 3-5 Stars is the promise we make to our clients and we keep working until we get there. This ‘assume nothing and test everything’ approach allows the optimisation loop of generate-test-fail-and-learn to work its magic. But what is the most reliable stimulus for ad testing early on in the creative development process?

What matters is choosing a format that’s likely to convey the intended feel of the finished film best.

The good news is that there’s a plethora of options to choose from (all with specific jargon attached): videomatics, stealomatics, cinematics, boardomatics to name a few. What it boils down to is making choices between 2D vs. 3D or between a hand-drawn feel vs. computer generated images.

There’s no single answer to what’s best (believe me – we have tested the same idea using different stimulus formats). What matters is choosing a format that’s likely to convey the intended feel of the finished film best. Here consider things like the relative importance of movement, energy, pace, character developments and their relationships, visual awe or sensory pleasure, face movement and gestures needed to tell the story.

OK so we’ve established that there are now real options available for testing creative early on. Good. But then what’s the relation between early stage and final films in testing? Here’s what Orlando Wood, MD Labs at BrainJuicer has to say: “The emotional response to an animatic is a good guide to how the finished film will perform, and we typically see an uplift for the finished film. Weak animatics might stand to gain a little more when they make the transition to finished film, but the leap is unlikely to be great. The hard truth is that a poor idea is a poor idea. And a good idea when well-crafted in animatic form, is likely to transition to an even better-scoring finished film. With a sympathetic and skilfully constructed treatment, there is no reason why testing as an animatic should punish a good idea. For instance, one recent IPA Gold winner tested well with us early in development as an animatic, was progressed and then achieved 5-stars as a finished film.”

So where’s the problem? We’re back to the assumptions we make and another chunky one is that “the idea will get better as it’s turned into a final film”. BrainJucier has tested animatics followed by finished films in all sorts of categories and markets and results really go both ways: most animatics tend to get better, true, but a surprisingly large number end up testing worse when converted into final films. That’s why we provide a margin of progression (positive) as a well as a regression margin (negative) when debriefing animatic test results.

Discrepancies between early stage and finished films often occur when length, music and early stage film style are not consistent with the finished film. So let’s end on two tips:

Make sure the length of the early stage animatic and finished film are the same (or at least close). Early stage films that are shorter or longer than the finished film can under- or over-represent the potential of the idea, obviously.

Use the music in the early stage film that you intend to use in the final film. That’s really important as music speaks volumes to our System 1 and so it’s an essential part of the creative, not a simple ad-on for the end.

The author, Mark Johnson, is Managing Director - Europe of System1 Agency. Mark joins from sister agency BrainJuicer and has worked in PR, Brand Strategy, Innovation and Consumer Insights on both the client and agency side. If you want to find out more about System1’s “secret sauce” for developing animatics, you can contact Mark at

Related Blogs


Abertura de anunciantes brasileiros a novos modelos anima agências estrangeiras

Read More

2 Things Your Agency Should be Doing to Create 5-Star Advertising

Read More

Why Adidas was right to shun viral spoof ad ‘Break Free’

Read More