This Successful Movie Animator’s After-Film Reviews Hold Insights for Your Own PIAs
The Post incident Analysis (PIA) sounds like a great idea: responders to an incident objectively and thoughtfully review their response to improve future responses. What’s not to like?
Well….
For many departments, PIAs simply don’t live up to their expectations. What begins as a practice of great promise gradually loses its positive energy until it’s quietly dropped.
Why? Because although PIAs sound good in theory, they all too often crash upon the cold hard realities of human nature. Ego and defense mechanisms join forces to keep even the bravest from engaging in open and honest discussion.
And, that’s before the finger-pointing starts. Sooner or later, you’re going to have a PIA where someone confidently opines on what went south during an incident and who caused it. Which, by the way, is rarely if ever the person pointing all this out.
But perhaps the biggest weakness to these reviews is that they rarely seem to change anything. After several PIA sessions, people will start to notice that they’re talking about the same problems over and over.
If it’s any consolation, this kind of review failure isn’t unique to the fire service—every industry struggles with objectively evaluating their performance. Human nature is universal, and so are the obstacles to honest inquiry about past performances.
Take, for example, Pixar Animation Studios, the media company that produced Toy Story, Finding Nemo and Monster’s Inc. According to Pixar’s former president Ed Catmull writing in his book Creativity, Inc., the company started holding their version of a PIA (an after-film review, or what Catmull affectionately calls a “post-mortem”) after finalizing A Bug’s Life in 1998.
That first review did well enough that the Pixar leadership decided to conduct them after every film wrap. But then something odd happened: The next few reviews didn’t match the success of the first one. In fact, they were busts.
Catmull and his team learned their lesson—even dedicated film artists and technicians were reluctant to stick their neck out and speak freely about their performance. Conversely, they were much less reluctant to engage CYA, kiss-up complimenting, and other disingenuous behaviors. (Sound familiar?)
But Catmull still believed their after-film reviews were a good thing. So, they persisted in their efforts to improve the review process. Eventually, those “post-mortems” evolved into something productive.
Out of that fine-tuning, Catmull says a number of principles for effective post reviews began to emerge. And, it’s those same 5 principles that just might transform your PIAs into something just as useful and transforming.
1. “Account for what’s been learned.”
Every incident response, big or small, should be viewed as a learning experience. There are always basic questions to ask every time: What worked? What didn’t work? What caused such and such to happen? Analyzing the answers to those questions and others can produce some overall conclusions.
Those conclusions, though, can quickly evaporate if you don’t account for the underlying lessons in some formal way. One way is to generate a “Conclusions Drawn” report from each PIA, and make it available to everyone in your agency.
When the findings of a PIA are codified in writing, they have a better chance of seeping into your department’s institutional bloodstream. Once it does, it could fuel important changes to your standard procedures and guidelines, as well as future training protocols.
2. “Teach others who weren’t there.”
It’s easy to assume PIA sessions only benefit those who responded to the incident in question. But that’s too limited an application: an effective PIA can have a positive effect on members who weren’t there, as well as those who were.
I recall one PIA after an MVA response I hadn’t made to a van transporting state prisoners. A key discussion centered around the added difficulty in extrication posed by restraints that chained the prisoners to the interior of the van. Although I wasn’t there, the discussion made a big impression on me.
You should aim for the discussion and conclusions of your PIAs to produce the same eye-opening experiences for every member of your department, not just for those who responded.
3. “Dealing with resentments.”
You’d think working on a happy, little animation film would be all giggles and smiles. Nope: Catmull readily admits producing even an artistically and financially successful film can create interpersonal conflict that has the potential to congeal into deep-seated resentment.
Consider then the tensions that can arise during an intense emergency response where mistakes could become a matter of life or death. It’s all too easy for tempers to flare—especially over premature conclusions—as responders head back to quarters muttering about what went wrong and by who.
This can cause resentment to unconsciously smolder among the crew. Add just the right spark later and it could cause an emotional flashover.
A PIA might actually help prick the resentment bubble—if you handle it correctly. You need an atmosphere where people can speak honestly but still maintain mutual respect for one another.
It’s best then, to have a few ground rules to that effect, and remind all participants—possibly using yourself as an example—that even a pro can have a less than stellar performances from time to time.
4. “Schedule and announce ahead of time to give people time to think and prepare.”
Some supervisors believe the sooner you hold a PIA after an incident the better, because events will still be fresh in everyone’s minds. But Pixar took a different view with their post-film reviews: do them later.
Catmull suggests that giving people a little breathing space after the event affords them time to think more clearly about what happened. This in turn can produce a more thoughtful and insightful discussion.
To preserve freshness, though, you can encourage responders to record their thoughts about the incident as soon as possible afterward. You can also ask them to mentally prepare for the PIA by giving them a few prompt questions to mull over in the meantime (“What do you think went right/wrong during the incident?”; “What obstacles did you encounter?”; “What surprised you about the incident?”).
5. “Look forward to the next project.”
A single PIA won’t immediately fix all the underlying problems that hindered a particular incident. Neither did Pixar’s post-mortems: in fact, Catmull says their reviews created more questions than answers.
Which, he said, was actually a good thing. Real, lasting change comes when you look beyond the knee-jerk reaction answers (“We’ve got to start doing better 360s!”) to more probing questions that don’t have immediate answers.
At your next PIA, try a technique called “5 Why Questions.” When someone brings up a problem at the scene, ask why it happened; to that answer, ask why again, on and on for a total of five times. By probing deeper with the simple question, “Why?”, you might open a bigger can of worms—a good thing if you’re wanting to uncover the real essence of a complex problem.
Having more questions than answers when you finish can be unsettling. Thus, keep the focus on the next incident: How will what happened here affect our next call? How can we immediately change to meet the challenge in a better way?
Stay on track with the facts
You may be looking at these Pixar principles as easier said than done. For sure, a lot depends on a moderator’s skill in keeping the discussion open and on track, allowing people to vent frustrations without veering off into personal attacks.
To that end, Catmull has a couple of practical tips for encouraging “respectful openness.”
First, use incident data (the facts about what happened) wherever possible. Data has no opinion—it is what it is. And, although data can’t tell the whole story by itself without analysis and interpretation, it can provide an anchor—and possibly a stimulus—for discussion.
Second, try this modified Pixar practice: have every participant (preferably before the meeting) make two lists: the top 5 things they think went right during the response and the top 5 things they think went wrong. Comparing these lists will not only help stimulate thinking, it will get participants to focus on the most important issues needing attention.
It’s not likely a single PIA will produce a transformative change in your department. But what they can do, if done consistently, is produce a cumulative effect that can lead your department to more effective responses.
David Webster served twenty-seven years with the Hattiesburg Fire Department in Mississippi before retiring as fire chief in 2013. Besides The Creative Fire Officer, He also writes marketing content for businesses.
