How to describe forecasts to busy executives
Share information not details, use repetition and be concise
Data-driven forecasts can offer a peek into a possible future. They allow you to plan, set goals and strategize around foreseeable problems. Depending on the use case, they can help you become more efficient with resources and costs.
If your team is developing or shifting to statistical forecasts for the first time (from more arithmetical Excel-based projections), presenting these to busy senior business executives requires some pre-planning.
To complete this task successfully, you need to
Convey the right information. Not all details are important.
Gain their trust in the forecasts. Without that trust, they will be reluctant to take action
Do all of the above quickly. So, make it concise and make it stick
Set the template for the next time you present forecasts. Templates make it efficient to transfer similar information each time
If you are able to check the boxes for these four points, you would have successfully delivered forecasts and developed a playbook that you can reuse
How to deliver forecasts effectively
1. Share information, not details.
For a recent mobility software client company, our team had to forecast daily car arrivals in parking lots across the United States.
Our data science colleagues tested a variety of potential inputs in the model, tossed some out and also tried a few different forecasting methodologies to generate forecasts. We had multiple internal discussions and obsessed over each and every detail about which input or signal was useful in the model for forecasting arrivals and to what degree.
When it was time to present the new forecasts to the execs for the first time, we needed to show thoroughness in our process (this also helps create trust). And we had to do it quickly and without frustrating our stakeholders with irrelevant details.
How do you strike that balance between sharing enough detail to create trust but not so much that you overwhelm an exec?
Modeling techniques matter to statisticians for developing robust solutions. However, business teams tend to be more focused on revenue or other business performance metrics. To them, forecasts matter to the extent that they will influence these metrics in the future. When considering what details to include, ask yourself if it directly impacts the decision that execs are trying to make.
If you tested multiple factors and multiple methods to narrow it down, share only the winners of your testing and experimentation. You can mention that you eliminated X number of approaches but no need to spell out which one/s.
Deliver it in an easy to skim format such as the one below. Just one slide or table should sufficiently convey what you accomplished.
For illustration purposes only
For the accompanying narrative for the slide or table, plan to spend no more than 3-4 sentences on explaining the contents of the table. Consider the SART format.
The situation (S) or context. Since they already know this, no need to repeat it unless you have more context to add
Action (A) that you took; e.g. we wanted to see if a,b,c factors affected outcome y and whether that signal was weak, moderate or strong
Results (R) - e.g. we explored a few techniques, applied time-series forecasting method and found signals listed in table 1 above
The Takeaway (T) in as few words as possible
2. Create trust
Your stakeholders have to trust your forecasts if they are to use them. It is rare to come across someone whose first reaction to any new forecast isn’t laced with a dash of skepticism, at least until they know more.
There are two types of trust that come into play when sharing forecasts or results from any statistical analysis
Trust in the process and
Trust in the results
Point #1 addressed trust in the process - that the appropriate method or procedure was used, all possible signals were considered and experienced judgment was applied.
Going back to our example of car arrivals, we created trust in the results by sharing two key pieces of information.
First, we used a holdout period to show how well forecasts were performing. For example, if we were using monthly data until October 31 of that year, we used data only until August 31 to train or build our statistical forecast model. We then produced forecasts for the months of September and October. We could then compare the forecasts with the actuals to see if they were close. If so, we could conclude that our forecast model had a high level of accuracy. This served as a ‘live demo’ of sorts for our exec audience.
As always, make sure that you cover this on 1-2 slides at the most. There is no need to visualize this accuracy for each and every month or year. Think of just how much you need to show to create a feeling of trust rather than to prove that you did for each unit of interest
Also include the range of values for accuracy to indicate that you did not cherry pick the ‘closest’ forecasts to actuals relationship
Second, from a plethora of options, we identified only two metrics to show forecast model accuracy. Resist the temptation to use too many accuracy metrics. It will only confuse the audience.
We selected the following two metrics only because they best suited this particular use case. We have used different ones in different projects, depending on the problem at hand. You might have to make your own choices here.R-Squared - This metric is relatively simple to explain to non-technical folks. It indicates what percent of the outcome you are forecasting can be explained by the inputs employed in the model.
For example, if you are forecasting sales of personal computers by month, the inputs include product and pricing data among others, and the reported r-square is 75%, you can frame your conclusion in layman’s terms as "Included inputs explain around 75% of the variation in personal computer sales”While R-Sq indicates percentage explained, we also used an ‘error’ metric called MAPE - Mean Absolute Percentage Error to show percentage missed by the model. This metric, also expressed in percentage, shows how far the forecasts were from actuals on average.
For example, a MAPE of 30% indicates that average absolute percentage difference between forecasts and actuals is 30%.
Additional ways to build trust are:
Show confidence in your results but also point out any caveats, ifs and buts to provide context. Talk about any pitfalls of interpretation. Remember that they may present it forward to their stakeholders or act on it and they must always have the complete picture
On that note, transparency is important. Provide documentation in layman’s terms with more details that they can refer to when needed. Better yet, create a wiki that you can update from time to time. Include your email address so that they can ask questions quickly - all tactics that help with trust-building
You will be asked questions. Don’t be defensive. Instead, be curious - what is their motivation for asking that question? Where are they coming from? Those are the underlying questions that need to be answered to assuage any concerns
If they had proposed any hypotheses at the start of the project, make sure you test them even if you don’t think they are likely to be significant. It shows respect and that you are open to suggestions - both of which promote trust. If you did not have enough time to test it, acknowledge that as well and include it in the next round of solutioning
3. Make it concise and make it stick
About brevity
We covered much of the brevity parts in the points above. When in doubt about what or how much to include, put yourself in their shoes - what do they care about the most? As mentioned above, they want to know how it will impact the business decision that they are trying to make. Keep that in the back of your mind as you consider what nuggets of analysis to share.
How do you make it concise?
Many people struggle with this one. I did too - it was a constant tug of war between my need to impress my audience and respond to their need to just give them the meat.
Let’s face it. We all want to appear competent. In the corporate world, our progress depends on it. I remember that early in my career, it was hard to avoid the temptation to show off how much rigor had been applied in any analysis. As time moved on and experience grew, I realized that real success lay in being concise yet complete.
To achieve this, try the SART approach mentioned above in your narrative or voiceover and share results in a single table or slide. If you must, include additional details in the appendix and mention that verbally.
About repetition
How do you make it stick such that your audience remembers the key takeaways?
Through repetition! As Eric Schmidt, the former Google CEO, used to say, “repetition doesn’t spoil the prayer.” Share additional forecasts in the same format where applicable and with the exact same colors and placement of key details.
Emphasize the important points by repeating them but acknowledging that you are repeating them; for example “as we saw on the previous slide/page/section, forecast accuracy as indicated by X metric is high at 90%”.
4. Set the template
I love templates! They provide structure, clarity, discipline by forcing you to stay within the constraints and a mechanism for prioritization. They make it efficient to transfer knowledge by deterring us from including unnecessary information.
It is outside the scope of this article to discuss the ingredients of a great template because it will depend exclusively on your use case and the information that you are trying to convey.
You might need to spend some time strategically thinking through how the template should be designed. But, it is worth that upfront effort.
Ultimately, the best way to discuss a forecast with different stakeholders is to tell its story by modulating the level of details depending on the audience.