Analytics and metrics: It’s all in the interpretation

The backdrop to this week’s post is a paper by Germann et al. (2013): Performance implications of deploying marketing analytics. The paper gives a sobering narrative of the use (or not) of analytics in marketing and the impact this can have on the improvement of the overall decision making process; not just for marketing practitioners but also for customers / audiences.

A couple of stand outs for me from the article were that in a recent study of 587 executives from large International companies, only 10% of the companies regularly deployed marketing analysis. The biggest push back on using analytics was that it slowed down the business and caused “analysis paralysis” (which is the inability to make decisions due to having to wait for data). (Peters and Waterman 1982)

The term “analysis paralysis” for me takes on a completely different form. It is not so much about not being able to make a decision due to waiting on data but similar to consumer choice paralysis, it is more about not being able to make a decision because of having too much information. Therefore, the bigger issue is not what to report on, but finding the balance between having the right data to tell the story, versus the management’s ability to actually process the information and then make decisions.

Image source Lab Bratz – It’s all in the way you look at it

One of my key deliverables at work is to compile reports on what was happening across our social media landscape. High level results (vanity metrics) provide the management team with a quick snap shot of how we are travelling. But what do these figures even mean, and are they even useful?

Understanding on what to report, and how to report it, is key here. All communication, including Social Media campaigns should be based around some form of strategy. As Avinash Kaushik notes in his article Digital Marketing and Measurement Tool, the strategy should contain clear goals and objectives, so the reporting should reflect and align with this. Some reporting you can cover with vanity metrics but in most cases you have to dig deeper, much deeper to be able to fully understand and convey the results.

Simple charts, high level results and key metrics will help get the message across much better than complicated spreadsheets full of data and often a simple infographic can be used as a summary. Adding a narrative to help explain the results is very important and if your report has a lot of terms that people may not understand (such as impressions, mentions etc.) a simple glossary can work wonders.

Social Media Analytics

To help explain how Social Media analytics and metrics works I have broken it down to three specific levels of information.

  • High-Level Reporting (what the result was)
  • Analysis (establishing what is happening in the data)
  • Interpretation (determining what the results actually mean and any implications)

High-Level Reporting (Vanity metrics)

This is high-level information that is provided in the form of metrics for specific attributes such as impressions, engagement, number of likes, shares, retweets, video views etc. Management love vanity metrics. They are easy to understand and it provides some indication that “something” is happening. Although, without additional information they can be very misleading. For example, “the results for Facebook comments this week are 200% better than last week”. What a great result, right?? Well that really depends on whether everyone is commenting on the fact that they think your brand sucks or not, or maybe the week before was just the worst ever and this one isn’t much better, it’s just better than last week. It’s easy to get caught up in the big numbers and positive results without really understanding what they mean. That’s where analysis comes in.

The man has spoken!!!

Analysis (examining the data)

Breaking down the metrics to look at things such as; trends over time, performance against benchmarks, performance against competition, platform comparison and so on. In simple terms this is looking at the data at a deeper level to identify things that are happening within the data. Analysis is an important step in identifying “what” is happening to your Social Media activity but often lacks the “why” and this is where the interpretation of the results come in.

Positively trending charts make people happy, but what do they actually mean?

Interpretation (deep level analysis of data and narrative explaining the results)

Data interpretation is a specific skill set that is seldom used or sufficiently resourced yet is one of the most critical stages of the whole data analysis process. It is much more than just looking at the data and picking trends or movement in the data, it is about understanding the results and their implications. It is about identifying not only what we know from the data but also what we don’t know and then filling that gap with factual data, knowledge and experience to identify opportunities and then communicate these findings in the form of a narrative to aid the decision making process.

In a nutshell

In the section where I work we don’t have a dedicated resource for marketing analysis and interpretation. Yet the interpretation stage is where a good analyst will start questioning the results, and will prove (or disprove) theory or assumptions to provide an evidence based narrative. Without interpretation of the results by someone that understands the data and the situation it represents, it is left to the audience to work out what’s happening and that could be a recipe for disaster.

For example, during a recent campaign an announcement was made that a video that we had released on social media was the most successful yet. This statement was made entirely based on vanity stats (the number of times the video was viewed). The data was analysed further and the resulting trend did in fact show the number of video views was higher than any other video in the previous six month period. High-five everyone, break out the champagne!!!

When it came to interpret the results however this wasn’t the case. The objective of the campaign was to encourage viewers to watch the video until the end. The previous campaign had been run on YouTube and the most recent one was an embedded video on Facebook. It also transpired that the number of video views also included auto play on Facebook and a deeper dig in to Facebook insights showed that 95% of those that viewed the video did so for less than 10 secs. In terms of the objective, this was actually one of the most underperforming campaigns to date and highlights why interpretation of the results is such a critical step. Without it, the next campaign would have just followed the same course.

Data, it is all in the interpretation. Reducing cheese intake saves lives!!

6 thoughts on “Analytics and metrics: It’s all in the interpretation

  1. Great blog post Keith, very interesting to see what your Facebook videos really meant, how unfortunate but I’m sure you will improve it for your next one!
    Until recently I worked at a market research agency, and you are spot on with the need to condense and aggregate high volumes of data into concise and practical charts and key metrics (which varies depending on what the client is interested in based on their objectives). Because we were an agency, it was our responsibility to deduce the results into a narrated ‘story’ to explain the results. We tracked a lot of campaigns/ads and used an external company called Eye Square for social media tracking. I also agree with you in that you must delve deeper into the data, regardless of the superficial tip-of-the-iceberg results, as many times we found certain ads to be within top percentiles of recognition and branding (obviously very important metrics), but further investigations into the data would reveal high levels of irritation from viewers!
    It was also very disappointing to watch some clients ignore the recommendations we provided based on data, one client went as far as to ask us to repeat certain metrics and measures by cutting specific lists of competitors in questionnaires and for the results to only include ads placed in certain channels in order to achieve a higher key metric used in their objectives! This just goes to show the poor use of rich data and a lack of understanding in what data should be used for and how it may improve a strategy. It seems some companies really absorb all the knowledge and data they can get their hands on, while for others it is a just another process they can tick off a checklist and show the boss how “well” they are performing.

    1. Hi Vasylika

      Yeah that sucks when clients ask you to manipulate the answers so the results appear better than they are. I’ve seen this happen a fair few times. I had a report recently that one of our managers didn’t want to forward on to their boss because all the stats were trending down. They just came up with some impressive sounding numbers and we’re going to send that through instead. Thankfully someone else pushed for it to go through as it was along with the recommendations and opportunities that the data identified. Ironically the reason for the downward trend wasn’t because of anything they were actually doing wrong, it was a change to the platform reach that caused the issue. They just freaked out I suspect because they didn’t understand the issue and were too focussed on the vanity stats.

  2. A very informative blog Keith. I really like your breakdown of the SM analytics into 3 levels.

    You gave a made a very convincing argument for using analytics and data interpretation. I don’t think anyone can deny that, yet why is it companies are still not investing in this? Even your company doesn’t have a dedicated person. How do you think we can persuade management of the value and importance of marketing analytics? Is it simply a cost issue?

    For me what stood out the most in the article was the need for a supportive analytics culture – first time I’ve heard of culture described this way and I’m from a HR background so culture is high on my radar. It seems this culture is lacking at your workplace and I hope through your work in providing evidence and impact you’ll be able to shift this mindset.

    1. Hi Anh, thanks for the feedback and your contribution to the topic. I think in our case it comes down to not understanding the value that marketing analytics can add to the business and budgets. The decision was made to focus on getting messages out of the door rather than establishing if what we are doing is working or not.

      It is very frustrating especially when you can clearly see things are not getting the cut through that we want but there are no resources available to work out why.

      We had a two day workshop last week and I must have raised this a dozen times, how important it is and why we need it. It was acknowledged that it is important but we just don’t have the budget to dedicate a member of staff to it. So we fire blindly and hope we hit the target.

Comments are closed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑