ROI in social media: A look at the arguments

Abstract

Return on Investment (ROI) has become the Holy Grail of social media. Marketers are being squeezed between admonishments to participate in the vast new online communications available to them and demands to justify the cost using conventional advertising metrics. New ‘ROI calculators’ are being created almost as fast as new social networking sites – then just as quickly being dismissed as being unworkable. In this article, Tia Fisher of eModeration takes a long view of the current state of ROI in social media, and examines the arguments for and against attempting to use any kind of metric to justify involvement in a social media program.

INTRODUCTION

With the publication last month of the Internet Advertising Bureau's (IAB) ‘Social Media Ad Metrics Definitions,’¹ it seemed a good moment to have a look at some of the controversy around the measurement of ROI in social media. Examination of the IAB's publication will follow later, but in the meantime, let's take a look at the background into which it appeared.

The ROI within social media has long been a bone of contention, and seems likely to become ever more so, with the equally lightning spread of both social media use and savage budget cuts. In a tightening economy, businesses need to make sure that they’re getting a return on their marketing investment. Are they in the right places? Doing the right things? With the right people? And how can they tell? To get an idea about the current raging debate, take a look at just one example post from MetricsMan, a blog about ‘The Fast Changing World of Social Media and Public Relations Research and Measurement,’ in which he states, ‘Are you are seeing the accountability bar being raised this year? In my corner of the world, the volume of conversation about social media ROI is high and accelerating. Unfortunately much of the conversation has been misinformed and misguided. It seems like every week brings another post attempting to reinvent the acronym or the meaning – ROI really means Return on Influence, or Return on Engagement is the new ROI, and on and on. There is another group of online Zen Masters who would have you believe social media ROI is old school thinking and not in tune with social media Zeitgeist.’²

Quoting David Alston in MarketingProfs, ‘The discussion of ROI has focused mostly on the search for the Holy Grail of a metric, but adapting traditional metrics to fit social media would be akin to sticking a square peg in a round hole.’³ ‘Inability to measure ROI’ was named by marketers as one of the most significant barriers to the adoption of social media tactics by their organization from a poll quoted by MarketingSherpa: ‘Marketers are under constant pressure to measure everything they do. The result is often a default to tactics that are more easily and accurately measureable, regardless of their effectiveness. This is especially true in social media marketing which often requires qualitative measurement rather than quantitative metrics that are more familiar to online marketers.’⁴

IT’S BECOMING EASIER TO SAY WHY COMPANIES SHOULD BE ENTERING THE SOCIAL MEDIA ARENA

Social media is where your actual and potential customers are interacting, and it shapes how they think. Universal McCann's Social Media Research Wave 3 research report (published in Spring 2008) looked at 17 000 Internet users in 29 countries. According to this study, social media can have a dramatic impact on your brand's reputation: ‘34% post opinions about products and brands on their blog and 36% think more positively about companies that have blogs.’⁵

‘In the past year, traffic to Blogger and Wordpress went up 44% and 50% respectively, while traffic to CNN, MSNBC and the New York Times went up an average of just 17%’ (Augie Ray, quoting figures derived from compete.com).⁶

Your existing customers are valuable resources. A recent study in 2008 by DEI Worldwide on the Impact of Social Media on Customer Behavior was able to provide the following interesting statistics:

• 70 per cent of consumers have visited social media sites to get information;

• 49 per cent of these consumers made a purchase decision based on the information they found through social media sites;

• 60 per cent of people in the study said they are likely to use social media sites to pass along information to other online;

• 45 per cent of people who searched for information via social media sites engaged in word of mouth compared to 36 per cent who found information on a company or news site.⁷

A customer is worth far more than their initial spend with your company: you need to factor in future purchases and the influence they may have through social media. Quoting Jacob Morgan in MarketingProfs, ‘If we are going to start looking at social media in terms of quantifiable numbers then we need to start understanding how much our customers are worth, and this means looking at more than just how much a customer spends. A customer's value is not equal to how much they spend at your store. It's far more.’ The following are a few questions he thinks organizations need to be able to answer:

• How much is a customer worth?

• How much does the average customer spend per transaction?

• Is a customer more likely to purchase a product after some sort of social media interaction?

• How much is it worth to convert an unhappy customer to a happy customer?

• On average how many people does a happy customer tell about a product or service?

• How much can an unhappy customer hurt you?⁸

The other reason for participation in social media is that your competitors are already doing it. The University of Massachusetts Dartmouth Center for Marketing Research recently released the results on the usage of social media in the Inc. 500: ‘Just over one quarter of the Inc. 500 reported social media was very important to their business/marketing strategy in 2007. That number has increased to 44% just one year later.’⁹ To quote Alston again, ‘More importantly, social media isn’t private. It's out there boldly for the entire Web to see (including the almighty Google), and a company's absence is only highlighted by such accessibility.’²

BUT ARE THE OLD METRICS OF ONLINE ROI STILL APPLICABLE TO SOCIAL MEDIA?

Measuring the impact of online advertising used to be relatively easy. It was all about analytics: Unique Visitors, Page Views, Cost per Clicks – safe, measurable, defined metrics. But those engaged in social media must now attempt a way of measuring not just the online advertising within social media, but the framework surrounding that advertising: a difficulty akin to playing three dimensional chess with moving chess pieces.

This is where the IAB's ‘Social Media Ad Metrics Definitions’ have gone some way to filling the gap. They provide a clear framework by which advertisers can gauge ad effectiveness across three main social media: social media sites, blogs and widgets/social media applications. The document divides social media into three distinct categories and defines the metrics specific to each type:

• social media sites;

• blogs;

• widgets and social media applications.¹

‘Evaluating the most important measurement terms will help marketers, agencies and publishers quantify the value added by consumers as they distribute the content throughout their personal networks, one of the defining characteristics of the platform’ (IAB press release).¹⁰

What this means in effect is that the IAB has clearly defined each platform, and then provided metrics by which the effectiveness of each might be measured when planning a campaign. In other words, how to judge where to place your ad in order to obtain the best ROI. The metrics include the following:

• Social media sites: Unique Visitors, Cost per unique visitor, Page views, Visits, Return Visits, Interaction rate, Time spent, Video installs, Relevant actions taken.

• Blogs: Conversation size (number of sites, links and reach of a conversation whose content includes conversation phrases relevant to the client), Site relevance (Conversation density, Author credibility, Content freshness and relevance).

• Widgets and Social Media Applications: Installs (number of applications), Active Users, Audience Profile, Unique User Reach, Growth, Influence, Installs (number installed per user).

The IAB's definitions have so far met a mixed response. Some had high praise for the idea, such as Catherine P. Taylor, on the Social Media Insider: ‘In other words, compared to old-time metrics like reach, frequency and the click-through, these metrics are deep, not only measuring whether people are engaged, but how they are engaging. It's like being able to measure the temperature with a thermometer rather than opening the front door and declaring it either hot or cold […] imagine that you’re an advertiser who sorely needs to understand social media. Then imagine yourself suddenly finding that you can not only monitor discussion around a certain topic near and dear to your brand but that you can also mention the number of people talking about it and their level of passion. Suddenly, social media goes from a huge, indefinable blob of conversations into something that has contours around which you can engage, plan and buy. That's huge.’¹¹

However, others weren’t so impressed, picking up on the fact that the qualitative aspect of social media measurement was sidestepped entirely. Augie Ray was one such, writing on his blog, ‘Perhaps one of the most concerning aspects of this report is that a reader might get the idea that every action is equal. The report suggests that “Comments posted” are worth measuring, but it says nothing whatsoever about sentiment within those comments. In the entire report, the following words do not appear even a single time: “Sentiment,” “Attitude,” “Rating,” “Positive,” and “Net Promoter Score.” Apparently the IAB thinks that all comments should be tabulated in aggregate, regardless of whether they are disparaging or complimentary.’¹²

In a further blog post in The Customer Collective, Augie puts forward his own four ‘dimensions’ by which clients should analyze a blog's suitability for their brand: ‘Evaluating which blogs are right for your brand requires consideration of factors that include reach, relevance, and credibility. Reach refers to the audience for a blog, including total number of readers, subscribers, and visits. Relevance concerns the extent to which a blog's content and audience is appropriate to the brand; this can be determined by evaluating the number and frequency of blog posts on a given topic, the blog's organic search results for desired terms, and the percentage of readers or visitors that match desired demographics. Credibility, the third dimension that determines if a given blog is right for a brand's outreach or sponsorship, is perhaps the most difficult to ascertain. Reach and Relevance can be determined by quantitative measures, but Credibility is more qualitative’.⁵

According to Ray, blog Credibility has dimensions, including Independence, Affinity and Transparency. ‘These four attributes – Independence, Affinity, Transparency and Presentation – are important determinants of a blog's credibility. These are not the only factors that enhance or diminish credibility, but they are among the most important. Other factors that may be important, depending on the blog and brand, include Blog Attribution (the more frequently sites and blogs attribute their sources, the more credible they will be received [to be]) and Blog Reputation (presence of awards, Diggs, etc.).’⁵

IS THAT, THEN, A FAIR MEASUREMENT OF SOCIAL MEDIA?

I would disagree, because nothing I’ve so far cited has really told me about anything other than new media versions of traditional media campaigns: where to place your ad, sponsored widget or sponsored conversation, and how to judge its effectiveness. How about real social media, finding out how your forum is contributing to your reduction in online support, how customer reviews are affecting sales path, whether your 30 second TV spot crowdsourced in an innovative user generated content (UGC) campaign really made your audience feel engaged? It's really not all about online advertising anymore. It's about what else is happening in the social networks, very likely somewhere your company isn’t. ‘Web Analytics provide traditional attributes of page views, unique site visitors, etc. This alone isn’t adequate anymore because customers are talking to each other (and much of it is not on your site)’ says social media expert Connie Benson.¹³ It's about engagement with your customers, your potential customers, and your critics, at every level of social interaction that modern communication has to offer. Therefore, while we shouldn’t forget about these useful new definitions of measurement for your paid-for advertisement, we need to look beyond that, into the buzz, the opinions, voices and experiences that people are sharing about your brand.

Jason Falls in Social Media Explorer feels that ‘Perhaps we shouldn’t measure social media ROI in the first place. The reason? The core reason social media programs are successful is because they’re about people, not money. Look at the social media buzz words – community, conversation, dialog, sharing – all of them are people-centric, consumer-centric. Social media isn’t about sales. It isn’t about market share. It isn’t about profit margins. It's not company or brand-centric.’¹⁴

BUT SOMETHING MUST BE MEASURED, IN ORDER TO GAUGE EFFECTIVENESS

The ubiquitous Jeremiah Owyang provided an excellent guide (still relevant) on how to measure your social media program as far back as 2006. He starts off with the basics: if you haven’t got a goal, then you can’t measure against it. Before you start, work out what your social media program is trying to do. Increase sales? Listen to customer feedback? Drive awareness? (To which I would add that you need to put your benchmarks in place here in order to judge success: Increase sales by how much? What are you going to do with that feedback? What percentage of awareness among your target audience are you aiming at?) Then build in measurement of this before you launch – not as an afterthought – as part of the process.

He guards against trying to measure it all – it can lead to ‘Analysis Paralysis,’ and don’t we know it! ‘Shel Israel suggests we don’t measure the value of a phone anymore – we know the value. At some point, business blogging will be like email – everywhere.’ Importantly, he exhorts us to monitor alerts as they happen (versus only reading reports): ‘A negative meme, an exploding battery could shatter your brand: are you watching in real time?’ And finally, he helpfully provides a list of attributes to measure:

• activity (web Analytics of blog or site);

• tone (sentiment);

• velocity (spread over time, URLs, trackbacks);

• attention (duration on site);

• participation (comments, trackbacks);

• many qualitative attributes (comments, what did they say, what did they mean).

‘You may never measure the same way for each program, the goals of each program will change the method in which you measure. The attributes will stay the same, but you’ll just use them in different ways to create a new report […] A bunch of facts and figures are not good enough to tell the story. What happened, where are we now, and where are we going? Did more folks come into the forum? How? And why? What are they doing now? […] What really matters are the opinions, voices, and experiences that people are sharing. The written anecdote that persuades prospects to customers (or the other way around) is very important. Be sure to monitor and use these in your reports.’¹⁵

Connie Benson is a big fan of Jeremiah's, and based her post on Social Media Measurement on a webinar he gave in 2008. She cites another list of attributes:

‘Attributes that need measurement:

• Attention=Time on website (sometimes it's not yours)

• Interaction=Click, Comment, or Embed

• Conversation Index: Ratio between blog posts and comments+trackbacks

• Velocity^*=Distance/Time (Rate of how fast your message is travelling in a given time) Jeremiah defines Velocity with examples

• Sentiment^*=Tone, opinion

• Qualitative^*=What did they say?

• Impacts^*=What did they do? (influencers)’

^*Brand Monitoring Solutions – these are helpful with the last four above.

‘Tracking the tone, perception and nuances within online conversations is getting easier. There are a number of solutions available that monitor discussion of your brand online. They offer options to track and create reports on frequency, influence and sentiment.’¹²

Connie recommends using Techrigy SM2 for social media monitoring: ‘It offers an efficient way to measure brand presence, brand perception, opinion and tone. And it also identifies the conversations that the brand should be engaging with.’ (Techrigy SM2 is also featured in a review of a massive 67 social media/web/reputation management tools and sites recently provided by Marc Meyer at Social Media Today: a useful reference.)

Connie provides a great guide for the community manager's role (versus the marketer's), listing the new places to monitor/measure (social bookmarking sites, social networking sites, aggregators, micromedia, forums, competitor sites), a methodology for finding new communities in which to listen and participate, and, finally, examples of things you can measure:

• 1 listen, understand the conversation, then participate;

• 2 measure the number of conversations;

• 3 monitor the percentage increase of conversations over time;

• 4 measure the reduced buying cycle and reduce support costs by encouraging self-support;

• 5 increased sales due to increased customer satisfaction in product due to involving them in product development cycle;

• 6 increased efficiency in developing products due to customer feedback at various stages;

• 7 minimize brand damage by responding quickly to customer's concerns online.

Connie also recommends that community managers provide a report on a monthly basis, including the following:

• ongoing definition of objectives;

• web analytics;

• interaction – Trends in members, topics, discovery of new communities;

• qualitative quotes – Helpful for feedback and marketing;

• recommendations – Based on interactions with the customers;

• benchmark based on previous report.¹²

WHERE DO THE MARKETERS CURRENTLY STAND ON THE ROI QUESTION?

Marketers are divided, I’d say, into three main camps.

The Deniers: What, really, is this obsession with ROI? This is the group who are most resistant to any attempt to measure social media; possibly because past attempts have been so clumsy and obviously missing the point.

The Definers: Never mind the community stuff. Just give me the click through rate. Please. IAB's new metrics were made for this group. They are the ones who’ll be the best audience for the new definitions, and are likely to be hugely comforted by a large benchmark and a level playing field upon which to chalk it.

The Dedicated: It's not a perfect science, and it's a lot of work. But we’re getting there. This group are realising the importance of using every available system of measurement we have: web analytics, buzz monitoring, community management listening – the whole gamut – to try to bring in as near to a 360 degree picture of people's reactions to and interactions with, a brand, as is currently possible.

I’ll leave you with a splendid quote from Jeremiah Owyang: ‘Social media measurement is like driving a modern car. You may have a dashboard with all the lights, toggles, gauges, and metrics, but remember, the most important piece of data to have in front of you is the GPS screen. The GPS screen indicates where you want to go (your objective), where you are now, and how to get there.’¹⁶