At last year’s AMEC awards, judges were disappointed at the overall standard in the category of social media evaluation. As we approach this year’s awards and with the industry having agreed new measurement principles, have we seen a great leap forward? Marino Donati reports

When a Facebook campaign to bring back defunct chocolate bar Wispa culminated with bannerwaving fans storming the stage at Glastonbury during Iggy Pop’s show three years ago, it was a surprising stunt.

When the accompanying online campaign actually succeeded in prompting confectionary giant Cadbury to relaunch the Wispa a year later, it was clear that the power of social media to influence corporate behaviour had reached a new level.

Yet, less than a year ago, as a judging panel of PR professionals, journalists and academics pored over the entries for the Association of measurement and evaluation of communication’s annual awards, they expressed collective disappointment that so few communicators had truly bought into social media evaluation.

It was clear then that the quality of social media evaluation and the availability of best practice examples to draw on had lagged behind monitoring of more traditional media.

However, the creation this summer of the Barcelona Principles, global PR measurement standards, is helping to focus minds on what makes good evaluation. US research group Altimeter and the Internet Advertising Bureau in the UK have also recently launched guidelines for measuring social media campaigns.

So as the AMEC communication effectiveness awards approach, are PRs and media monitoring companies and their clients getting better at measuring the impact of social media?

Many monitoring experts feel that client companies are gaining an increasingly sophisticated understanding of the importance of social media campaigns and how to evaluate them. This has been spurred on by more real world experience, better monitoring tools, improved accessibility and quality of data from social media portals.

Mark Rogers, chief executive of Market Sentinel, which analysed the Wispa relaunch, says: “I have more and more people showing quite a sophisticated understanding of social media and there are more examples and experience of what works and what won’t work. Things like Facebook campaigns are easier to monitor and the data is more easily accessible. There are new ways of tracking. If I was an ethical brand for example it’s easier now to measure the key things that people are saying they expect or are important from an ethical brand.

“Data sources are getting better, Wikipedia is more sophisticated and Twitter has been a great boon which was the big change because it lowered the bar for commentary. We are finding new ways of measuring things every day. There are now a lot of really good PR firms that get it now. They are beginning to think of it as something where they can steal a lot of business from marketing because it’s about understanding the context of your brand.”

According to some experts, the historical problem with social media evaluation was that collection of information was seen as an end in itself, the evaluation lacked context, was not carried out by the most appropriate people, and was not taken seriously at executive levels.

Chris Thomas worked on evaluating the 2008 launch of HTC’s G1 smartphone, the first to use Google’s Android mobile phone operation system, and promoted as the “iPhone killer”, when he was at consultancy The Conversation Group.

He says that the best social media evaluation makes sure that there is a solid methodology behind the evaluation, which uses properly differentiated analysis tools and is geared towards giving PR firms robust metrics for future use.

Thomas says: “I think that people are discovering that monitoring social media is not that different at its root from other types of media, but you just have to decide what information is important. People are getting to grips with how to home in on relevant audiences, and people are increasingly confident about understanding the implications of social media. So now, as well as new guidelines you can see big companies wanting to get more involved in the sector, with the likes of IBM buying text analytics company Coremetrics [in June].”

“We have a media issue forecasting tool which looks at what the issues that matter to your business are and how are they becoming more important. Who are the most important people in the blogosphere, what they are talking about and what is the impact?”

Thomas is now head of research at Media Measurement which worked on evaluating PR firm Borkowski’s campaign for Nissan’s launch of its Cube car at the beginning of the year. The campaign significantly leveraged social media and has been shortlisted for an AMEC award (see box).

Media Measurement had a list of goals for the media analysis including finding out what type of websites discussed the Nissan Cube, which press activities were most successful in generating conversation, what was the sentiment surrounding how the Cube was discussed, all with a view to providing guidelines to use on future campaigns. The more “human” approach to analysis, which looked at quality not just quantity of messages about the product, helped identified a nascent online community to be tapped into for future campaigns.

But how easy is it to show a direct impact of social media on a company in financial terms?

In the case of Wispa, Cadbury, whose keenly budgeted launch campaign had social media at its heart, has sold £86.5m of the bars since its relaunch. But Wispa brand manager Ross Farquhar says it is not as simple as that.

“It’s difficult to measure exactly the direct impact but that’s the same with other types of media,” he says. “You might say that Wispa would never have come back without social media but that’s probably too blunt a view and there were other factors involved: how many fans we had, how many people were talking about the brand, the sentiment and chatter levels were all monitored, which is similar to what you could do with an outdoor campaign.”

Generally, getting evaluation to show a direct impact of social media on profile, or the bottom line is difficult. Rogers says that expectations of what evaluation will reveal can be a problem but it is the role of evaluators to help educate clients.

“The challenge is that people expect it to be as measurable as click through rates and conversion rates,” he says. “But with social media there can be a conversation that has no impact on your sales. You have to understand which conversations you are interested in. When people say we want to monitor our brand in social media my question is ‘why and what are you looking to achieve by this?’ What doesn’t work is monitoring and then not doing anything about it.”

Echo Research uses innovations like its Echo Sonar tool to create a desktop application that gives clients a real time snapshot of the buzz surrounding its brand, but is always trying to develop new technologies.

Echo Research senior director Ben Lloyd says: “We have a media issue forecasting tool which looks at what the issues that matter to your business are and how are they becoming more important. Who are the most important people in the blogosphere, what they are talking about and what is the impact? Who are the players, what are they saying and what are the trends? We are also trying to stay one step ahead and we’re always looking at new ways to understand social media.”

But he too emphasises the need to work together with clients: “There’s no one size fits all. You have to work carefully with clients, it’s a collaboration and they all want different things.”

“The Barcelona Principles provide a really solid platform, standardising creating real value and focussing on outcomes, and it gives it more credibility to the whole thing.”

Philip Lynch, director at Kantar Media Intelligence says that understanding of social media and monitoring in the US is still more advanced that in the UK.

“In the US, they are more open to the use of social media tools and embedding them in their business.” he says. “But they have had a longer run at it. In the UK we’re past the ‘awareness’ stage now and onto the ‘discovery’ phase. Two years ago there wasn’t even a consensus that blogs were important. Traditional media has been unchanged for decades. With social media, the range and diversity of platforms is expanding and growing at such a rate, it’s a challenge.”

He too, says that progress in the UK and Europe is gaining momentum. “The Barcelona Principles are good for supporting the idea that PR is not a deliverable in its own right and focusing on outcomes. You can’t talk about AVE without understanding what your strategic drivers are and how it changes customer opinion and thought.”

Many evaluation operators agree on the increasing status of social media evaluation in the boardroom.

Thomas says: “It’s making it easier for communications to be listened to by senior managers than public relations has been able to do. Social media has the ability to tap into more and with a far finer grain.”

Lynch agrees: “With social media you can measure everything so in a way it makes its contribution more transparent. You can measure visitors to website, opinions and much more, so you can do more to validate it.”

“Social media offers communications operators a great opportunity to open new channels of engagement and a much clearer way of demonstrating the value of communications.” 

This year’s AMEC shortlist for social media evaluation

• Echo Research was commissioned by Syngenta AG, a global agribusiness company, to draw up a ‘Contacts Book’ of leading influencers of sentiment within the online community

• As Nissan launched its new car the Leaf, Media Measurement was commissioned to design a research programme that would measure and evaluate the success of the product launch

• Kantar Media’s monitoring and evaluation provided German retailer Tchibo with a first impression of what was being said and who was talking about the brand online

• Kraft Foods’ Shopper Insights Group and Cymfony, a Kantar Media company, partnered to understand how consumers were using social media and how it affected their grocery shopping

• For the launch of the Nissan Cube, PR firm Borkowski sought to identify and engage influencers in the design community. It approached Media Measurement to provide evaluation, metrics and insight • Mashreq Bank commissioned Media Watch’s social media division ‘SocialEyez’ to research social media forums to create positive sentiment growth