A recent “social media ROI” report from the University of Massachusetts Dartmouth talks about how college and university admissions offices view and use social media. The report summary trumpets the findings as an indication that social media is significantly altering the way colleges and universities recruit students. The report (or at least the headline) even goes so far as to trumpet social media tools as “a Game Changer.” The headline and report also tout the return on investment (ROI) social media tools bring to higher education.
The report is filled with some good information, but I couldn’t really pinpoint any actual ROI indicators.
I’m no ROI expert, but it seems to me that the authors of this report have confused cost reductions with ROI. For instance, the report points out that institutions surveyed “report 33% less spent on printing, 24% less spent on newspaper ads and 17% less spent on radio and TV ads.”
But reducing costs, or reducing your media spend, isn’t the same thing as investing. Cutting costs does not equal increased investment.
One uncritical report about this research (posted on businessgrow.com) attracted the interest of several higher ed folks, including three very bright guys: Andrew Gossen of Cornell (@agossen), higher ed consultant Michael Stoner (@mStonerblog) and Mike Petroff, a digital strategist at Harvard (@mikepetroff). Gossen and Stoner took to Twitter to chat about the findings with a more skeptical view. But of course, there’s only so much you can discuss 140 characters at a time.
So Stoner visited businessgrow.com to share his thoughts on the subject, then expanded them in this blog post on his own site. (Tracy Playle of Pickle Jar Communications — @picklejar — also does a fine job of dissecting the Massachusetts Dartmouth study.) Both Stoner’s comment and his blog post are worth reading, as he eloquently discusses the nuances of higher education that the Massachusetts Dartmouth study seems to overlook.
The money quote, however, comes from Stoner’s comment on businessgrow.com blog:
[T]he story is more complicated than The University of Massachusetts Dartmouth Center for Marketing Research data indicate.
For example, Noel-Levitz, an enrollment and admissions consulting firm, does a yearly study of prospective students to see what they say about various channels (in 2012, 2000 students, 51% high school seniors). This year, 72% said they used “brochures/mail” to research colleges. When asked the best ways to learn about academic programs (the most important content to them), 71% said brochures were best; 38% cited social media as the best source. Only 10% said that social media was the best source of info for financial aid and 35% for information about the campus location and community. So while they do use Facebook (and other social channels), these aren’t the only, or even the primary, sources of information.
Indeed, this confirms my observation that adults continuously over-estimate the appetite of teens for whatever it is adults we think they should be interested in. We’ve seen from our own experience that teens love printed pieces — they just aren’t impressed with the kind of glossy viewbooks that most admission offices send out.
Interestingly, just a week after that Massachusetts Dartmouth report came out, businessgrow.com’s report about another study — the Social Habit Research Project — provided the perfect definition of ROI for businesses. It’ comes from social media marketer Rhonda Hurwitz, and it’s a definition that can be modified to fit the realm of higher education:
For me, it always comes down to figuring out how to connect social media usage or activity to revenue. I would ask “have you ever bought a product or service due to a social media interaction” … “have you ever recommended a product or service to others via social media?”
So, for higher education, perhaps the ROI questions should be:
- Has a prospective student ever decided to attend a college due to a social media interaction?
- Has a prospective donor ever given to his or her alma mater due to a social media interaction?
- Has a current student ever recommended a college or university to a prospect via social media?
And perhaps we should be asking them of our customers (students, alumni, etc.) rather than those in charge of delivering the services, recruiting students or raising money. Maybe then we would gain a better sense of the true ROI of our social media activities.
* Disclaimer: I am no ROI guru. Far from it. But I agree with Stoner when he says, in his recent blog post on this topic, that “Measuring ROI is complicated, especially in higher ed.” It’s messy, and it’s a hassle. And that’s probably why we don’t do much of it.
Image: Clara Peller as the “Where’s the Beef?” lady from Wendy’s classic early 1980s ad campaign.
Higher Ed Marketing 
Related, for those interested in this topic: What does ROI mean to you?
Very good recap of the points in discussion, Andy!
As I mentioned to Michael over Twitter, this is funny how the more things changed the more they stay the same when it comes to this specific research. Last year, I wrote a very similar post in relation to the results of the study conducted by Dr. Barnes.
http://collegewebeditor.com/blog/index.php/archives/2011/08/09/why-higher-ed-is-not-there-yet-with-social-media-marketing/
The sample size and the methodology are both very valid, but I can’t help think that the questions that were asked could have done much more to help us understand better the state of social media in admissions offices. Noel Levitz has a very similar study done with admissions offices that is definitely worth checking as well.
Anyway, something else to keep in mind is the fact that social media is just a part of the equation – even when it comes to ROI and measurement. If you were just going to use your 3 proposed questions to measure the ROI, it would probably be very low.
As you know, students don’t make the decision to attend a school just because of the interactions they had on social media nor do they make this important decision after browsing a brochure. Institutions have embraced integrated marketing because students will be influenced by multiple channels as they walk them through the final decision — or not.
As far as the NL Study results are concerned, I was actually surprised to see that brochures were such an important channel for students – something that was actually in total contradiction with the findings of the focus group put together by Stamats and recorded by Higher Ed Live: TEEN Talk.
When I asked Stephanie Geyer from NL about it last week at eduWeb, she explained that the question that led to this answer was actually pertaining to what channels/media students prefer to make their initial list of possible schools. Given the amount of options out there, it is definitely easier to use a physical artifact (the brochure) to do the initial filtering. However, when the number of schools becomes more manageable, then other media/channels come into play.
Hi Karine! Thanks for weighing in and for sharing your thoughts — and your post from last August. Yes, all of this recent discussion about ROI certainly does sound familiar, doesn’t it? You may recall that I used your August 2011 post as a springboard for a post of my own that focused not on ROI but on the role of key performance indicators (KPIs) to measure effectiveness. I thought about referring back to it in my post, but decided not to open that can of worms yet.
As you point out, social media is but one of many factors that come in to play in the decision-making process for students (and their families) when it comes to selecting a college. It is also one of many factors related to other customer relations functions (alumni relations, public relations, student retention, etc.).
The Massachusetts Dartmouth surveys, both this year’s and last year’s, also overlook other important factors related to social media usage in higher education. For many, social media often plays more of a customer service function than a recruitment or fundraising function, and in that regard an entirely different set of metrics may be called for. Perhaps we should look at social media’s role in retention, alumni sentiment and such matters.
Obviously, as you stated last year and Michael Stoner states this year, we’re not there yet in terms of coming up with an effective measuring stick. We haven’t even figured out what we should measure! But if we take the conventional approaches to calculating ROI (as I mention above, citing Rhonda Hurwitz’s comment) it all goes back to things that can be measured, whether those things are products, students or dollars.
You point out that the ROI “would probably be very low” if we used my three proposed questions. That’s precisely the point. And precisely why no good social media manager would ever suggest such a thing. Who wants to be held accountable for such low numbers?
Glad to see the higher ed industry is revisiting the social media ROI discussion again. With so much talk about Facebook, Twitter, Pinterest etc. most people forgot to stop and ask why we’re doing any of this, or if it is even making an impact. From my perspective (biased since I work for a vendor) traditional social media channels will not provide the necessary student services because they aren’t designed with an institutional use case in mind.
Although I’m all for people experimenting with tactics and solutions, I’m a bit sick of all the fluff with little to no science to back it up. To follow up on Karine’s comment re: brochures –
“I was actually surprised to see that brochures were such an important channel for students – something that was actually in total contradiction with the findings of the focus group put together by Stamats and recorded by Higher Ed Live: TEEN Talk.” – From what I remember from the TEENS talk is that the students did like unique, authentic, and eye-catching print materials, however they didn’t like cheesy stock photo’s and lot’s of texts.
I’m glad you chimed in, Brandon. I think I agree with your assessment that traditional social media channels (is it too soon to call social media “traditional”? Maybe not) aren’t designed for institutional use. We’ve been trying to shoehorn our brands into that tight fit, and some social media platforms have tried to accommodate us somewhat.