The problem with alumni engagement metrics

What university doesn’t want to measure the engagement of their alumni? Not many if the latest International CASE Alumni Relations Survey (ICARS) results are anything to go by. But why are we measuring it? And what makes it so challenging? Here’s my take on the subject and some ideas for taking alumni metrics to the next level.

Alumni engagement is…

At its most basic level, alumni engagement is generally taken to mean participation and involvement in the university beyond graduation. When advancement professionals talk about engagement we’re usually meaning participation in alumni relations, volunteering and giving programmes – although involvement in any element of university life is usually desirable (if we know it’s happening!).

How it’s measured

14804061203_5ba074687d_h

Most institutions in Europe who responded to ICARS are tracking the number of alumni who are participating in activities and programmes that are run by the development office. Each way of participating is either reported in absolute numbers (800 volunteered as an e-mentor) or measured as a percentage of the whole alumni population (10% attended an event). Some have combined these measures, devising scales or scores for engagement where alumni participating in multiple ways (or in more valuable ways) are at the top or ‘most engaged’, and those that are more passive are at the bottom or ‘least engaged’.

Several institutions are also taking into account whether alumni are getting in touch with the university of their own accord and whether they have up-to-date contact details.

Here are a few of the common ways that alumni are ‘participating’ or showing engagement according to UK higher education institutions:

  • Receiving, opening and clicking-through emails
  • Following or interacting on social media
  • Returning contact detail update forms (online or paper) and completing surveys
  • Attending events
  • Contributing time, skills, networks
  • Contributing philanthropic donations
  • Being a member of a group

Why it’s important

We have a fundamental need in any role or business to measure and quantify the impact our hard work is having. Measuring alumni engagement may show us that our events have been well-targeted and attractive (high % event attendance) and that our e-newsletters have been good quality (excellent open rates and click-throughs). It’s a behaviour-based feedback loop that helps us reflect on our performance.

The engagement to contribution theory

Industry wisdom puts forward that participation is important to measure because it’s an indicator of affinity – and it’s a good use of our time to try to increase participation in order to grow (or identify) the number of alumni who feel warmly towards the university.

Many will go on to say that alumni affinity is important to build in order to try to increase the likelihood of alumni taking further action to help the mission of the university. These desirable actions are too numerous to list here, suffice to say ongoing volunteer contributions and philanthropic donations are only the most obvious.

If I were to boil it down rather crudely, it looks a bit like this:

paticipation-to-affinity-to-contributions

Quantifying reach

Measuring engagement in various programmes (particularly to an individual alumna level) helps to quantify the ‘reach’ of the university’s development office. It’s impressive to say that the office contacted over 100,000 alumni this year, and perhaps even more impressive to say that 30,000 were in touch with us, donated or attended an event. In other words, it’s a powerful method of showing senior management in the institution how industrious we are and how receptive our audience is.

The problem with measuring engagement

Like any measure of performance, alumni engagement has its limitations and is problematic when considered in isolation of other measures.

Participation for participation’s sake

If I were to encourage a team to increase engagement, would we not just increase activity, providing more and more opportunities to alumni to get in touch? If participation was incentivised without balancing it with measures of return on investment and strategic alignment or quality, it could be that ‘more means more’ is the philosophy this metric inspires.

If you had the budget and manpower there is nothing wrong with doing more, it’s just difficult to keep on justifying additional spend based on volume and reach alone. To be certain your new events and communications are a wise decision, you’ll no doubt need to be able to connect your activities to building affinity (and perhaps even further contributions from alumni).

If action means engagement, what does inaction mean?

The best way of describing the limitations of measuring alumni engagement is to consider it from the perspective of the alumna. As an alum, if I never open your emails, am I disengaged? According to the most common methods of measurement, yes – because I’m not participating. But do I feel affinity towards the university? Well that’s a tough one. You’re unable to know because I’m not taking any visible action that tells you so.

The problem is that large numbers of UK university alumni fall into the lower engagement categories – the unresponsive or passive groups. A classification of ‘disengaged’ has led some universities to exclude these individuals from alumni magazine mailings or calling programmes, writing them off as ‘too hard to reach’ or ‘unlikely to give’ – and depriving them of most opportunities to assert their ‘engagement’ in the future. Most worryingly, by taking these actions then institutions may be allowing any affinity these alumni felt to begin to fade.

Using scales or ladders of engagement

Lastly, the biggest question mark I have next to engagement metrics relates to the use of behaviour scales. The challenge with these scales is at best they show a snapshot of the proportions of the community that the institution is engaging (great for showing reach and who your super-committed alumni are!), and at worst they can encourage teams to make an assumption that one sign of engagement naturally precedes another i.e. each step is a ‘journey of engagement’ that alumni will follow with appropriate encouragement.

The ladder of engagement philosophy may therefore give rise to less effective fundraising or volunteer recruitment strategies. For example, teams might spend time persuading e-newsletter openers to attend an event in order to make them more likely to donate – where resource could arguably be better spent identifying and approaching people who are likely to donate based on their demographics and charitable giving history (i.e. using a focused predictive model). The problem isn’t so much the scale itself, it’s how it might be used to influence management decisions.

If affinity is what’s important then…

I keep coming back to – what do we really want to influence and measure? Are we trying to reach and interact with a lot of people, or are we also trying to create more affinity to help the university’s mission? And is participation the only way to see this affinity and increase it?

Most advancement professionals will probably agree that there are multiple factors that influence the affinity alumni feel for their university. Sophisticated market research may uncover interesting correlations between their experience as a student and their warmth towards the university in later years, or perhaps even how personal circumstances might influence their interest and pride in the university at certain moments in their life. The services, benefits, events and communications we provide are only part of the puzzle.

I wonder if a more interesting question to ask is: if growing and ‘making visible’ affinity is actually what’s important, how else can we do this in addition to participation? What’s also within our control?

In recent months I have started to come to the conclusion that perhaps not enough attention is being paid to what I would have called (in a previous life) ‘customer satisfaction’. That is, whether we are meeting and/or surpassing the expectations of our alumni. Part of this is seeking regular feedback on our programmes and services from our alumni, and part of it is understanding and respecting the individual needs and interests of our alumni. I am personally more likely to think favourably of an organisation that sends me relevant and tailored communications to my preferred schedule, than one that broadcasts a generic newsletter to me. And if I think favourably of that organisation, it follows that I’m more likely to use their services or donate to their causes.

So here’s what I propose:

  • Keep in touch with your alumni and use every available opportunity to seek their feedback on what’s working and what’s not
  • Invest in market research to understand what’s most important to each alum, and build your communication segmentation and programmes around this
  • Create a measure of affinity and/or satisfaction using information straight from the horse’s mouth (along with engagement), and check this regularly to see how you are affecting these measures
  • Try to not exclude anyone from your communications and events programmes who hasn’t asked to be taken out – no matter how ‘disengaged’ they appear. Give them lots of opportunities to adjust their preferences, perhaps test out new opportunities to tempt them and treat them as a valuable member of your community. Just because you haven’t heard from them, it doesn’t mean they don’t feel affinity! Protect it where you can.

How are you measuring engagement and affinity in your university? Are you having similar discussions in your teams – and what have you concluded? Please comment below with your thoughts and reactions, or drop me a line!

2 thoughts on “The problem with alumni engagement metrics

  1. Chris Smart says:

    An excellent summary of the problem Holly. My only build on this is each institution needs to think very carefully about what it is they have to offer their alumni community. It’s great to attend reunions, events, get careers advice, even volunteer to help/mentor current students but is this enough? So, before you start to decide what to measure and how to do that, you need to decide what the offer is. To my mind this must be done in conjunction with alums and the answers will probably be different for different sub-populations e.g. based on age, subject of study, interests, charitable giving preferences etc. For example, if a University has a number of alums who are passionate about raising money for cancer research, or at least keeping up to date with the latest research in the area, together with academic expertise in cancer research, finding a way to engage alums in these activities would likely increase affinity which may result in them getting involved. Once you have this kind of data about your alumni you can start to plan activities and measure success. It requires a lot of effort, and a decent CRM, but the rewards are almost certainly there.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s