If you’ve been following along from December’s blog, you’ll know that last month’s challenge was to make a list of things you’d like to know from (or about) your alumni and supporters. In this blog, I’ll share my magic formula with you and show you how to use it to plan a successful research project.
Two of the biggest enemies of research are unreasonable expectations and scope creep. The magic formula, while simple, forces you to consider the data you need and any limitations to this data at the project outset. It also helps you keep an open mind, so you don’t constrain yourself to one methodology too soon in the planning phase.
The magic formula
Step 1: Apply the formula
To apply the formula, write down each of the things you’d like to know from your audiences according to this X, Y, Z structure.
For example, if you want to know the barriers to attending your institution’s events, you might write this down as:
“I want to know the barriers to attending our events from alumni who haven’t attended one of our events in the past two years in order to make changes to our events programme to attract a wider audience.”
You can see instantly that this format is going to be a lot more helpful to you than the original “I want to know what our alumni think the barriers to attending our events are”.
Once you’ve written all your questions down in this way, you might spot themes emerging such as common audiences you need to seek feedback from, or one or two particularly important decisions that require multiple types of data.
Step 2: Determine the methodology
Quantitative and qualitative methodologies both have their uses but which you would choose (and how you would deploy them) depends largely on the information you seek, the audience you need to hear from and how you’re going to use the results – i.e. each element of the magic formula.
Using the example from Step 1, here are some of the factors I’d be considering:
- Firstly, how confident are you that you know what the potential event attendance barriers are? Imagine you were writing a multiple-choice survey question along the lines of “The main reason I have not attended an event recently is…”. If you don’t know what your multiple-choice answer options are, or you’re concerned you’d be introducing bias, you might need to do some preliminary qualitative research to figure them out. This could be as simple as phoning a few people who haven’t attended an event recently and asking for their thoughts, or it could mean inviting alumni to participate in a preliminary focus group where you can dig a bit deeper.
- Secondly, how easy is it to get in touch with your target audience, i.e. people who haven’t attended an event recently? It might pay to profile this group to see what percentages you can contact via email, post or phone. This may differ also by age – are there groups you’re keener to see at events than others, or do you need a representative sample across all ages? Knowing what channels are available for any quantitative research you may want to undertake BEFORE making a decision on methodology is crucial.
- Thirdly, how reliable do you need the data to be? There may be changes you’re happy to make based on the views of a few representative focus groups, but generally the costlier and more disruptive the changes, the more confidence you will need to have in the study’s results. While a combination of qualitative and quantitative methods will give you more certainty, consider also whether it makes more sense to seek feedback at a relevant time for the audience, so they are not recalling decisions they made months or years before. Using our example, the most reliable data would be gathered at the exact moment an alumna decides not to attend an event. Is there a way of including a quick survey as standard with event invitations?
Other methodology considerations may include whether you’ll need to give your audience the time and space to reflect or imagine new scenarios (qualitative may be best here), and any geographic or budget/resource limitations (can you run online focus groups instead of face-to-face, or limit the survey format to online only?).
Step 3: Ask yourself, is the answer to that question alone enough to make changes?
I’ve used the event barriers example specifically because I think it throws up another interesting question to add to the mix. If you know you’re going to use the results of your study to make big changes to your events programme, these changes may have unintended consequences for your current loyal event attendees. This is particularly true if you’ve been asked to restructure the events programme rather than just add in additional events for new audiences.
You may therefore want to include a secondary research question with this project:
“I want to know what motivates alumni to attend our current events from alumni who have attended at least one event in the past two years in order to ensure any changes we make to our events programme do not put off large numbers of our current event attendees”
You would then go through the process again considering the methodology that would work best for this research question.
Step 4: Have you got one research study, more than one, a recurring study or an integrated research programme?
Once you’re written out your key questions according to the formula, reflected on who you target audiences are and the best way to seek the information you require, you may discover that what you had initially thought of as one alumni census-style survey is now best served by three smaller and more targeted research studies with different methodologies.
Of course, you might not have the budget or time to do all three and need to make trade-offs with timing or data reliability, but at least you will be doing so with your eyes open. You can explain any limitations to your leadership team before the study even begins and effectively manage their expectations.
The benefits of the magic formula
Think of this formula as your insurance policy for a successful project. If you aren’t disciplined with its application you may be at risk of not meeting your leadership team’s expectations, using inadequate research methodologies or asking for feedback from the wrong audiences. Note: if you’ve undertaken a similar research project before and gathered lots of data you didn’t need, spent more budget than planned or haven’t done anything with the results, it may be because you skipped this vital step!
Other big benefits to using the formula include:
- Knowing how you’re going to use the data keeps you focused when designing survey questionnaires or focus group scripts. You’ll know exactly what questions you need to ask (and have a better idea how to ask them) and can challenge any additional questions that other team members wish to include that are outside of your scope.
- Clarity with your target audience makes it a lot easier to decide who needs to be invited to your focus groups (or to participate in your surveys) and which classification questions you might need within the surveys.
- Having tight scope and knowing how the data will be used is respectful of your audiences’ time and contribution, and allows you to accurately and persuasively position the survey to encourage their participation.
- If you’re thinking of commissioning an agency to help you, applying the magic formula will also mean you can provide them with an excellent brief and may even receive a sharper quote.
Time to get started!
If you’re planning to conduct alumni or supporter research in 2018, I hope this has got you off to a flying start (broomsticks optional!).
If you have any questions, or perhaps a different approach that you’ve found useful to share, please feel free to post them below or drop me a line. Otherwise, happy researching this year everyone!