Resource

Assessing the Success of Analytics-led Interventions

Learning AnalyticsStudent Success

Description

A short guide to evaluating analytics-led interventions, covering definitions of success, evaluation planning, possible indicators and the challenges of attributing changes to an intervention.

Benefit of this resource and how to make the best use of it

This two-page National Forum guidance note explores how higher education institutions can evaluate the effectiveness of analytics-led interventions. It encourages institutions to define what success means in relation to their own aims and to plan evaluation during the design and implementation of an analytics strategy.

The note distinguishes the ability to answer questions about students’ learning or engagement from the impact of actions taken in response. It highlights the difficulty of attributing changes to an intervention when other factors may also be involved. Possible indicators include engagement with learning resources, changes in individual students’ grades over time, use of support services, development of graduate attributes and changes in estimated withdrawal risk. It cautions against relying on grades alone or treating an observed improvement as proof of causation.

How it can be used

  • Clarify the intended outcomes of an analytics-led intervention before choosing measures of success.
  • Build evaluation and evidence collection into the intervention’s design.
  • Consider a range of indicators that reflect students’ learning, engagement and individual circumstances.
  • Support critical discussion of findings, including alternative explanations for observed changes.

People, communities and initiatives

Contributors

Communities

Initiatives

Creative Commons Attribution (CC BY)

This work is licensed under a CC BY license, allowing sharing and adaptation with proper attribution.

https://creativecommons.org/licenses/by/4.0/
Assessing the Success of Analytics-led Interventions

Access this resource

Links open in a new tab.

Additional files

Ancillary, supplementary or associated files…

Additional resource information

Resource details

Media format

Licence

Resource created

15/08/2017

Persistent identifier

File attachments

Share this resource

Report an issue

Adapting this resource? Share your version!

If you have modified or adopted this resource, share your version here. Tracking adaptations helps us measure impact and connects others with useful updates.