Co-authored by Keith Heggart, Jenny Wallace and Lucy Blakemore
L.A. is epidemically everywhere and discernible only in glimpses.
James Ellroy, American crime fiction writer and essayist
James Ellroy may not strictly have been writing about Learning Analytics in this quote, but he might as well have been. Educators have never had access to as much data as they do now, and yet for many, finding meaning and practical applications in this data remains elusive. So how can we make sure that this data is used efficiently to improve student learning?
L.A. and the NAPLAN effect
Learning analytics often involves collecting data about student learning with the aim of improving the learning and teaching environment. This data can take many forms and is gathered in different ways; Canvas records when and for how long users log in, for example, and UTS asks students to rate their learning experience via the Student Feedback Survey (SFS). This information can then be used to identify patterns or make predictions about learning and teaching.
Lots of the data learning institutions gather is essentially retrospective; we look back at subject or course engagement analytics, completions, and grades. While the analysis of this data might be fed forward into the next iteration of the subject and inform future learning design, it has limited benefit for the students who generated it. Like the often-criticised NAPLAN testing, results may show too little, too late – especially in the space of a 6-12 week subject.
There is also the issue of what learning analytics can truly tell us about the learning process. A successful university experience does not look the same for all learners, and some students will learn lessons that aren’t itemised in a marking scheme or recorded in a learning management system. Not everything that counts can be counted.
Retro is nice, real time is nicer
So, what benefit can we gain from using learning analytics? I propose that learning analytics are most effective when used prescriptively. This means identifying a point of need and designing a learning strategy to address the issue in a timely manner and in a way that can have an impact for current learners.
For instance, if data shows that most students failed a specific weekly quiz, there are several interventions that can occur immediately to try and turn the situation around, from configuration checks to make sure the quiz was set up correctly, to reviewing and practicing the quiz content again in a tutorial.
Tackling L.A. roadblocks: small steps
There can be lots of barriers to using learning analytics, particularly in prescriptive ways. It’s hard to find time to act on insights from learning analytics during a demanding subject with many students, even if you already have the skills and know-how. There are also ethical considerations around exactly what data is collected, how it is stored, who it belongs to, and what decisions it informs.
Many educators may simply be unaware of the benefits from effective interpretation of learning analytics both for teachers and students; dashboards may be clunky and unintuitive, and show data that is not immediately valuable.
Using learning analytics to address specific pain points is a good place to start; if a subject is experiencing frequent late assignment submissions, for example, analysing and comparing submission dates across subjects could allow tutors to make small changes to deadlines to assist students in managing their workload. From smaller wins like this, we can start considering bigger challenges like changing an assessment structure to make better use of the opportunities afforded by learning analytics.
Come and explore L.A. with the Learning Design Meetup!
If you’re keen to delve deeper, the next quarterly UTS Learning Design Meetup in June is sharing practitioner reflections and examples of impact with learning analytics. Before you come along, they would like to understand where you’re at with Learning Analytics. How have you used L.A. (if at all)? How confident are you in putting them to use? What are the skills learning designers need to know, and what questions or concerns do you have?
Share your responses and take a peek at the results straight after: https://www.menti.com/al947qazo5y7. To join the Meetup, contact Mais Fatayer and be part of the conversation in the Learning Design Meetup Teams channel.