Learning analytics have often been hyped to provide quick solutions to sometimes rather complex questions. In her presentation ‘Behind the (Page) Views,’ at the June Learning Design Meetup ‘Learning Analytics: Crunching Data, Design and Pedagogy’, Donna Rooney shared insights about the practical realities of these aspirations. What does it really take to make sense of Canvas New Analytics data? Can the numbers really unlock meaningful insights into learning and learning outcomes?
Canvas analytics only tell part of the story
At first glance, discussion boards in Canvas only seem valuable if students contribute to the conversation. So, when data analytics from discussion-focussed Masters subject Learning and its Trajectories showed that 90% of interactions with a forum were views and only 10% were contributions, Donna questioned whether the intended learning aim was achieved.
The data represented student engagement with discussion forums that contained assessed blog tasks at the end of each learning unit. The pages included a variety of prompts to which students were asked to respond, and further materials to support the task and writing process. Students were encouraged to read, like and comment on their peers’ blog posts.
Does lurking equal listening and learning?
The initial learning analytics data looked concerning, but when Donna dug deeper into the posts, she noticed that students were having authentic conversations and actively engaging with each other. Further investigation revealed that, on average, students viewed a discussion page 180 times!
This phenomenon of ‘lurking’ versus actively posting intrigued Donna and raised the question: is lurking a valuable part of the learning process? To gain more clarity, Donna interviewed her students to find out what they were doing when viewing the page and how this contributed to their learning experiences. This process helped her identify four distinct categories of lurking:
- Lurking for assessment: students engaged with the assessment materials on the discussion page.
- Lurking for learning: Students interacted with others’ blogs, using them as sources of learning and inspiration.
- Lurking for connection: Students engaged with each other, fostering interactions and connections within the online community.
- Social media lurking: Students used the discussion page as a means to build an online community, similar to social media platforms.
Social learning at its best
The interviews showed that the assessment tasks played a crucial role in fostering an online community. Students wrote blogs for an audience other than the lecturer. They explored diverse perspectives on the same theory by reading others’ blogs. They compared their own work to their peers’ to enhance their writing and help with the understanding of presented topics. Effectively, learners engaged in activities more commonly associated with social media than with traditional education: they actively liked and followed classmates and conversations.
The initially disheartening figure of 90% lurking actually was “addictive engagement” as one student put it. Seen through pedagogical and learning design lenses, the blog discussion forums epitomise educationally purposeful activities.
Donna’s experience highlighted the crucial role of qualitative feedback in making sense of quantitative data. She reminded us that the true power of learning analytics lies not just in the numbers, but in asking the right questions to gain meaningful insights.
If you would like to find out more about Donna’s research, keep an eye out – there’s a publication currently under review which will hopefully be available soon.
To explore more on Learning Analytics, check out summaries and recordings from our two other speakers at the June Learning Design Meetup. Mais Fatayer shares highlights from guest presenter Danny Liu in Fostering human relationships through learning analytics, whilst Kirsty Kitto (Connected Intelligence Centre, UTS) explores the topic of Cooking data with care: how can we do Learning Analytics ethically?.