CADA: a teacher-facing learning analytics dashboard to foster teachers’ awareness of students’ participation and discourse patterns in online discussions

Abstract

Despite the potential of learning analytics (LA) to support teachers’ everyday practice, its adoption has not been fully embraced due to the limited involvement of teachers as co-designers of LA systems and interventions. This is the focus of the study described in this paper. Following a design-based research (DBR) approach and guided by concepts from the socio-cultural perspective and human-computer interaction (HCI), we design, test, and evaluate a teacher-facing LA dashboard, the Canvas Discussion Analytics Dashboard (CADA), in real educational settings. The goal of this dashboard is to support teachers’ roles in online environments through insights into students’ participation and discourse patterns. We evaluate CADA through 10 in-depth interviews with university teachers to examine their experiences using CADA in seven blended undergraduate and graduate courses over a one-year period. The findings suggest that engaging teachers throughout the analytics tool design process and giving them control/agency over LA tools can favour their adoption in practice. Additionally, the alignment of dashboard metrics with relevant theoretical constructs allows teachers to monitor the learning designs and make course design changes on the fly. The teachers in this study emphasise the need for LA dashboards to provide actionable insights by moving beyond what things are towards how things should be. This study has several contributions. First, we make an artefact contribution (e.g. CADA), an LA dashboard to support teachers with insights into students’ online discussions. Second, by leveraging theory, and working with the teachers to develop and implement a dashboard in authentic teaching environments, we make an empirical, theoretical and methodological contribution to the field of learning analytics and technology enhanced learning. We synthesise these through practical design and implementation considerations for researchers, dashboard developers, and higher education institutions.