Multimodal approaches to video analysis of digital learning environments

Contemporary digital learning environments, such as tangibles, mobile and sensor-based technologies are inherently multimodal, both in terms of representation modalities and interaction modes. Drawing on social semiotics, multimodality offers a valuable approach for analysing video data, as it systematically attends to the interpretation of a wide range of communicational forms (e.g. gaze, posture, action, speech) used for making meaning. This paper explores a multimodal approach to video data in the context of ubiquitous technologies for learning.