2024
1.
Feng, Shihui; Yan, Lixiang; Zhao, Linxuan; Maldonado, Roberto Martinez; Gašević, Dragan
Heterogenous Network Analytics of Small Group Teamwork: Using Multimodal Data to Uncover Individual Behavioral Engagement Strategies Proceedings Article
In: Proceedings of the 14th Learning Analytics and Knowledge Conference, pp. 587–597, Association for Computing Machinery, Kyoto,Japan, 2024, ISBN: 9798400716188.
@inproceedings{feng-lak24,
title = {Heterogenous Network Analytics of Small Group Teamwork: Using Multimodal Data to Uncover Individual Behavioral Engagement Strategies},
author = {Shihui Feng and Lixiang Yan and Linxuan Zhao and Roberto Martinez Maldonado and Dragan Gašević},
url = {https://doi.org/10.1145/3636555.3636918},
doi = {10.1145/3636555.3636918},
isbn = {9798400716188},
year = {2024},
date = {2024-03-15},
urldate = {2024-01-01},
booktitle = {Proceedings of the 14th Learning Analytics and Knowledge Conference},
pages = {587–597},
publisher = {Association for Computing Machinery},
address = {Kyoto,Japan},
series = {LAK '24},
abstract = {Individual behavioral engagement is an important indicator of active learning in collaborative settings, encompassing multidimensional behaviors mediated through various interaction modes. Little existing work has explored the use of multimodal process data to understand individual behavioral engagement in face-to-face collaborative learning settings. In this study we bridge this gap, for the first time, introducing a heterogeneous tripartite network approach to analyze the interconnections among multimodal process data in collaborative learning. Students’ behavioral engagement strategies are analyzed based on their interaction patterns with various spatial locations and verbal communication types using a heterogeneous tripartite network. The multimodal collaborative learning process data were collected from 15 teams of four students. We conducted stochastic blockmodeling on a projection of the heterogeneous tripartite network to cluster students into groups that shared similar spatial and oral engagement patterns. We found two distinct clusters of students, whose characteristic behavioural engagement strategies were identified by extracting interaction patterns that were statistically significant relative to a multinomial null model. The two identified clusters also exhibited a statistically significant difference regarding students’ perceived collaboration satisfaction and teacher-assessed team performance level. This study advances collaboration analytics methodology and provides new insights into personalized support in collaborative learning.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Individual behavioral engagement is an important indicator of active learning in collaborative settings, encompassing multidimensional behaviors mediated through various interaction modes. Little existing work has explored the use of multimodal process data to understand individual behavioral engagement in face-to-face collaborative learning settings. In this study we bridge this gap, for the first time, introducing a heterogeneous tripartite network approach to analyze the interconnections among multimodal process data in collaborative learning. Students’ behavioral engagement strategies are analyzed based on their interaction patterns with various spatial locations and verbal communication types using a heterogeneous tripartite network. The multimodal collaborative learning process data were collected from 15 teams of four students. We conducted stochastic blockmodeling on a projection of the heterogeneous tripartite network to cluster students into groups that shared similar spatial and oral engagement patterns. We found two distinct clusters of students, whose characteristic behavioural engagement strategies were identified by extracting interaction patterns that were statistically significant relative to a multinomial null model. The two identified clusters also exhibited a statistically significant difference regarding students’ perceived collaboration satisfaction and teacher-assessed team performance level. This study advances collaboration analytics methodology and provides new insights into personalized support in collaborative learning.
2.
Echeverria, Vanessa; Yan, Lixiang; Zhao, Linxuan; Abel, Sophie; Alfredo, Riordan; Dix, Samantha; Jaggard, Hollie; Wotherspoon, Rosie; Osborne, Abra; Shum, Simon Buckingham; Gasevic, Dragan; Martinez-Maldonado, Roberto
TeamSlides: a Multimodal Teamwork Analytics Dashboard for Teacher-guided Reflection in a Physical Learning Space Proceedings Article
In: Proceedings of the 14th Learning Analytics and Knowledge Conference, pp. 112–122, Association for Computing Machinery, Kyoto,Japan, 2024, ISBN: 9798400716188.
@inproceedings{echeverria-lak24,
title = {TeamSlides: a Multimodal Teamwork Analytics Dashboard for Teacher-guided Reflection in a Physical Learning Space},
author = {Vanessa Echeverria and Lixiang Yan and Linxuan Zhao and Sophie Abel and Riordan Alfredo and Samantha Dix and Hollie Jaggard and Rosie Wotherspoon and Abra Osborne and Simon Buckingham Shum and Dragan Gasevic and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3636555.3636857},
doi = {10.1145/3636555.3636857},
isbn = {9798400716188},
year = {2024},
date = {2024-03-15},
urldate = {2024-01-01},
booktitle = {Proceedings of the 14th Learning Analytics and Knowledge Conference},
pages = {112–122},
publisher = {Association for Computing Machinery},
address = {Kyoto,Japan},
series = {LAK '24},
abstract = {Advancements in Multimodal Learning Analytics (MMLA) have the potential to enhance the development of effective teamwork skills and foster reflection on collaboration dynamics in physical learning environments. Yet, only a few MMLA studies have closed the learning analytics loop by making MMLA solutions immediately accessible to educators to support reflective practices, especially in authentic settings. Moreover, deploying MMLA solutions in authentic settings can bring new challenges beyond logistic and privacy issues. This paper reports the design and use of TeamSlides, a multimodal teamwork analytics dashboard to support teacher-guided reflection. We conducted an in-the-wild classroom study involving 11 teachers and 138 students. Multimodal data were collected from students working in team healthcare simulations. We examined how teachers used the dashboard in 22 debrief sessions to aid their reflective practices. We also interviewed teachers to discuss their perceptions of the dashboard’s value and the challenges faced during its use. Our results suggest that the dashboard effectively reinforced discussions and augmented teacher-guided reflection practices. However, teachers encountered interpretation conflicts, sometimes leading to mistrust or misrepresenting the information. We discuss the considerations needed to overcome these challenges in MMLA research.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Advancements in Multimodal Learning Analytics (MMLA) have the potential to enhance the development of effective teamwork skills and foster reflection on collaboration dynamics in physical learning environments. Yet, only a few MMLA studies have closed the learning analytics loop by making MMLA solutions immediately accessible to educators to support reflective practices, especially in authentic settings. Moreover, deploying MMLA solutions in authentic settings can bring new challenges beyond logistic and privacy issues. This paper reports the design and use of TeamSlides, a multimodal teamwork analytics dashboard to support teacher-guided reflection. We conducted an in-the-wild classroom study involving 11 teachers and 138 students. Multimodal data were collected from students working in team healthcare simulations. We examined how teachers used the dashboard in 22 debrief sessions to aid their reflective practices. We also interviewed teachers to discuss their perceptions of the dashboard’s value and the challenges faced during its use. Our results suggest that the dashboard effectively reinforced discussions and augmented teacher-guided reflection practices. However, teachers encountered interpretation conflicts, sometimes leading to mistrust or misrepresenting the information. We discuss the considerations needed to overcome these challenges in MMLA research.
3.
Alfredo, Riordan; Echeverria, Vanessa; Jin, Yueqiao; Swiecki, Zachari; Gašević, Dragan; Martinez-Maldonado, Roberto
SLADE: A Method for Designing Human-Centred Learning Analytics Systems Proceedings Article
In: Proceedings of the 14th Learning Analytics and Knowledge Conference, pp. 24–34, Association for Computing Machinery, Kyoto,Japan, 2024, ISBN: 9798400716188.
@inproceedings{10.1145/3636555.3636847,
title = {SLADE: A Method for Designing Human-Centred Learning Analytics Systems},
author = {Riordan Alfredo and Vanessa Echeverria and Yueqiao Jin and Zachari Swiecki and Dragan Gašević and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3636555.3636847},
doi = {10.1145/3636555.3636847},
isbn = {9798400716188},
year = {2024},
date = {2024-01-01},
booktitle = {Proceedings of the 14th Learning Analytics and Knowledge Conference},
pages = {24–34},
publisher = {Association for Computing Machinery},
address = {Kyoto,Japan},
series = {LAK '24},
abstract = {There is a growing interest in creating Learning Analytics (LA) systems that incorporate student perspectives. Yet, many LA systems still lean towards a technology-centric approach, potentially overlooking human values and the necessity of human oversight in automation. Although some recent LA studies have adopted a human-centred design stance, there is still limited research on establishing safe, reliable, and trustworthy systems during the early stages of LA design. Drawing from a newly proposed framework for human-centred artificial intelligence, we introduce SLADE, a method for ideating and identifying features of human-centred LA systems that balance human control and computer automation. We illustrate SLADE’s application in designing LA systems to support collaborative learning in healthcare. Twenty-one third-year students participated in design sessions through SLADE’s four steps: i) identifying challenges and corresponding LA systems; ii) prioritising these LA systems; iii) ideating human control and automation features; and iv) refining features emphasising safety, reliability, and trustworthiness. Our results demonstrate SLADE’s potential to assist researchers and designers in: 1) aligning authentic student challenges with LA systems through both divergent ideation and convergent prioritisation; 2) understanding students’ perspectives on personal agency and delegation to teachers; and 3) fostering discussions about the safety, reliability, and trustworthiness of LA solutions.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
There is a growing interest in creating Learning Analytics (LA) systems that incorporate student perspectives. Yet, many LA systems still lean towards a technology-centric approach, potentially overlooking human values and the necessity of human oversight in automation. Although some recent LA studies have adopted a human-centred design stance, there is still limited research on establishing safe, reliable, and trustworthy systems during the early stages of LA design. Drawing from a newly proposed framework for human-centred artificial intelligence, we introduce SLADE, a method for ideating and identifying features of human-centred LA systems that balance human control and computer automation. We illustrate SLADE’s application in designing LA systems to support collaborative learning in healthcare. Twenty-one third-year students participated in design sessions through SLADE’s four steps: i) identifying challenges and corresponding LA systems; ii) prioritising these LA systems; iii) ideating human control and automation features; and iv) refining features emphasising safety, reliability, and trustworthiness. Our results demonstrate SLADE’s potential to assist researchers and designers in: 1) aligning authentic student challenges with LA systems through both divergent ideation and convergent prioritisation; 2) understanding students’ perspectives on personal agency and delegation to teachers; and 3) fostering discussions about the safety, reliability, and trustworthiness of LA solutions.
4.
Zhao, Linxuan; Echeverria, Vanessa; Swiecki, Zachari; Yan, Lixiang; Alfredo, Riordan; Li, Xinyu; Gasevic, Dragan; Martinez-Maldonado, Roberto
Epistemic Network Analysis for End-users: Closing the Loop in the Context of Multimodal Analytics for Collaborative Team Learning Proceedings Article
In: Proceedings of the 14th Learning Analytics and Knowledge Conference, pp. 90–100, Association for Computing Machinery, Kyoto, Japan, 2024, ISBN: 9798400716188.
@inproceedings{zhao-lak24,
title = {Epistemic Network Analysis for End-users: Closing the Loop in the Context of Multimodal Analytics for Collaborative Team Learning},
author = {Linxuan Zhao and Vanessa Echeverria and Zachari Swiecki and Lixiang Yan and Riordan Alfredo and Xinyu Li and Dragan Gasevic and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3636555.3636855},
doi = {10.1145/3636555.3636855},
isbn = {9798400716188},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the 14th Learning Analytics and Knowledge Conference},
pages = {90–100},
publisher = {Association for Computing Machinery},
address = {Kyoto, Japan},
series = {LAK '24},
abstract = {Effective collaboration and team communication are critical across many sectors. However, the complex dynamics of collaboration in physical learning spaces, with overlapping dialogue segments and varying participant interactions, pose assessment challenges for educators and self-reflection difficulties for students. Epistemic network analysis (ENA) is a relatively novel technique that has been used in learning analytics (LA) to unpack salient aspects of group communication. Yet, most LA works based on ENA have primarily sought to advance research knowledge rather than directly aid teachers and students by closing the LA loop. We address this gap by conducting a study in which we i) engaged teachers in designing human-centred versions of epistemic networks; ii) formulated an NLP methodology to code physically distributed dialogue segments of students based on multimodal (audio and positioning) data, enabling automatic generation of epistemic networks; and iii) deployed the automatically generated epistemic networks in 28 authentic learning sessions and investigated how they can support teaching. The results indicate the viability of completing the analytics loop through the design of streamlined epistemic network representations that enable teachers to support students’ reflections.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Effective collaboration and team communication are critical across many sectors. However, the complex dynamics of collaboration in physical learning spaces, with overlapping dialogue segments and varying participant interactions, pose assessment challenges for educators and self-reflection difficulties for students. Epistemic network analysis (ENA) is a relatively novel technique that has been used in learning analytics (LA) to unpack salient aspects of group communication. Yet, most LA works based on ENA have primarily sought to advance research knowledge rather than directly aid teachers and students by closing the LA loop. We address this gap by conducting a study in which we i) engaged teachers in designing human-centred versions of epistemic networks; ii) formulated an NLP methodology to code physically distributed dialogue segments of students based on multimodal (audio and positioning) data, enabling automatic generation of epistemic networks; and iii) deployed the automatically generated epistemic networks in 28 authentic learning sessions and investigated how they can support teaching. The results indicate the viability of completing the analytics loop through the design of streamlined epistemic network representations that enable teachers to support students’ reflections.
2023
5.
Martinez-Maldonado, Roberto; Echeverria, Vanessa; Fernandez-Nieto, Gloria; Yan, Lixiang; Zhao, Linxuan; Alfredo, Riordan; Li, Xinyu; Dix, Samantha; Jaggard, Hollie; Wotherspoon, Rosie; Osborne, Abra; Shum, Simon Buckingham; Gašević, Dragan
Lessons Learnt from a Multimodal Learning Analytics Deployment In-the-Wild Journal Article
In: ACM Trans. Comput.-Hum. Interact., vol. 31, no. 1, 2023, ISSN: 1073-0516.
@article{10.1145/3622784,
title = {Lessons Learnt from a Multimodal Learning Analytics Deployment In-the-Wild},
author = {Roberto Martinez-Maldonado and Vanessa Echeverria and Gloria Fernandez-Nieto and Lixiang Yan and Linxuan Zhao and Riordan Alfredo and Xinyu Li and Samantha Dix and Hollie Jaggard and Rosie Wotherspoon and Abra Osborne and Simon Buckingham Shum and Dragan Gašević},
url = {https://doi.org/10.1145/3622784},
doi = {10.1145/3622784},
issn = {1073-0516},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {ACM Trans. Comput.-Hum. Interact.},
volume = {31},
number = {1},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations “in-the-wild”. These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers’ tasks. These practicalities have been rarely investigated. This article addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators in the context of nursing education. The lessons learnt were synthesised into topics related to (i) technological/physical aspects of the deployment; (ii) multimodal data and interfaces; (iii) the design process; (iv) participation, ethics and privacy; and (v) sustainability of the deployment.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations “in-the-wild”. These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers’ tasks. These practicalities have been rarely investigated. This article addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators in the context of nursing education. The lessons learnt were synthesised into topics related to (i) technological/physical aspects of the deployment; (ii) multimodal data and interfaces; (iii) the design process; (iv) participation, ethics and privacy; and (v) sustainability of the deployment.
6.
Li, Xinyu; Yan, Lixiang; Zhao, Linxuan; Martinez-Maldonado, Roberto; Gasevic, Dragan
CVPE: A Computer Vision Approach for Scalable and Privacy-Preserving Socio-Spatial, Multimodal Learning Analytics Proceedings Article
In: LAK23: 13th International Learning Analytics and Knowledge Conference, pp. 175–185, Association for Computing Machinery, Arlington, TX, USA, 2023, ISBN: 9781450398657.
@inproceedings{li2024cpve,
title = {CVPE: A Computer Vision Approach for Scalable and Privacy-Preserving Socio-Spatial, Multimodal Learning Analytics},
author = {Xinyu Li and Lixiang Yan and Linxuan Zhao and Roberto Martinez-Maldonado and Dragan Gasevic},
url = {https://doi.org/10.1145/3576050.3576145},
doi = {10.1145/3576050.3576145},
isbn = {9781450398657},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {LAK23: 13th International Learning Analytics and Knowledge Conference},
pages = {175–185},
publisher = {Association for Computing Machinery},
address = {Arlington, TX, USA},
series = {LAK2023},
abstract = {Capturing data on socio-spatial behaviours is essential in obtaining meaningful educational insights into collaborative learning and teamwork in co-located learning contexts. Existing solutions, however, have limitations regarding scalability and practicality since they rely largely on costly location tracking systems, are labour-intensive, or are unsuitable for complex learning environments. To address these limitations, we propose an innovative computer-vision-based approach – Computer Vision for Position Estimation (CVPE) – for collecting socio-spatial data in complex learning settings where sophisticated collaborations occur. CVPE is scalable and practical with a fast processing time and only needs low-cost hardware (e.g., cameras and computers). The built-in privacy protection modules also minimise potential privacy and data security issues by masking individuals’ facial identities and provide options to automatically delete recordings after processing, making CVPE a suitable option for generating continuous multimodal/classroom analytics. The potential of CVPE was evaluated by applying it to analyse video data about teamwork in simulation-based learning. The results showed that CVPE extracted socio-spatial behaviours relatively reliably from video recordings compared to indoor positioning data. These socio-spatial behaviours extracted with CVPE uncovered valuable insights into teamwork when analysed with epistemic network analysis. The limitations of CVPE for effective use in learning analytics are also discussed.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Capturing data on socio-spatial behaviours is essential in obtaining meaningful educational insights into collaborative learning and teamwork in co-located learning contexts. Existing solutions, however, have limitations regarding scalability and practicality since they rely largely on costly location tracking systems, are labour-intensive, or are unsuitable for complex learning environments. To address these limitations, we propose an innovative computer-vision-based approach – Computer Vision for Position Estimation (CVPE) – for collecting socio-spatial data in complex learning settings where sophisticated collaborations occur. CVPE is scalable and practical with a fast processing time and only needs low-cost hardware (e.g., cameras and computers). The built-in privacy protection modules also minimise potential privacy and data security issues by masking individuals’ facial identities and provide options to automatically delete recordings after processing, making CVPE a suitable option for generating continuous multimodal/classroom analytics. The potential of CVPE was evaluated by applying it to analyse video data about teamwork in simulation-based learning. The results showed that CVPE extracted socio-spatial behaviours relatively reliably from video recordings compared to indoor positioning data. These socio-spatial behaviours extracted with CVPE uncovered valuable insights into teamwork when analysed with epistemic network analysis. The limitations of CVPE for effective use in learning analytics are also discussed.
7.
Zhao, Linxuan; Tan, Yuanru; Gašević, Dragan; Shaffer, David Williamson; Yan, Lixiang; Alfredo, Riordan; Li, Xinyu; Martinez-Maldonado, Roberto
Analysing Verbal Communication in Embodied Team Learning Using Multimodal Data and Ordered Network Analysis Proceedings Article
In: Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings, pp. 242–254, Springer-Verlag, Tokyo, Japan, 2023, ISBN: 978-3-031-36271-2.
@inproceedings{10.1007/978-3-031-36272-9_20,
title = {Analysing Verbal Communication in Embodied Team Learning Using Multimodal Data and Ordered Network Analysis},
author = {Linxuan Zhao and Yuanru Tan and Dragan Gašević and David Williamson Shaffer and Lixiang Yan and Riordan Alfredo and Xinyu Li and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1007/978-3-031-36272-9_20},
doi = {10.1007/978-3-031-36272-9_20},
isbn = {978-3-031-36271-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings},
pages = {242–254},
publisher = {Springer-Verlag},
address = {Tokyo, Japan},
abstract = {In embodied team learning activities, students are expected to learn to collaborate with others while freely moving in a physical learning space to complete a shared goal. Students can thus interact in various team configurations, resulting in increased complexity in their communication dynamics since unrelated dialogue segments can concurrently happen at different locations of the learning space. This can make it difficult to analyse students’ team dialogue solely using audio data. To address this problem, we present a study in a highly dynamic healthcare simulation setting to illustrate how spatial data can be combined with audio data to model embodied team communication. We used ordered network analysis (ONA) to model the co-occurrence and the order of coded co-located dialogue instances and identify key differences in the communication dynamics of high and low performing teams.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In embodied team learning activities, students are expected to learn to collaborate with others while freely moving in a physical learning space to complete a shared goal. Students can thus interact in various team configurations, resulting in increased complexity in their communication dynamics since unrelated dialogue segments can concurrently happen at different locations of the learning space. This can make it difficult to analyse students’ team dialogue solely using audio data. To address this problem, we present a study in a highly dynamic healthcare simulation setting to illustrate how spatial data can be combined with audio data to model embodied team communication. We used ordered network analysis (ONA) to model the co-occurrence and the order of coded co-located dialogue instances and identify key differences in the communication dynamics of high and low performing teams.
8.
Yan, Lixiang; Martinez-Maldonado, Roberto; Zhao, Linxuan; Li, Xinyu; Gašević, Dragan
Physiological Synchrony and Arousal as Indicators of Stress and Learning Performance in Embodied Collaborative Learning Proceedings Article
In: Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings, pp. 602–614, Springer-Verlag, Tokyo, Japan, 2023, ISBN: 978-3-031-36271-2.
@inproceedings{Yan2023physiologycalsync,
title = {Physiological Synchrony and Arousal as Indicators of Stress and Learning Performance in Embodied Collaborative Learning},
author = {Lixiang Yan and Roberto Martinez-Maldonado and Linxuan Zhao and Xinyu Li and Dragan Gašević},
url = {https://doi.org/10.1007/978-3-031-36272-9_49},
doi = {10.1007/978-3-031-36272-9_49},
isbn = {978-3-031-36271-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings},
pages = {602–614},
publisher = {Springer-Verlag},
address = {Tokyo, Japan},
abstract = {Advancements in sensing technologies, artificial intelligence (AI) and multimodal learning analytics (MMLA) are making it possible to model learners’ affective and physiological states. Physiological synchrony and arousal have been increasingly used to unpack students’ affective and cognitive states (e.g., stress), which can ultimately affect their learning performance and satisfaction in collaborative learning settings. Yet, whether these physiological features can be meaningful indicators of students’ stress and learning performance during highly dynamic, embodied collaborative learning (ECL) remains unclear. This paper explores the role of physiological synchrony and arousal as indicators of stress and learning performance in ECL. We developed two linear mixed models with the heart rate and survey data of 172 students in high-fidelity clinical simulations. The findings suggest that physiological synchrony measures are significant indicators of students’ perceived stress and collaboration performance, and physiological arousal measures are significant indicators of task performance, even after accounting for the variance explained by individual and group differences. These findings could contribute empirical evidence to support the development of analytic tools for supporting collaborative learning using AI and MMLA.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Advancements in sensing technologies, artificial intelligence (AI) and multimodal learning analytics (MMLA) are making it possible to model learners’ affective and physiological states. Physiological synchrony and arousal have been increasingly used to unpack students’ affective and cognitive states (e.g., stress), which can ultimately affect their learning performance and satisfaction in collaborative learning settings. Yet, whether these physiological features can be meaningful indicators of students’ stress and learning performance during highly dynamic, embodied collaborative learning (ECL) remains unclear. This paper explores the role of physiological synchrony and arousal as indicators of stress and learning performance in ECL. We developed two linear mixed models with the heart rate and survey data of 172 students in high-fidelity clinical simulations. The findings suggest that physiological synchrony measures are significant indicators of students’ perceived stress and collaboration performance, and physiological arousal measures are significant indicators of task performance, even after accounting for the variance explained by individual and group differences. These findings could contribute empirical evidence to support the development of analytic tools for supporting collaborative learning using AI and MMLA.
9.
Zhao, Linxuan; Swiecki, Zachari; Gasevic, Dragan; Yan, Lixiang; Dix, Samantha; Jaggard, Hollie; Wotherspoon, Rosie; Osborne, Abra; Li, Xinyu; Alfredo, Riordan; Martinez-Maldonado, Roberto
METS: Multimodal Learning Analytics of Embodied Teamwork Learning Proceedings Article
In: LAK23: 13th International Learning Analytics and Knowledge Conference, pp. 186–196, Association for Computing Machinery, Arlington,TX,USA, 2023, ISBN: 9781450398657.
@inproceedings{zhao2023mets,
title = {METS: Multimodal Learning Analytics of Embodied Teamwork Learning},
author = {Linxuan Zhao and Zachari Swiecki and Dragan Gasevic and Lixiang Yan and Samantha Dix and Hollie Jaggard and Rosie Wotherspoon and Abra Osborne and Xinyu Li and Riordan Alfredo and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3576050.3576076},
doi = {10.1145/3576050.3576076},
isbn = {9781450398657},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {LAK23: 13th International Learning Analytics and Knowledge Conference},
pages = {186–196},
publisher = {Association for Computing Machinery},
address = {Arlington,TX,USA},
series = {LAK2023},
abstract = {Embodied team learning is a form of group learning that occurs in co-located settings where students need to interact with others while actively using resources in the physical learning space to achieve a common goal. In such situations, communication dynamics can be complex as team discourse segments can happen in parallel at different locations of the physical space with varied team member configurations. This can make it hard for teachers to assess the effectiveness of teamwork and for students to reflect on their own experiences. To address this problem, we propose METS (Multimodal Embodied Teamwork Signature), a method to model team dialogue content in combination with spatial and temporal data to generate a signature of embodied teamwork. We present a study in the context of a highly dynamic healthcare team simulation space where students can freely move. We illustrate how signatures of embodied teamwork can help to identify key differences between high and low performing teams: i) across the whole learning session; ii) at different phases of learning sessions; and iii) at particular spaces of interest in the learning space.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Embodied team learning is a form of group learning that occurs in co-located settings where students need to interact with others while actively using resources in the physical learning space to achieve a common goal. In such situations, communication dynamics can be complex as team discourse segments can happen in parallel at different locations of the physical space with varied team member configurations. This can make it hard for teachers to assess the effectiveness of teamwork and for students to reflect on their own experiences. To address this problem, we propose METS (Multimodal Embodied Teamwork Signature), a method to model team dialogue content in combination with spatial and temporal data to generate a signature of embodied teamwork. We present a study in the context of a highly dynamic healthcare team simulation space where students can freely move. We illustrate how signatures of embodied teamwork can help to identify key differences between high and low performing teams: i) across the whole learning session; ii) at different phases of learning sessions; and iii) at particular spaces of interest in the learning space.
10.
Alfredo, Riordan; Nie, Lanbing; Kennedy, Paul; Power, Tamara; Hayes, Carolyn; Chen, Hui; McGregor, Carolyn; Swiecki, Zachari; Gašević, Dragan; Martinez-Maldonado, Roberto
"That Student Should be a Lion Tamer!" StressViz: Designing a Stress Analytics Dashboard for Teachers Proceedings Article
In: LAK23: 13th International Learning Analytics and Knowledge Conference, pp. 57–67, Association for Computing Machinery, Arlington,TX,USA, 2023, ISBN: 9781450398657.
@inproceedings{alfredo2024stressviz,
title = {"That Student Should be a Lion Tamer!" StressViz: Designing a Stress Analytics Dashboard for Teachers},
author = {Riordan Alfredo and Lanbing Nie and Paul Kennedy and Tamara Power and Carolyn Hayes and Hui Chen and Carolyn McGregor and Zachari Swiecki and Dragan Gašević and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3576050.3576058},
doi = {10.1145/3576050.3576058},
isbn = {9781450398657},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {LAK23: 13th International Learning Analytics and Knowledge Conference},
pages = {57–67},
publisher = {Association for Computing Machinery},
address = {Arlington,TX,USA},
series = {LAK2023},
abstract = {In recent years, there has been a growing interest in creating multimodal learning analytics (LA) systems that automatically analyse students’ states that are hard to see with the "naked eye", such as cognitive load and stress levels, but that can considerably shape their learning experience. A rich body of research has focused on detecting such aspects by capturing bodily signals from students using wearables and computer vision. Yet, little work has aimed at designing end-user interfaces that visualise physiological data to support tasks deliberately designed for students to learn from stressful situations. This paper addresses this gap by designing a stress analytics dashboard that encodes students’ physiological data into stress levels during different phases of an authentic team simulation in the context of nursing education. We conducted a qualitative study with teachers to understand (i) how they made sense of the stress analytics dashboard; (ii) the extent to which they trusted the dashboard in relation to students’ cortisol data; and (iii) the potential adoption of this tool to communicate insights and aid teaching practices.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In recent years, there has been a growing interest in creating multimodal learning analytics (LA) systems that automatically analyse students’ states that are hard to see with the "naked eye", such as cognitive load and stress levels, but that can considerably shape their learning experience. A rich body of research has focused on detecting such aspects by capturing bodily signals from students using wearables and computer vision. Yet, little work has aimed at designing end-user interfaces that visualise physiological data to support tasks deliberately designed for students to learn from stressful situations. This paper addresses this gap by designing a stress analytics dashboard that encodes students’ physiological data into stress levels during different phases of an authentic team simulation in the context of nursing education. We conducted a qualitative study with teachers to understand (i) how they made sense of the stress analytics dashboard; (ii) the extent to which they trusted the dashboard in relation to students’ cortisol data; and (iii) the potential adoption of this tool to communicate insights and aid teaching practices.
2022
11.
Echeverria, Vanessa; Martinez-Maldonado, Roberto; Yan, Lixiang; Zhao, Linxuan; Fernandez-Nieto, Gloria; Gasevic, Dragan; Shum, Simon Buckingham
HuCETA: A Framework for Human-Centered Embodied Teamwork Analytics Journal Article
In: IEEE Pervasive Computing, vol. 22, no. 1, pp. 39-49, 2022, ISSN: 1558-2590.
@article{Echeverria_Martinez-Maldonado_Yan_Zhao_Fernandez-Nieto_Gasevic_Shum_2022,
title = {HuCETA: A Framework for Human-Centered Embodied Teamwork Analytics},
author = {Vanessa Echeverria and Roberto Martinez-Maldonado and Lixiang Yan and Linxuan Zhao and Gloria Fernandez-Nieto and Dragan Gasevic and Simon Buckingham Shum},
url = {https://ieeexplore.ieee.org/abstract/document/9965572},
doi = {10.1109/MPRV.2022.3217454},
issn = {1558-2590},
year = {2022},
date = {2022-11-29},
urldate = {2022-11-29},
journal = {IEEE Pervasive Computing},
volume = {22},
number = {1},
pages = {39-49},
abstract = {Collocated teamwork remains a pervasive practice across all professional sectors. Even though live observations and video analysis have been utilized for understanding embodied interaction of team members, these approaches are impractical for scaling up the provision of feedback that can promote developing high-performance teamwork skills. Enriching spaces with sensors capable of automatically capturing team activity data can improve learning and reflection. Yet, connecting the enormous amounts of data such sensors can generate with constructs related to teamwork remains challenging. This article presents a framework to support the development of human-centered embodied teamwork analytics by 1) enabling hybrid human–machine multimodal sensing; 2) embedding educators’ and experts’ knowledge into computational team models; and 3) generating human-driven data storytelling interfaces for reflection and decision making. This is illustrated through an in-the-wild study in the context of healthcare simulation, where predictive modeling, epistemic network analysis, and data storytelling are used to support educators and nursing teams.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Collocated teamwork remains a pervasive practice across all professional sectors. Even though live observations and video analysis have been utilized for understanding embodied interaction of team members, these approaches are impractical for scaling up the provision of feedback that can promote developing high-performance teamwork skills. Enriching spaces with sensors capable of automatically capturing team activity data can improve learning and reflection. Yet, connecting the enormous amounts of data such sensors can generate with constructs related to teamwork remains challenging. This article presents a framework to support the development of human-centered embodied teamwork analytics by 1) enabling hybrid human–machine multimodal sensing; 2) embedding educators’ and experts’ knowledge into computational team models; and 3) generating human-driven data storytelling interfaces for reflection and decision making. This is illustrated through an in-the-wild study in the context of healthcare simulation, where predictive modeling, epistemic network analysis, and data storytelling are used to support educators and nursing teams.
12.
Yan, Lixiang; Martinez-Maldonado, Roberto; Zhao, Linxuan; Dix, Samantha; Jaggard, Hollie; Wotherspoon, Rosie; Li, Xinyu; Gašević, Dragan
The role of indoor positioning analytics in assessment of simulation-based learning Journal Article
In: British Journal of Educational Technology, vol. n/a, no. n/a, pp. 1-26, 2022.
@article{https://doi.org/10.1111/bjet.13262,
title = {The role of indoor positioning analytics in assessment of simulation-based learning},
author = {Lixiang Yan and Roberto Martinez-Maldonado and Linxuan Zhao and Samantha Dix and Hollie Jaggard and Rosie Wotherspoon and Xinyu Li and Dragan Gašević},
url = {https://bera-journals.onlinelibrary.wiley.com/doi/abs/10.1111/bjet.13262},
doi = {https://doi.org/10.1111/bjet.13262},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
journal = {British Journal of Educational Technology},
volume = {n/a},
number = {n/a},
pages = {1-26},
abstract = {Simulation-based learning provides students with unique opportunities to develop key procedural and teamwork skills in close-to-authentic physical learning and training environments. Yet, assessing students' performance in such situations can be challenging and mentally exhausting for teachers. Multimodal learning analytics can support the assessment of simulation-based learning by making salient aspects of students' activities visible for evaluation. Although descriptive analytics have been used to study students' motor behaviours in simulation-based learning, their validity and utility for assessing performance remain unclear. This study aims at addressing this knowledge gap by investigating how indoor positioning analytics can be used to generate meaningful insights about students' tasks and collaboration performance in simulation-based learning. We collected and analysed the positioning data of 304 healthcare students, organised in 76 teams, through correlation, predictive and epistemic network analyses. The primary findings were (1) large correlations between students' spatial-procedural behaviours and their group performances; (2) predictive learning analytics that achieved an acceptable level (0.74 AUC) in distinguishing between low-performing and high-performing teams regarding collaboration performance; and (3) epistemic networks that can be used for assessing the behavioural differences across multiple teams. We also present the teachers' qualitative evaluation of the utility of these analytics and implications for supporting formative assessment in simulation-based learning. Practitioner notes What is currently known about this topic Assessing students' performance in simulation-based learning is often challenging and mentally exhausting. The combination of learning analytics and sensing technologies has the potential to uncover meaningful behavioural insights in physical learning spaces. Observational studies have suggested the potential value of analytics extracted from positioning data as indicators of highly-effective behaviour in simulation-based learning. What this paper adds Indoor positioning analytics for supporting teachers' formative assessment and timely feedback on students' group/team-level performance in simulation-based learning. Empirical evidence supported the potential use of epistemic networks for assessing the behavioural differences between low-performing and high-performing teams. Teachers' positively validated the utility of indoor positioning analytics in supporting reflective practices and formative assessment in simulation-based learning. Implications for practitioners Indoor positioning tracking and spatial analysis can be used to investigate students' teamwork and task performance in simulation-based learning. Predictive learning analytics should be developed based on features that have direct relevance to teachers' learning design. Epistemic networks analysis and comparison plots can be useful in identifying and assessing behavioural differences across multiple teams.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Simulation-based learning provides students with unique opportunities to develop key procedural and teamwork skills in close-to-authentic physical learning and training environments. Yet, assessing students' performance in such situations can be challenging and mentally exhausting for teachers. Multimodal learning analytics can support the assessment of simulation-based learning by making salient aspects of students' activities visible for evaluation. Although descriptive analytics have been used to study students' motor behaviours in simulation-based learning, their validity and utility for assessing performance remain unclear. This study aims at addressing this knowledge gap by investigating how indoor positioning analytics can be used to generate meaningful insights about students' tasks and collaboration performance in simulation-based learning. We collected and analysed the positioning data of 304 healthcare students, organised in 76 teams, through correlation, predictive and epistemic network analyses. The primary findings were (1) large correlations between students' spatial-procedural behaviours and their group performances; (2) predictive learning analytics that achieved an acceptable level (0.74 AUC) in distinguishing between low-performing and high-performing teams regarding collaboration performance; and (3) epistemic networks that can be used for assessing the behavioural differences across multiple teams. We also present the teachers' qualitative evaluation of the utility of these analytics and implications for supporting formative assessment in simulation-based learning. Practitioner notes What is currently known about this topic Assessing students' performance in simulation-based learning is often challenging and mentally exhausting. The combination of learning analytics and sensing technologies has the potential to uncover meaningful behavioural insights in physical learning spaces. Observational studies have suggested the potential value of analytics extracted from positioning data as indicators of highly-effective behaviour in simulation-based learning. What this paper adds Indoor positioning analytics for supporting teachers' formative assessment and timely feedback on students' group/team-level performance in simulation-based learning. Empirical evidence supported the potential use of epistemic networks for assessing the behavioural differences between low-performing and high-performing teams. Teachers' positively validated the utility of indoor positioning analytics in supporting reflective practices and formative assessment in simulation-based learning. Implications for practitioners Indoor positioning tracking and spatial analysis can be used to investigate students' teamwork and task performance in simulation-based learning. Predictive learning analytics should be developed based on features that have direct relevance to teachers' learning design. Epistemic networks analysis and comparison plots can be useful in identifying and assessing behavioural differences across multiple teams.
13.
Fernandez-Nieto, Gloria; Kitto, Kirsty; Shum, Simon Buckingham; Martinez-Maldonado, Roberto
Beyond the Learning Analytics Dashboard: Alternative Ways to Communicate Student Data Insights Combining Visualisation, Narrative and Storytelling Proceedings Article
In: LAK22: 12th International Learning Analytics and Knowledge Conference, pp. 219–229, ACM, Online USA, 2022, ISBN: 978-1-4503-9573-1.
@inproceedings{FernandezNieto_Kitto_BuckinghamShum_Martinez-Maldonado_2022,
title = {Beyond the Learning Analytics Dashboard: Alternative Ways to Communicate Student Data Insights Combining Visualisation, Narrative and Storytelling},
author = {Gloria Fernandez-Nieto and Kirsty Kitto and Simon Buckingham Shum and Roberto Martinez-Maldonado},
url = {https://dl.acm.org/doi/10.1145/3506860.3506895},
doi = {10.1145/3506860.3506895},
isbn = {978-1-4503-9573-1},
year = {2022},
date = {2022-03-01},
urldate = {2022-03-01},
booktitle = {LAK22: 12th International Learning Analytics and Knowledge Conference},
pages = {219–229},
publisher = {ACM},
address = {Online USA},
abstract = {Learning Analytics (LA) dashboards have become a popular medium for communicating to teachers analytical insights obtained from student data. However, recent research indicates that LA dashboards can be complex to interpret, are often not grounded in educational theory, and frequently provide little or no guidance on how to interpret them. Despite these acknowledged problems, few suggestions have been made as to how we might improve the visual design of LA tools to support richer and alternative ways to communicate student data insights. In this paper, we explore three design alternatives to represent student multimodal data insights by combining data visualisation, narratives and storytelling principles. Based on foundations in data storytelling, three visual-narrative interfaces were designed with teachers: i) visual data slices, ii) a tabular visualisation, and iii) a written report. These were validated as a part of an authentic study where teachers explored activity logs and physiological data from co-located collaborative learning classes in the context of healthcare education. Results suggest that alternatives to LA dashboards can be considered as effective tools to support teachers’ reflection, and that LA designers should identify the representation type that best fits teachers’ needs.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Learning Analytics (LA) dashboards have become a popular medium for communicating to teachers analytical insights obtained from student data. However, recent research indicates that LA dashboards can be complex to interpret, are often not grounded in educational theory, and frequently provide little or no guidance on how to interpret them. Despite these acknowledged problems, few suggestions have been made as to how we might improve the visual design of LA tools to support richer and alternative ways to communicate student data insights. In this paper, we explore three design alternatives to represent student multimodal data insights by combining data visualisation, narratives and storytelling principles. Based on foundations in data storytelling, three visual-narrative interfaces were designed with teachers: i) visual data slices, ii) a tabular visualisation, and iii) a written report. These were validated as a part of an authentic study where teachers explored activity logs and physiological data from co-located collaborative learning classes in the context of healthcare education. Results suggest that alternatives to LA dashboards can be considered as effective tools to support teachers’ reflection, and that LA designers should identify the representation type that best fits teachers’ needs.
14.
Zhao, Linxuan; Yan, Lixiang; Gasevic, Dragan; Dix, Samantha; Jaggard, Hollie; Wotherspoon, Rosie; Alfredo, Riordan; Li, Xinyu; Martinez-Maldonado, Roberto
Modelling Co-Located Team Communication from Voice Detection and Positioning Data in Healthcare Simulation Proceedings Article
In: LAK22: 12th International Learning Analytics and Knowledge Conference, pp. 370–380, Association for Computing Machinery, Online, USA, 2022, ISBN: 9781450395731.
@inproceedings{10.1145/3506860.3506935,
title = {Modelling Co-Located Team Communication from Voice Detection and Positioning Data in Healthcare Simulation},
author = {Linxuan Zhao and Lixiang Yan and Dragan Gasevic and Samantha Dix and Hollie Jaggard and Rosie Wotherspoon and Riordan Alfredo and Xinyu Li and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3506860.3506935},
doi = {10.1145/3506860.3506935},
isbn = {9781450395731},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {LAK22: 12th International Learning Analytics and Knowledge Conference},
pages = {370–380},
publisher = {Association for Computing Machinery},
address = {Online, USA},
series = {LAK22},
abstract = {In co-located situations, team members use a combination of verbal and visual signals to communicate effectively, among which positional forms play a key role. The spatial patterns adopted by team members in terms of where in the physical space they are standing, and who their body is oriented to, can be key in analysing and increasing the quality of interaction during such face-to-face situations. In this paper, we model the students’ communication based on spatial (positioning) and audio (voice detection) data captured from 92 students working in teams of four in the context of healthcare simulation. We extract non-verbal events (i.e., total speaking time, overlapped speech,and speech responses to team members and teachers) and investigate to what extent they can serve as meaningful indicators of students’ performance according to teachers’ learning intentions. The contribution of this paper to multimodal learning analytics includes: i) a generic method to semi-automatically model communication in a setting where students can freely move in the learning space; and ii) results from a mixed-methods analysis of non-verbal indicators of team communication with respect to teachers’ learning design.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In co-located situations, team members use a combination of verbal and visual signals to communicate effectively, among which positional forms play a key role. The spatial patterns adopted by team members in terms of where in the physical space they are standing, and who their body is oriented to, can be key in analysing and increasing the quality of interaction during such face-to-face situations. In this paper, we model the students’ communication based on spatial (positioning) and audio (voice detection) data captured from 92 students working in teams of four in the context of healthcare simulation. We extract non-verbal events (i.e., total speaking time, overlapped speech,and speech responses to team members and teachers) and investigate to what extent they can serve as meaningful indicators of students’ performance according to teachers’ learning intentions. The contribution of this paper to multimodal learning analytics includes: i) a generic method to semi-automatically model communication in a setting where students can freely move in the learning space; and ii) results from a mixed-methods analysis of non-verbal indicators of team communication with respect to teachers’ learning design.
15.
Fernandez-Nieto, Gloria; An, Pengcheng; Zhao, Jian; Shum, Simon Buckingham; Martinez-Maldonado, Roberto
Classroom Dandelions: Visualising Participant Position, Trajectory and Body Orientation Augments Teachers’ Sensemaking Proceedings Article
In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery, New Orleans, LA, USA, 2022, ISBN: 9781450391573.
@inproceedings{10.1145/3491102.3517736,
title = {Classroom Dandelions: Visualising Participant Position, Trajectory and Body Orientation Augments Teachers’ Sensemaking},
author = {Gloria Fernandez-Nieto and Pengcheng An and Jian Zhao and Simon Buckingham Shum and Roberto Martinez-Maldonado},
url = {https://doi.org/10.1145/3491102.3517736},
doi = {10.1145/3491102.3517736},
isbn = {9781450391573},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems},
publisher = {Association for Computing Machinery},
address = {New Orleans, LA, USA},
series = {CHI '22},
abstract = {Despite the digital revolution, physical space remains the site for teaching and learning embodied knowledge and skills. Both teachers and students must develop spatial competencies to effectively use classroom spaces, enabling fluid verbal and non-verbal interaction. While video permits rich activity capture, it provides no support for quickly seeing activity patterns that can assist learning. In contrast, position tracking systems permit the automated modelling of spatial behaviour, opening new possibilities for feedback. This paper introduces the design rationale for ”Dandelion Diagrams” that integrate participant location, trajectory and body orientation over a variable period. Applied in two authentic teaching contexts (a science laboratory, and a nursing simulation) we show how heatmaps showing only teacher/student location led to misinterpretations that were resolved by overlaying Dandelion Diagrams. Teachers also identified a variety of ways they could aid professional development. We conclude Dandelion Diagrams assisted sensemaking, but discuss the ethical risks of over-interpretation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Despite the digital revolution, physical space remains the site for teaching and learning embodied knowledge and skills. Both teachers and students must develop spatial competencies to effectively use classroom spaces, enabling fluid verbal and non-verbal interaction. While video permits rich activity capture, it provides no support for quickly seeing activity patterns that can assist learning. In contrast, position tracking systems permit the automated modelling of spatial behaviour, opening new possibilities for feedback. This paper introduces the design rationale for ”Dandelion Diagrams” that integrate participant location, trajectory and body orientation over a variable period. Applied in two authentic teaching contexts (a science laboratory, and a nursing simulation) we show how heatmaps showing only teacher/student location led to misinterpretations that were resolved by overlaying Dandelion Diagrams. Teachers also identified a variety of ways they could aid professional development. We conclude Dandelion Diagrams assisted sensemaking, but discuss the ethical risks of over-interpretation.
2021
16.
Fernandez-Nieto, Gloria; Echeverria, Vanessa; Shum, Simon Buckingham; Mangaroska, Katerina; Kitto, Kirsty; Palominos, Evelyn; Axisa, Carmen; Martinez-Maldonado, Roberto
Storytelling With Learner Data: Guiding Student Reflection on Multimodal Team Data Journal Article
In: IEEE Transactions on Learning Technologies, vol. 14, no. 5, pp. 695–708, 2021, ISSN: 1939-1382, 2372-0050.
@article{Fernandez-Nieto_Echeverria_Shum_Mangaroska_Kitto_Palominos_Axisa_Martinez-Maldonado_2021,
title = {Storytelling With Learner Data: Guiding Student Reflection on Multimodal Team Data},
author = {Gloria Fernandez-Nieto and Vanessa Echeverria and Simon Buckingham Shum and Katerina Mangaroska and Kirsty Kitto and Evelyn Palominos and Carmen Axisa and Roberto Martinez-Maldonado},
url = {https://ieeexplore.ieee.org/document/9632388},
doi = {10.1109/TLT.2021.3131842},
issn = {1939-1382, 2372-0050},
year = {2021},
date = {2021-10-01},
urldate = {2021-10-01},
journal = {IEEE Transactions on Learning Technologies},
volume = {14},
number = {5},
pages = {695–708},
abstract = {There is growing interest in creating learning analytics feedback interfaces that support students directly. While dashboards and other visualizations are proliferating, the evidence is that many fail to provide meaningful insights that help students reflect productively. The contribution of this article is qualitative and quantitative evidence from two studies evaluating a multimodal teamwork analytics tool in authentic clinical teamwork simulations. Collocated activity data are rendered to help nursing students reflect on errors and stress-related incidents during simulations. The user interface explicitly guides student reflection using data storytelling principles, tuned to the intended learning outcomes. The results demonstrate the potential of interfaces that “tell one data story at a time,” by helping students to identify misconceptions and errors; think about strategies they might use to address errors, and reflect on their arousal levels. The results also illuminate broader issues around automated formative assessment, and the intelligibility and accountability of learning analytics.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
There is growing interest in creating learning analytics feedback interfaces that support students directly. While dashboards and other visualizations are proliferating, the evidence is that many fail to provide meaningful insights that help students reflect productively. The contribution of this article is qualitative and quantitative evidence from two studies evaluating a multimodal teamwork analytics tool in authentic clinical teamwork simulations. Collocated activity data are rendered to help nursing students reflect on errors and stress-related incidents during simulations. The user interface explicitly guides student reflection using data storytelling principles, tuned to the intended learning outcomes. The results demonstrate the potential of interfaces that “tell one data story at a time,” by helping students to identify misconceptions and errors; think about strategies they might use to address errors, and reflect on their arousal levels. The results also illuminate broader issues around automated formative assessment, and the intelligibility and accountability of learning analytics.
17.
Fernandez-Nieto, Gloria; Martinez-Maldonado, Roberto; Kitto, Kirsty; Shum, Simon Buckingham
Modelling Spatial Behaviours in Clinical Team Simulations using Epistemic Network Analysis: Methodology and Teacher Evaluation Proceedings Article
In: LAK21: 11th International Learning Analytics and Knowledge Conference, pp. 386–396, ACM, Irvine CA USA, 2021, ISBN: 978-1-4503-8935-8.
@inproceedings{Fernandez-Nieto_Martinez-Maldonado_Kitto_BuckinghamShum_2021,
title = {Modelling Spatial Behaviours in Clinical Team Simulations using Epistemic Network Analysis: Methodology and Teacher Evaluation},
author = {Gloria Fernandez-Nieto and Roberto Martinez-Maldonado and Kirsty Kitto and Simon Buckingham Shum},
url = {https://dl.acm.org/doi/10.1145/3448139.3448176},
doi = {10.1145/3448139.3448176},
isbn = {978-1-4503-8935-8},
year = {2021},
date = {2021-04-01},
urldate = {2021-04-01},
booktitle = {LAK21: 11th International Learning Analytics and Knowledge Conference},
pages = {386–396},
publisher = {ACM},
address = {Irvine CA USA},
abstract = {In nursing education through team simulations, students must learn to position themselves correctly in coordination with colleagues. However, with multiple student teams in action, it is difficult for teachers to give detailed, timely feedback on these spatial behaviours to each team. Indoor-positioning technologies can now capture student spatial behaviours, but relatively little work has focused on giving meaning to student activity traces, transforming low-level x/y coordinates into language that makes sense to teachers. Even less research has investigated if teachers can make sense of that feedback. This paper therefore makes two contributions. (1) Methodologically, we document the use of Epistemic Network Analysis (ENA) as an approach to model and visualise students’ movements. To our knowledge, this is the first application of ENA to analyse human movement. (2) We evaluated teachers’ responses to ENA diagrams through qualitative analysis of video-recorded sessions. Teachers constructed consistent narratives about ENA diagrams’ meaning, and valued the new insights ENA offered. However, ENA’s abstract visualisation of spatial behaviours was not intuitive, and caused some confusions. We propose, therefore, that the power of ENA modelling can be combined with other spatial representations such as a classroom map, by overlaying annotations to create a more intuitive user experience.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In nursing education through team simulations, students must learn to position themselves correctly in coordination with colleagues. However, with multiple student teams in action, it is difficult for teachers to give detailed, timely feedback on these spatial behaviours to each team. Indoor-positioning technologies can now capture student spatial behaviours, but relatively little work has focused on giving meaning to student activity traces, transforming low-level x/y coordinates into language that makes sense to teachers. Even less research has investigated if teachers can make sense of that feedback. This paper therefore makes two contributions. (1) Methodologically, we document the use of Epistemic Network Analysis (ENA) as an approach to model and visualise students’ movements. To our knowledge, this is the first application of ENA to analyse human movement. (2) We evaluated teachers’ responses to ENA diagrams through qualitative analysis of video-recorded sessions. Teachers constructed consistent narratives about ENA diagrams’ meaning, and valued the new insights ENA offered. However, ENA’s abstract visualisation of spatial behaviours was not intuitive, and caused some confusions. We propose, therefore, that the power of ENA modelling can be combined with other spatial representations such as a classroom map, by overlaying annotations to create a more intuitive user experience.
18.
Martinez-Maldonado, Roberto; Gašević, Dragan; Echeverria, Vanessa; Fernandez-Nieto, Gloria; Swiecki, Zachari; Shum, Simon Buckingham
What Do You Mean by Collaboration Analytics? A Conceptual Model Journal Article
In: Journal of Learning Analytics, vol. 8, no. 1, pp. 126–153, 2021, ISSN: 19297750.
@article{Martinez-Maldonado_Gašević_Echeverria_FernandezNieto_Swiecki_BuckinghamShum_2021,
title = {What Do You Mean by Collaboration Analytics? A Conceptual Model},
author = {Roberto Martinez-Maldonado and Dragan Gašević and Vanessa Echeverria and Gloria Fernandez-Nieto and Zachari Swiecki and Simon Buckingham Shum},
url = {https://learning-analytics.info/index.php/JLA/article/view/7227},
doi = {10.18608/jla.2021.7227},
issn = {19297750},
year = {2021},
date = {2021-04-01},
urldate = {2021-04-01},
journal = {Journal of Learning Analytics},
volume = {8},
number = {1},
pages = {126–153},
abstract = {Using data to generate a deeper understanding of collaborative learning is not new, but automatically analyzing log data has enabled new means of identifying key indicators of effective collaboration and teamwork that can be used to predict outcomes and personalize feedback. Collaboration analytics is emerging as a new term to refer to computational methods for identifying salient aspects of collaboration from multiple group data sources for learners, educators, or other stakeholders to gain and act upon insights. Yet, it remains unclear how collaboration analytics go beyond previous work focused on modelling group interactions for the purpose of adapting instruction. This paper provides a conceptual model of collaboration analytics to help researchers and designers identify the opportunities enabled by such innovations to advance knowledge in, and provide enhanced support for, collaborative learning and teamwork. We argue that mapping from low-level data to higher-order constructs that are educationally meaningful, and that can be understood by educators and learners, is essential to assessing the validity of collaboration analytics. Through four cases, the paper illustrates the critical role of theory, task design, and human factors in the design of interfaces that inform actionable insights for improving collaboration and group learning.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Using data to generate a deeper understanding of collaborative learning is not new, but automatically analyzing log data has enabled new means of identifying key indicators of effective collaboration and teamwork that can be used to predict outcomes and personalize feedback. Collaboration analytics is emerging as a new term to refer to computational methods for identifying salient aspects of collaboration from multiple group data sources for learners, educators, or other stakeholders to gain and act upon insights. Yet, it remains unclear how collaboration analytics go beyond previous work focused on modelling group interactions for the purpose of adapting instruction. This paper provides a conceptual model of collaboration analytics to help researchers and designers identify the opportunities enabled by such innovations to advance knowledge in, and provide enhanced support for, collaborative learning and teamwork. We argue that mapping from low-level data to higher-order constructs that are educationally meaningful, and that can be understood by educators and learners, is essential to assessing the validity of collaboration analytics. Through four cases, the paper illustrates the critical role of theory, task design, and human factors in the design of interfaces that inform actionable insights for improving collaboration and group learning.
19.
Fernandez-Nieto, Gloria; Martinez-Maldonado, Roberto; Echeverria, Vanessa; Kitto, Kirsty; An, Pengcheng; Shum, Simon Buckingham
What Can Analytics for Teamwork Proxemics Reveal About Positioning Dynamics In Clinical Simulations? Journal Article
In: Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW1, pp. 1–24, 2021, ISSN: 2573-0142.
@article{Fernandez-Nieto_Martinez-Maldonado_Echeverria_Kitto_An_BuckinghamShum_2021,
title = {What Can Analytics for Teamwork Proxemics Reveal About Positioning Dynamics In Clinical Simulations?},
author = {Gloria Fernandez-Nieto and Roberto Martinez-Maldonado and Vanessa Echeverria and Kirsty Kitto and Pengcheng An and Simon Buckingham Shum},
doi = {10.1145/3449284},
issn = {2573-0142},
year = {2021},
date = {2021-04-01},
journal = {Proceedings of the ACM on Human-Computer Interaction},
volume = {5},
number = {CSCW1},
pages = {1–24},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
