Designing a Peer Assessment for Identification of Students’ Group Work Problems

Group projects are part of the core educational experience in higher education, but many students report bad experiences. Group problems may undermine learning and cause stress and frustration. This may be prevented by monitoring and supporting groups, but this is often not feasible for teachers, who lack time and resources. This research aims to find a method for early identification of group work problems via computer-supported assessment. First, interviews and focus groups provided insights into the most common group problems and which visual features students preferred in a peer assessment. Next, two assessment versions were created: a simple, time-efficient version, and a more engaging, interactive one. We also created an initial version of E-Mate, a virtual agent that provides initial feedback on the assessment. These were tested in a field study. Most students reported a positive experience with the peer assessment, regardless of the visualization used. Teachers were also positive about its usefulness. The research also supports the use of five attributes to assess group collaboration.

Content

Author and article information

Contributors

Danny Veth

Isabella Saccardi

Judith Masthoff

Conference

Publication date: July 2022

Publication date (Print): July 2022

Pages: 1-10

Affiliations

[0001]Utrecht University, the Netherlands

Article

DOI: 10.14236/ewic/HCI2022.12

SO-VID: 903dc9e5-84bb-4c44-80a4-fc550a43681d

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 35th International BCS Human-Computer Interaction Conference

Conference acronym: HCI2022

Conference number: 35

Conference location: Keele, Staffordshire

Conference date: July 11th to 13th, 2022

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Towards a Human-Centred Digital Society

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2022.12

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

Adeniran, A. (2020), Investigating Real-Time Assessment and Support for Online Collaborative Learning, PhD thesis, University of Aberdeen.
Badea, G. and Popescu, E. (2019), Instructor support module in a web-based peer assessment platform, in ‘Proc. System Theory, Control and Computing’, IEEE, pp. 691–696.
Bales, R. (1988), Overview of the SYMLOG system: Measuring and changing behavior in groups, SYMLOG Consulting Group Woodland Hills, CA.
Bamberger, P. A., Erev, I., Kimmel, M. and Oref-Chen, T. (2005), ‘Peer assessment, individual performance, and contribution to group processes: The impact of rater anonymity’, Group & Organization Management 30(4), 344–377.
Barfield, R. L. (2003), ‘Students’ perceptions of and satisfaction with group grades and the group experience in the college classroom’, Assessment & Evaluation in Higher Education 28(4), 355–370.
Burdett, J. (2003), ‘Making groups work: University students’ perceptions’, International Education Journal 4(3), 177–191.
Cervone, H. F. (2014), ‘Effective communication for project success’, OCLC Systems and Services: International digital library perspectives .
Colbeck, C. L., Campbell, S. E. and Bjorklund, S. A. (2000), ‘Grouping in the dark: What college students learn from group projects’, The Journal of Higher Education 71(1), 60–83.
Craig, T. Y. and Kelly, J. R. (1999), ‘Group cohesiveness and creative performance.’, Group dynamics: Theory, research, and practice 3(4), 243.
Dahlin, K. B., Weingart, L. R. and Hinds, P. J. (2005), ‘Team diversity and information use’, Academy of management journal 48(6), 1107–1123.
Emans, B., Koopman, P., Rutte, C. and Steensma, H. (1996), ‘Teams in organisaties’, Gedrag en Organisatie 6, 309–327.
Freeman, M. and McKenzie, J. (2002), ‘Spark, a confidential web–based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects’, British journal of educational technology 33(5), 551–569.
Garandeau, C. F., Lee, I. A. and Salmivalli, C. (2014), ‘Inequality matters: Classroom status hierarchy and adolescents’ bullying’, Journal of youth and adolescence 43(7), 1123–1133.
Go, E. and Sundar, S. S. (2019), ‘Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions’, Computers in Human Behavior 97, 304–316.
Harrison, D. A. and Klein, K. J. (2007), ‘What’s the difference? diversity constructs as separation, variety, or disparity in organizations’, Academy of management review 32(4), 1199–1228.
Huang, C.-C. (2009), ‘Knowledge sharing and group cohesiveness on performance: An empirical study of technology r&d teams in taiwan’, Technovation 29(11), 786–797.
Ingham, A. G., Levinger, G., Graves, J. and Peckham, V. (1974), ‘The ringelmann effect: Studies of group size and group performance’, Journal of experimental social psychology 10(4), 371–384.
Issa, T. (2012), ‘Promoting learning skills through teamwork assessment and self/peer evaluation in higher education.’, International Association for Development of the Information Society .
Karau, S. J. and Williams, K. D. (1993), ‘Social loafing: A meta-analytic review and theoretical integration.’, Journal of personality and social psychology 65(4), 681.
Latané, B., Williams, K. and Harkins, S. (1979), ‘Many hands make light the work: The causes and consequences of social loafing.’, Journal of personality and social psychology 37(6), 822.
Li, L. (2017), ‘The role of anonymity in peer assessment’, Assessment & Evaluation in Higher Education 42(4), 645–656.
Liu, L. A., Chua, C. H. and Stahl, G. K. (2010), ‘Quality of communication experience: Definition, measurement, and implications for intercultural negotiations.’, J. of Applied Psychology 95(3), 469.
Liu, S., Zheng, C., Demasi, O., Sabour, S., Li, Y., Yu, Z., Jiang, Y. and Huang, M. (2021), ‘Towards emotional support dialog systems’, arXiv preprint arXiv:2106.01144.
Loddington, S., Pond, K., Wilkinson, N. and Willmot, P. (2009), ‘A case study of the development of webpa: An online peer-moderated marking tool’, British J. of Educational Tech. 40(2), 329–341.
Lolli, J. C. (2013), ‘Interpersonal communication skills and the young hospitality leader: Are they prepared?’, Int. J. of hospitality management 32, 295–298.
Mackie, D. M. and Goethals, G. R. (1987), Individual and group goals., in C. Hendrix, ed., ‘Group processes’, Sage Publications, Inc.
MacMillan, J., Entin, E. E. and Serfaty, D. (2004), Communication overhead: The hidden cost of team cognition., in E. Salas and S. M. Fiore, eds, ‘Team cognition: Understanding the factor that drive process and performance’, APA, pp. 61–82.
Martinazzi, R. (1998), Design and development of a peer evaluation instrument for” student learning teams”, in ‘Proc. of Frontiers in Education Conference.’, Vol. 2, IEEE, pp. 784–789.
Murray, J.-A. and Boyd, S. (2015), ‘A preliminary evaluation of using webpa for online peer assessment of collaborative performance by groups of online distance learners.’, Int. Journal of E-Learning & Distance Education 30(2), n2.
Phielix, C., Prins, F. J., Kirschner, P. A., Erkens, G. and Jaspers, J. (2011), ‘Group awareness of social and cognitive performance in a cscl environment: Effects of a peer feedback and reflection tool’, Computers in human behavior 27(3), 1087–1102.
Ringelmann, M. (1913), Recherches sur les moteurs animés. Travail de l’homme., Annales de l’Institut national agronomique.
Roberts, T. S. and McInnerney, J. M. (2007), ‘Seven problems of online group learning (and their solutions)’, Journal of Educational Technology & Society 10(4), 257–268.
Schoenecker, T. S., Martell, K. D. and Michlitsch, J. F. (1997), ‘Diversity, performance, and satisfaction in student group projects: An empirical study’, Research in Higher Education 38(4), 479–495.
Sharma, M. D., Khachan, J., Chan, B. and O’Byrne, J. (2005), ‘An investigation of the effectiveness of electronic classroom communication systems in large lecture classes’, Australasian Journal of Educational Technology 21(2).
Smith, K. A. and Masthoff, J. (2018), Can a Virtual Agent provide good Emotional Support?, in ‘Proc. British Human Computer Interaction’, pp. 1–10.
Tost, L. P., Gino, F. and Larrick, R. P. (2013), ‘When power makes others speechless: The negative impact of leader power on team performance’, Academy of Management J. 56(5), 1465–1486.
Wildman, R. C. (1977), ‘Effects of anonymity and social setting on survey responses’, Public Opinion Quarterly 41(1), 74–79.
Williams, K., Harkins, S. G. and Latané, B. (1981), ‘Identifiability as a deterrant to social loafing: Two cheering experiments.’, Journal of Personality and Social Psychology 40(2), 303.
Wubbels, T. et al. (1985), Discipline problems of beginning teachers, interactional teacher behaviour mapped out., in ‘Resources in Education’, ERIC.

Comments

Comment on this article

[1] Adeniran, A. (2020), Investigating Real-Time Assessment and Support for Online Collaborative Learning, PhD thesis, University of Aberdeen.

[2] Badea, G. and Popescu, E. (2019), Instructor support module in a web-based peer assessment platform, in ‘Proc. System Theory, Control and Computing’, IEEE, pp. 691–696.

[3] Bales, R. (1988), Overview of the SYMLOG system: Measuring and changing behavior in groups, SYMLOG Consulting Group Woodland Hills, CA.

[4] Bamberger, P. A., Erev, I., Kimmel, M. and Oref-Chen, T. (2005), ‘Peer assessment, individual performance, and contribution to group processes: The impact of rater anonymity’, Group & Organization Management 30(4), 344–377.

[5] Barfield, R. L. (2003), ‘Students’ perceptions of and satisfaction with group grades and the group experience in the college classroom’, Assessment & Evaluation in Higher Education 28(4), 355–370.

[6] Burdett, J. (2003), ‘Making groups work: University students’ perceptions’, International Education Journal 4(3), 177–191.

[7] Cervone, H. F. (2014), ‘Effective communication for project success’, OCLC Systems and Services: International digital library perspectives .

[8] Colbeck, C. L., Campbell, S. E. and Bjorklund, S. A. (2000), ‘Grouping in the dark: What college students learn from group projects’, The Journal of Higher Education 71(1), 60–83.

[9] Craig, T. Y. and Kelly, J. R. (1999), ‘Group cohesiveness and creative performance.’, Group dynamics: Theory, research, and practice 3(4), 243.

[10] Dahlin, K. B., Weingart, L. R. and Hinds, P. J. (2005), ‘Team diversity and information use’, Academy of management journal 48(6), 1107–1123.

[11] Emans, B., Koopman, P., Rutte, C. and Steensma, H. (1996), ‘Teams in organisaties’, Gedrag en Organisatie 6, 309–327.

[12] Freeman, M. and McKenzie, J. (2002), ‘Spark, a confidential web–based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects’, British journal of educational technology 33(5), 551–569.

[13] Garandeau, C. F., Lee, I. A. and Salmivalli, C. (2014), ‘Inequality matters: Classroom status hierarchy and adolescents’ bullying’, Journal of youth and adolescence 43(7), 1123–1133.

[14] Go, E. and Sundar, S. S. (2019), ‘Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions’, Computers in Human Behavior 97, 304–316.

[15] Harrison, D. A. and Klein, K. J. (2007), ‘What’s the difference? diversity constructs as separation, variety, or disparity in organizations’, Academy of management review 32(4), 1199–1228.

[16] Huang, C.-C. (2009), ‘Knowledge sharing and group cohesiveness on performance: An empirical study of technology r&d teams in taiwan’, Technovation 29(11), 786–797.

[17] Ingham, A. G., Levinger, G., Graves, J. and Peckham, V. (1974), ‘The ringelmann effect: Studies of group size and group performance’, Journal of experimental social psychology 10(4), 371–384.

[18] Issa, T. (2012), ‘Promoting learning skills through teamwork assessment and self/peer evaluation in higher education.’, International Association for Development of the Information Society .

[19] Karau, S. J. and Williams, K. D. (1993), ‘Social loafing: A meta-analytic review and theoretical integration.’, Journal of personality and social psychology 65(4), 681.

[20] Latané, B., Williams, K. and Harkins, S. (1979), ‘Many hands make light the work: The causes and consequences of social loafing.’, Journal of personality and social psychology 37(6), 822.

[21] Li, L. (2017), ‘The role of anonymity in peer assessment’, Assessment & Evaluation in Higher Education 42(4), 645–656.

[22] Liu, L. A., Chua, C. H. and Stahl, G. K. (2010), ‘Quality of communication experience: Definition, measurement, and implications for intercultural negotiations.’, J. of Applied Psychology 95(3), 469.

[23] Liu, S., Zheng, C., Demasi, O., Sabour, S., Li, Y., Yu, Z., Jiang, Y. and Huang, M. (2021), ‘Towards emotional support dialog systems’, arXiv preprint arXiv:2106.01144.

[24] Loddington, S., Pond, K., Wilkinson, N. and Willmot, P. (2009), ‘A case study of the development of webpa: An online peer-moderated marking tool’, British J. of Educational Tech. 40(2), 329–341.

[25] Lolli, J. C. (2013), ‘Interpersonal communication skills and the young hospitality leader: Are they prepared?’, Int. J. of hospitality management 32, 295–298.

[26] Mackie, D. M. and Goethals, G. R. (1987), Individual and group goals., in C. Hendrix, ed., ‘Group processes’, Sage Publications, Inc.

[27] MacMillan, J., Entin, E. E. and Serfaty, D. (2004), Communication overhead: The hidden cost of team cognition., in E. Salas and S. M. Fiore, eds, ‘Team cognition: Understanding the factor that drive process and performance’, APA, pp. 61–82.

[28] Martinazzi, R. (1998), Design and development of a peer evaluation instrument for” student learning teams”, in ‘Proc. of Frontiers in Education Conference.’, Vol. 2, IEEE, pp. 784–789.

[29] Murray, J.-A. and Boyd, S. (2015), ‘A preliminary evaluation of using webpa for online peer assessment of collaborative performance by groups of online distance learners.’, Int. Journal of E-Learning & Distance Education 30(2), n2.

[30] Phielix, C., Prins, F. J., Kirschner, P. A., Erkens, G. and Jaspers, J. (2011), ‘Group awareness of social and cognitive performance in a cscl environment: Effects of a peer feedback and reflection tool’, Computers in human behavior 27(3), 1087–1102.

[31] Ringelmann, M. (1913), Recherches sur les moteurs animés. Travail de l’homme., Annales de l’Institut national agronomique.

[32] Roberts, T. S. and McInnerney, J. M. (2007), ‘Seven problems of online group learning (and their solutions)’, Journal of Educational Technology & Society 10(4), 257–268.

[33] Schoenecker, T. S., Martell, K. D. and Michlitsch, J. F. (1997), ‘Diversity, performance, and satisfaction in student group projects: An empirical study’, Research in Higher Education 38(4), 479–495.

[34] Sharma, M. D., Khachan, J., Chan, B. and O’Byrne, J. (2005), ‘An investigation of the effectiveness of electronic classroom communication systems in large lecture classes’, Australasian Journal of Educational Technology 21(2).

[35] Smith, K. A. and Masthoff, J. (2018), Can a Virtual Agent provide good Emotional Support?, in ‘Proc. British Human Computer Interaction’, pp. 1–10.

[36] Tost, L. P., Gino, F. and Larrick, R. P. (2013), ‘When power makes others speechless: The negative impact of leader power on team performance’, Academy of Management J. 56(5), 1465–1486.

[37] Wildman, R. C. (1977), ‘Effects of anonymity and social setting on survey responses’, Public Opinion Quarterly 41(1), 74–79.

[38] Williams, K., Harkins, S. G. and Latané, B. (1981), ‘Identifiability as a deterrant to social loafing: Two cheering experiments.’, Journal of Personality and Social Psychology 40(2), 303.

[39] Wubbels, T. et al. (1985), Discipline problems of beginning teachers, interactional teacher behaviour mapped out., in ‘Resources in Education’, ERIC.

Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org