Comparing Paper and Software Tool for Participatory Design from Scratch

Several paper-based approaches and software tools have been developed to support Participatory Design (PD) in research and practice. However, a formal comparison of paper and software tool is missing in the PD literature. To address this gap we present an empirical study with 28 Informatics students comparing a paper-based approach with our digital tool PDotCapturer. Results show a slight advantage of paper regarding quantitative results and a significant statistical difference in one of the 18 qualities tested (regarding aesthetics [e.g. symmetrical, creative], usability [e.g. user control and freedom, flexibility and efficiency of use], and relevancy [appropriate or not]): designs created with the tool are more ‘Pleasant’. Subjectively the participants preferred paper for some of the activities and the tool for others. The results of our comparison show that the tool could be used instead of paper to benefit from some tool-advantages over paper (e.g. digital data gathering and analysis support).

Content

Author and article information

Contributors

Matthias Heintz

Effie Lai-Chong Law

Nervo Verdezoto

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 1-12

Affiliations

[0001]University of Leicester

University Road, Leicester, LE1 7RH, UK

Article

DOI: 10.14236/ewic/HCI2017.51

SO-VID: d1816f7d-a314-414a-975e-95433cfb30fc

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts (EVA 2017)

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 11 – 13 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2017.51

Self URI (journal page): https://ewic.bcs.org/

References

J. Brooke 1996 Sus-a quick and dirty usability scale Usability evaluation in industry 189 194 4 7
P. CamposN. Nunes 2007 Towards useful and usable interaction design tools: Canonsketch Interacting with Computers 19 56 597 613
E. CarmelR.D. WhitakerJ.F. George 1993 PD and joint application design: a transatlantic comparison Communications of the ACM 36 6 40 48
J.M. CarrollG. ChinM.B. RossonD.C. Neale 2000 The development of cooperation: five years of participatory design in the virtual school Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, DIS '00. Brooklyn 17-19 08 New York: ACM. 239251
A. ClementP. Van den Besselaar 1993 A retrospective look at PD projects Communications of the ACM 36 6 29 37
K. DanielssonA.M. NaghshD. GummA. Warr 2008 Distributed participatory design CHI'08 extended abstracts on Human factors in computing systems Florence 05-10 04 2008 New York: ACM 3953 3956
A. Druin 2002 The role of children in the design of new technology Behaviour & Information Technology 21 1 1 25
A. DruinB.B. BedersonJ.P. HourcadeL. ShermanG. RevelleM. PlatnerS. Weng 2001 Designing a digital library for young children Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL '01 Roanoke 24-28 06 2001 New York: ACM 398 405
J.H. GennariM. Reddy 2000 Participatory design and an eligibility screening tool Proceedings of the AMIA symposium Los Angeles 04-08 11 2000 American Medical Informatics Association 290 294
J.J. Gibson 1979 The Ecological Approach to Visual Perception New York:Houghton Mifflin
J. Greenbaum 1993 A design of one's own: towards participatory design in the united states Participatory design: Principles and practices,1993 27 37
M. L. GuhaA. DruinG. ChipmanJ. A. FailsS. SimmsA. Farber 2004 Mixing ideas: a new technique for working with young children as design partners Proceedings of the 2004 conference on Interaction design and children: building a community Baltimore 01-03 06 2004 New York: ACM 35 42
D.C. Gumm 2006 Distributed participatory design: An inherent paradoxon? Proceedings of IRIS29
K. HalskovN. B. Hansen 2015 The diversity of participatory design research practice at PDC 2002-2012 International Journal of HumanComputer Studies 74 81 92
M. HeintzE.L.-C. LawS. GovaertsA. HolzerD. Gillet 2014 Pdot: participatory design online tool CHI'14 Extended Abstracts on Human Factors in Computing Systems Toronto 26 04 - 01 05 2014 New York: ACM 2581 2586
S.R. KellyE. MazzoneM. HortonJ.C. Read 2006 Bluebells: a design method for childcentred product development Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles Oslo 14-18 10 2006 New York: ACM 361 368
Y-K. LimE. StoltermanJ. Tenenber 2008 The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas ACM Transactions on Computer-Human Interaction (TOCHI) 15 2 7:1 7:27
Z. LiuD.G. Stork 2000 Is paperless really more? Communications of the ACM 43 11 94 97
S. LohmannJ. ZieglerP. Heim 2008 Involving end users in distributed requirements engineering Engineering Interactive Systems Springer 221 228
L. MaroisJ.E. VialletF. PoirierC. Chauvin 2010 Experimenting introductory tools for innovation and participatory design Proceedings of the 11th Biennial Participatory Design Conference Sydney 19 11 - 03 12 2010 New York: ACM 259 262
N. MoravejiJ. LiJ. DingP. O'KelleyS. Woolf 2007 Comicboarding: Using comics as proxies for participatory design with children Proceedings of the SIGCHI Conference on Human Factors in Computing Systems San Jose 30 04 - 03 05 2007 New York: ACM 1371 1374
M.J. Muller 1991 Pictive-an exploration in participatory design Proceedings of the SIGCHI Conference on Human Factors in Computing Systems New Orleans 27 04 - 02 05 1991 New York: ACM 225 231
M.J. MullerD.M. WildmanE.A. White 1993 Taxonomy of PD practices: A brief practitioner's guide Communications of the ACM 36 6 26 28
A.M. NaghshD. Andy 2004 Gabbeh: a tool to support collaboration in electronic paper prototyping CSCW 2004 the ACM Conference on Computer Supported Cooperative Work Chicago 06-10 11 2004 New York: ACM
M.W. NewmanJ. LinJ.I. HongJ.A. Landay 2003 DENIM: An informal web site design tool inspired by observations of practice HumanComputer Interaction 18 3 259 324
J. Nielsen 1994 Heuristic evaluation Usability inspection methods 17 1 25 62
T. PoratN. Tractinsky 2012 It's a pleasure buying here: The effects of webstore design on consumers' emotions and attitudes HumanComputer Interaction 27 3 235 276
K. Porayska-PomstaC. FrauenbergerH. PainG. RajendranT. SmithR. MenziesM.E. FosterA. AlcornS. WassS. BernadiniK. AvramidesW. Keay-BrightJ. ChenA. WallerK. GuldbergJ. GoodO. Lemon 2012 Developing technology for autism: an interdisciplinary approach Personal and Ubiquitous Computing 16 2 117 127
S.K. PuriE. ByrneJ.L. NhampossaZ.B. Quraishi 2004 Contextuality of participation in is design: A developing country perspective Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices - Volume 1, PDC 04 Toronto 27-31 07 2004 New York: ACM 42 52
J.C. ReadD. FittonG. SimM. Horton 2016 How ideas make it through to designs: Process and practice Proceedings of the 9th Nordic Conference on Human-Computer Interaction, NordiCHI '16 Gothenburg 23-27 10 2016 New York: ACM 16:1 16:10
E.B.-N. SandersE. BrandtT. Binder 2010 A framework for organizing the tools and techniques of participatory design Proceedings of the 11th Biennial Participatory Design Conference Sydney 19 11 - 03 12 2010 New York: ACM 195 198
E.B.-N. SandersP.J. Stappers 2014 Probes, toolkits and prototypes: three approaches to making in codesigning CoDesign: International Journal of CoCreation in Design and the Arts 10 1 5 14
A.J. SellenR.H. Harper 2003 The myth of the paperless office MIT press
D. SvanaesG. Seland 2004 Putting the users center stage: Role playing and low-prototyping enable end users to design mobile systems Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '04 Vienna 24-29 04 2004 New York: ACM 479 486
G. Walsh 2011 Distributed participatory design CHI'11 Extended Abstracts on Human Factors in Computing Systems Vancouver 07-12 05 2011 New York: ACM 1061 1064
G. WalshA. DruinM.L. GuhaE. BonsignoreE. FossJ. C. YipE. GolubT. CleggQ. BrownR. Brewer et al 2012 Disco: a co-design online tool for asynchronous distributed child and adult design partners Proceedings of the 11th International Conference on Interaction Design and Children Bremen 12-15 06 2012 New York: ACM 11 19
G. WalshA. DruinM.L. GuhaE. FossE. GolubL. HatleyE. BonsignoreS. Franckel 2010 Layered elaboration: a new technique for co-design with children Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Atlanta 10-15 04 2010 New York: ACM 1237 1240
G. WalshE. Foss 2015 A case for intergenerational distributed co-design: The online kidsteam example Proceedings of the 14th International Conference on Interaction Design and Children Medford 21-25 06 2015 New York: ACM 99 108
G. WalshE. FossJ. YipA. Druin 2013 FACIT PD: a framework for analysis and creation of intergenerational techniques for participatory design Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Paris 27 04 - 02 05 2013 New York: ACM 2893 2902

Comments

Comment on this article

[1] J. Brooke 1996 Sus-a quick and dirty usability scale Usability evaluation in industry 189 194 4 7

[2] P. CamposN. Nunes 2007 Towards useful and usable interaction design tools: Canonsketch Interacting with Computers 19 56 597 613

[3] E. CarmelR.D. WhitakerJ.F. George 1993 PD and joint application design: a transatlantic comparison Communications of the ACM 36 6 40 48

[4] J.M. CarrollG. ChinM.B. RossonD.C. Neale 2000 The development of cooperation: five years of participatory design in the virtual school Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, DIS '00. Brooklyn 17-19 08 New York: ACM. 239251

[5] A. ClementP. Van den Besselaar 1993 A retrospective look at PD projects Communications of the ACM 36 6 29 37

[6] K. DanielssonA.M. NaghshD. GummA. Warr 2008 Distributed participatory design CHI'08 extended abstracts on Human factors in computing systems Florence 05-10 04 2008 New York: ACM 3953 3956

[7] A. Druin 2002 The role of children in the design of new technology Behaviour & Information Technology 21 1 1 25

[8] A. DruinB.B. BedersonJ.P. HourcadeL. ShermanG. RevelleM. PlatnerS. Weng 2001 Designing a digital library for young children Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL '01 Roanoke 24-28 06 2001 New York: ACM 398 405

[9] J.H. GennariM. Reddy 2000 Participatory design and an eligibility screening tool Proceedings of the AMIA symposium Los Angeles 04-08 11 2000 American Medical Informatics Association 290 294

[10] J.J. Gibson 1979 The Ecological Approach to Visual Perception New York:Houghton Mifflin

[11] J. Greenbaum 1993 A design of one's own: towards participatory design in the united states Participatory design: Principles and practices,1993 27 37

[12] M. L. GuhaA. DruinG. ChipmanJ. A. FailsS. SimmsA. Farber 2004 Mixing ideas: a new technique for working with young children as design partners Proceedings of the 2004 conference on Interaction design and children: building a community Baltimore 01-03 06 2004 New York: ACM 35 42

[13] D.C. Gumm 2006 Distributed participatory design: An inherent paradoxon? Proceedings of IRIS29

[14] K. HalskovN. B. Hansen 2015 The diversity of participatory design research practice at PDC 2002-2012 International Journal of HumanComputer Studies 74 81 92

[15] M. HeintzE.L.-C. LawS. GovaertsA. HolzerD. Gillet 2014 Pdot: participatory design online tool CHI'14 Extended Abstracts on Human Factors in Computing Systems Toronto 26 04 - 01 05 2014 New York: ACM 2581 2586

[16] S.R. KellyE. MazzoneM. HortonJ.C. Read 2006 Bluebells: a design method for childcentred product development Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles Oslo 14-18 10 2006 New York: ACM 361 368

[17] Y-K. LimE. StoltermanJ. Tenenber 2008 The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas ACM Transactions on Computer-Human Interaction (TOCHI) 15 2 7:1 7:27

[18] Z. LiuD.G. Stork 2000 Is paperless really more? Communications of the ACM 43 11 94 97

[19] S. LohmannJ. ZieglerP. Heim 2008 Involving end users in distributed requirements engineering Engineering Interactive Systems Springer 221 228

[20] L. MaroisJ.E. VialletF. PoirierC. Chauvin 2010 Experimenting introductory tools for innovation and participatory design Proceedings of the 11th Biennial Participatory Design Conference Sydney 19 11 - 03 12 2010 New York: ACM 259 262

[21] N. MoravejiJ. LiJ. DingP. O'KelleyS. Woolf 2007 Comicboarding: Using comics as proxies for participatory design with children Proceedings of the SIGCHI Conference on Human Factors in Computing Systems San Jose 30 04 - 03 05 2007 New York: ACM 1371 1374

[22] M.J. Muller 1991 Pictive-an exploration in participatory design Proceedings of the SIGCHI Conference on Human Factors in Computing Systems New Orleans 27 04 - 02 05 1991 New York: ACM 225 231

[23] M.J. MullerD.M. WildmanE.A. White 1993 Taxonomy of PD practices: A brief practitioner's guide Communications of the ACM 36 6 26 28

[24] A.M. NaghshD. Andy 2004 Gabbeh: a tool to support collaboration in electronic paper prototyping CSCW 2004 the ACM Conference on Computer Supported Cooperative Work Chicago 06-10 11 2004 New York: ACM

[25] M.W. NewmanJ. LinJ.I. HongJ.A. Landay 2003 DENIM: An informal web site design tool inspired by observations of practice HumanComputer Interaction 18 3 259 324

[26] J. Nielsen 1994 Heuristic evaluation Usability inspection methods 17 1 25 62

[27] T. PoratN. Tractinsky 2012 It's a pleasure buying here: The effects of webstore design on consumers' emotions and attitudes HumanComputer Interaction 27 3 235 276

[28] K. Porayska-PomstaC. FrauenbergerH. PainG. RajendranT. SmithR. MenziesM.E. FosterA. AlcornS. WassS. BernadiniK. AvramidesW. Keay-BrightJ. ChenA. WallerK. GuldbergJ. GoodO. Lemon 2012 Developing technology for autism: an interdisciplinary approach Personal and Ubiquitous Computing 16 2 117 127

[29] S.K. PuriE. ByrneJ.L. NhampossaZ.B. Quraishi 2004 Contextuality of participation in is design: A developing country perspective Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices - Volume 1, PDC 04 Toronto 27-31 07 2004 New York: ACM 42 52

[30] J.C. ReadD. FittonG. SimM. Horton 2016 How ideas make it through to designs: Process and practice Proceedings of the 9th Nordic Conference on Human-Computer Interaction, NordiCHI '16 Gothenburg 23-27 10 2016 New York: ACM 16:1 16:10

[31] E.B.-N. SandersE. BrandtT. Binder 2010 A framework for organizing the tools and techniques of participatory design Proceedings of the 11th Biennial Participatory Design Conference Sydney 19 11 - 03 12 2010 New York: ACM 195 198

[32] E.B.-N. SandersP.J. Stappers 2014 Probes, toolkits and prototypes: three approaches to making in codesigning CoDesign: International Journal of CoCreation in Design and the Arts 10 1 5 14

[33] A.J. SellenR.H. Harper 2003 The myth of the paperless office MIT press

[34] D. SvanaesG. Seland 2004 Putting the users center stage: Role playing and low-prototyping enable end users to design mobile systems Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '04 Vienna 24-29 04 2004 New York: ACM 479 486

[35] G. Walsh 2011 Distributed participatory design CHI'11 Extended Abstracts on Human Factors in Computing Systems Vancouver 07-12 05 2011 New York: ACM 1061 1064

[36] G. WalshA. DruinM.L. GuhaE. BonsignoreE. FossJ. C. YipE. GolubT. CleggQ. BrownR. Brewer et al 2012 Disco: a co-design online tool for asynchronous distributed child and adult design partners Proceedings of the 11th International Conference on Interaction Design and Children Bremen 12-15 06 2012 New York: ACM 11 19

[37] G. WalshA. DruinM.L. GuhaE. FossE. GolubL. HatleyE. BonsignoreS. Franckel 2010 Layered elaboration: a new technique for co-design with children Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Atlanta 10-15 04 2010 New York: ACM 1237 1240

[38] G. WalshE. Foss 2015 A case for intergenerational distributed co-design: The online kidsteam example Proceedings of the 14th International Conference on Interaction Design and Children Medford 21-25 06 2015 New York: ACM 99 108

[39] G. WalshE. FossJ. YipA. Druin 2013 FACIT PD: a framework for analysis and creation of intergenerational techniques for participatory design Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Paris 27 04 - 02 05 2013 New York: ACM 2893 2902

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65