Comparative Evaluation? Yes, But With Which Alternative UI?

User’s feedback provides valuable information suitable to help designers to improve their work. In this paper, we present a study on user’s feedback when evaluating a User Interface (UI) by comparison. Our aim is to define the properties that the alternative UI must satisfy to maximize the benefits of comparative evaluation. The UIs considered in the study were designed using the CAMELEON Reference Framework (CRF), covering variations at each level of abstraction. We study the impact of each variation on users’ feedback. We show that when the alternative design refers to the same task model as the original one but using a different abstract UI, the number of negative returns is significantly higher, making the comparative evaluation more productive.

Content

Author and article information

Contributors

Gaëlle Calvary

Meriem Riahi

Faouiz Moussa

Sara Bouzit

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 1-7

Affiliations

[ ¹ ]Univ.Tunis ElManar, Faculty

of Science of Tunis.

CRISTAL Laboratory

[ ² ]Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, F-38000

Grenoble France

[ ³ ]National Higher Engineering

School of Tunis

Taha Hussein Avenue, Tunis.

Tunisia

[ ⁴ ]CRISTAL Labortory

National School of Computer

Sciences, Mannouba University, Tunisia

Article

DOI: 10.14236/ewic/HCI2017.105

SO-VID: 7d47f9ed-1cd7-47b7-aa16-e234ac666eaf

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts (EVA 2017)

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 11 – 13 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2017.105

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

A. XuBrian P. Bailey 2012 What Do You Think? A Case Study of Benefit, Expectation,and Interaction in a Large Online Critique Community Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work 295 304
B. LeeSavil SrivastavaRanjitha KumarRonen BrafmanScott R Klemmer 2010 Designing with Interactive Example Galleries Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. (CHI '10) 2257 2266
ConcurTaskTrees April 2017 https://en.wikipedia.org/wiki/ConcurTaskTrees
R. S. Dicks 2002 Mis-Usability: On the Uses and Misuses of Usability Testing In Proceedings of the 20th annual international conference on Computer documentation
D. MassonAlexandre DemeureGaelle Calvary 2011 Examples Galleries Generated by Interactive Genetic Algorithms . In procedings of the Second Conference on Creativity and Innovation in Design. 61 71
D.T. NguyenThomas GancarzFelicia NgLaura A. DabbishSteven P. Dow 2017 Fruitful Feedback: Positive Affective Language and Source Anonymity Improve Critique Reception and Work Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17) 1024 1034
G. Calvary Joëlle CoutazDavid TheveninQ. LimbourgL. BouillonJean Vanderdonckt 2003 A Unifying Reference Framework for multi-target user interfaces. Interacting With Computers Vol. 15/3 289 308
G. JohnS. Design Prototypes 1990 A Knowledge Representation Schema for Design. AI Magazine: Volume 11 Issue 4, Winter 1990
G. MeixnerGaëlle CalvaryJoëlle Coutaz Introduction to Model-Based User Interfaces January 2014 https://www.w3.org/TR/mbui-intro/
S.R. HerringC.C. ChangJ. KrantzlerB.P. Bailey 2009 Getting inspired!: understanding how and why examples are used in creative design practice Process of the 27th international conference on Human factors in computing systems. ACM 2009 87 96
J. ManuelJuan M. González CallerosGerrit MeixnerFabio PaternòJaroslav PullmannDave RaggettJean Vanderdonckt Model-Based UI XG Final Report. on April 2010 https://www.w3.org/2007/uwa/editors-drafts/mbui/Model-Based-UI-XG-FinalReport.html
J. HuiAmos GlennRachel JueElizabeth GerberSteven Dow 2015 Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced Feedback Proceeding of Third AAAI Conference on Human Computation and Crowdsourcing HCOMP-15)
K. LutherAmy PavelWei WuJari-lee TolentinoManeesh AgrawalaBjörn HartmannSteven Dow 2014 CrowdCrit: Crowdsourcing and Aggregating Visual Design Critique Proceedings of the companion publication of the 17th ACM conference on Computer supported cooperative work & social computing. 21 24
M. TohidiWilliam BuxtonRonald BaeckerAbigail Sellen 2006 Getting the Right Design and the Design Right: Testing Many Is Better Than One Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI’2006 1243 1252
M. TohidiWilliam BuxtonRonald BaeckerAbigail Sellen 2006 User Sketches: A Quick, Inexpensive, and Effective way to Elicit More Reflective User Feedback Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (NordiCHI’2006) 105 114
N. HiliY. LaurillauS. Dupuy-ChessaG. Calvary 2015 Innovative Key Features for Mastering Model Complexity: FlexiLab, a Multimodel Editor Illustrated on Task Modeling Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems.
M. E. WiklundChristopher ThurrottJoseph S. Dumas 1992 Does the Fidelity of Software Prototypes Affect the Perception of Usability? In procedings of Human Factors Society 36th Annual Meeting, (1992 399 403

Comments

Comment on this article

[1] A. XuBrian P. Bailey 2012 What Do You Think? A Case Study of Benefit, Expectation,and Interaction in a Large Online Critique Community Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work 295 304

[2] B. LeeSavil SrivastavaRanjitha KumarRonen BrafmanScott R Klemmer 2010 Designing with Interactive Example Galleries Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. (CHI '10) 2257 2266

[3] ConcurTaskTrees April 2017 https://en.wikipedia.org/wiki/ConcurTaskTrees

[4] R. S. Dicks 2002 Mis-Usability: On the Uses and Misuses of Usability Testing In Proceedings of the 20th annual international conference on Computer documentation

[5] D. MassonAlexandre DemeureGaelle Calvary 2011 Examples Galleries Generated by Interactive Genetic Algorithms . In procedings of the Second Conference on Creativity and Innovation in Design. 61 71

[6] D.T. NguyenThomas GancarzFelicia NgLaura A. DabbishSteven P. Dow 2017 Fruitful Feedback: Positive Affective Language and Source Anonymity Improve Critique Reception and Work Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17) 1024 1034

[7] G. Calvary Joëlle CoutazDavid TheveninQ. LimbourgL. BouillonJean Vanderdonckt 2003 A Unifying Reference Framework for multi-target user interfaces. Interacting With Computers Vol. 15/3 289 308

[8] G. JohnS. Design Prototypes 1990 A Knowledge Representation Schema for Design. AI Magazine: Volume 11 Issue 4, Winter 1990

[9] G. MeixnerGaëlle CalvaryJoëlle Coutaz Introduction to Model-Based User Interfaces January 2014 https://www.w3.org/TR/mbui-intro/

[10] S.R. HerringC.C. ChangJ. KrantzlerB.P. Bailey 2009 Getting inspired!: understanding how and why examples are used in creative design practice Process of the 27th international conference on Human factors in computing systems. ACM 2009 87 96

[11] J. ManuelJuan M. González CallerosGerrit MeixnerFabio PaternòJaroslav PullmannDave RaggettJean Vanderdonckt Model-Based UI XG Final Report. on April 2010 https://www.w3.org/2007/uwa/editors-drafts/mbui/Model-Based-UI-XG-FinalReport.html

[12] J. HuiAmos GlennRachel JueElizabeth GerberSteven Dow 2015 Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced Feedback Proceeding of Third AAAI Conference on Human Computation and Crowdsourcing HCOMP-15)

[13] K. LutherAmy PavelWei WuJari-lee TolentinoManeesh AgrawalaBjörn HartmannSteven Dow 2014 CrowdCrit: Crowdsourcing and Aggregating Visual Design Critique Proceedings of the companion publication of the 17th ACM conference on Computer supported cooperative work & social computing. 21 24

[14] M. TohidiWilliam BuxtonRonald BaeckerAbigail Sellen 2006 Getting the Right Design and the Design Right: Testing Many Is Better Than One Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI’2006 1243 1252

[15] M. TohidiWilliam BuxtonRonald BaeckerAbigail Sellen 2006 User Sketches: A Quick, Inexpensive, and Effective way to Elicit More Reflective User Feedback Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (NordiCHI’2006) 105 114

[16] N. HiliY. LaurillauS. Dupuy-ChessaG. Calvary 2015 Innovative Key Features for Mastering Model Complexity: FlexiLab, a Multimodel Editor Illustrated on Task Modeling Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems.

[17] M. E. WiklundChristopher ThurrottJoseph S. Dumas 1992 Does the Fidelity of Software Prototypes Affect the Perception of Usability? In procedings of Human Factors Society 36th Annual Meeting, (1992 399 403

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65