Blog
About
Search
Advanced search

117
views
11
references
 Top references
0
cited by
 Most cited
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
1 collections
    0
    shares
      357
      similar
       All similar
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Heterogeneity in the Usability Evaluation Process

      Author(s):

      Publication date (Print):

      Conference name: People and Computers XXII Culture, Creativity, Interaction (HCI)

      Conference theme: Computers XXII Culture, Creativity, Interaction

      Conference date: 1 - 5 September 2008

      Keywords: Usability Evaluation, Five Users Debate, Evaluation, Stochastic Models, Overdispersion, Process Prediction

      Read this article at

      ScienceOpen
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Current prediction models for usability evaluations are based on stochastic distributions derived from series of Bernoulli processes. The underlying assumption of these models is a homogeneous detection probability despite of it being intuitively unrealistic. This paper contributes a simple statistical test for existence of heterogeneity in the process. The compound beta-binomial model is proposed to incorporate sources of heterogeneity and compared to the binomial model. Analysis of several data sets from the literature illustrates the methods and reveals that heterogeneity occurs in most situations. Finally, it is demonstrated how heterogeneity biases the prediction of evaluation processes. Open research questions are discussed and preliminary advice for practitioners for controlling their processes is given.

          Related collections

          Most cited references 11

          • Record: found
          • Abstract: not found
          • Conference Proceedings: not found

          Heuristic evaluation of user interfaces

           Jakob Nielsen,  Rolf Molich (1990)
          Article 0 comments Cited 96 times – based on 0 reviews
            Bookmark
            • Record: found
            • Abstract: not found
            • Conference Proceedings: not found

            A mathematical model of the finding of usability problems

             Jakob Nielsen,  Thomas Landauer (1993)
            Article 0 comments Cited 85 times – based on 0 reviews
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Sample sizes for usability studies: additional considerations.

               J R Lewis (1994)
              Recently, Virzi (1992) presented data that support three claims regarding sample sizes for usability studies: (1) observing four or five participants will allow a usability practitioner to discover 80% of a product's usability problems, (2) observing additional participants will reveal fewer and fewer new usability problems, and (3) more severe usability problems are easier to detect with the first few participants. Results from an independent usability study clearly support the second claim, partially support the first, but fail to support the third. Problem discovery shows diminishing returns as a function of sample size. Observing four to five participants will uncover about 80% of a product's usability problems as long as the average likelihood of problem detection ranges between 0.32 and 0.42, as in Virzi. If the average likelihood of problem detection is lower, then a practitioner will need to observe more than five participants to discover 80% of the problems. Using behavioral categories for problem severity (or impact), these data showed no correlation between problem severity (impact) and rate of discovery. The data provided evidence that the binomial probability formula may provide a good model for predicting problem discovery curves, given an estimate of the average likelihood of problem detection. Finally, data from economic simulations that estimated return on investment (ROI) under a variety of settings showed that only the average likelihood of problem detection strongly influenced the range of sample sizes for maximum ROI.
              Article 0 comments Cited 22 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Contributors
                Martin Schmettow
                Conference
                Publication date: September 2008
                Publication date (Print): September 2008
                Pages: 89-98
                Affiliations
                University of Passau

                Information Systems II

                Innstr. 43

                94032 Passau, Germany
                Article
                DOI: 10.14236/ewic/HCI2008.9
                Copyright statement: © Martin Schmettow. Published by BCS Learning and Development Ltd. People and Computers XXII Culture, Creativity, Interaction

                License:

                This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

                Conference name: People and Computers XXII Culture, Creativity, Interaction
                Conference acronym: HCI
                Conference number: 22
                Conference location: Conference Liverpool John Moores University, UK.
                Conference date: 1 - 5 September 2008
                Conference sponsor: Electronic Workshops in Computing (eWiC)
                Conference theme: Computers XXII Culture, Creativity, Interaction
                Product
                Product Information: 1477-9358BCS Learning & Development
                Self URI (journal page): https://ewic.bcs.org/
                Categories
                Series title: Electronic Workshops in Computing

                Data availability:

                ScienceOpen disciplines: Applied computer science, Computer science, Security & Cryptology, Graphics & Multimedia design, General computer science, Human-computer-interaction

                Keywords: Process Prediction, Usability Evaluation, Five Users Debate, Evaluation, Stochastic Models, Overdispersion

                Comments

                Comment on this article