Blog
About

120
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Effects of Expertise Assessment on the Quality of Task Routing in Human Computation

      , ,

      2nd International Workshop on Social Media for Crowdsourcing and Human Computation (SoHuman 2013) (SOHUMAN)

      Social Media for Crowdsourcing and Human Computation (SoHuman 2013)

      1 May 2013

      Human computation, expertise assessment, task routing

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Human computation systems are characterized by the use of human workers to solve computationally difficult problems. Expertise profiling involves assessment and representation of a worker’s expertise, in order to route human computation tasks to appropriate workers. This paper studies the relationship between the assessment workload on workers and the quality of task routing. Three expertise assessment approaches were compared with the help of a user study, using two different groups of human workers. The first approach requests workers to provide self-assessment of their knowledge. The second approach measures the knowledge of workers through their performance against tasks with known responses. We propose a third approach based on a combination of self-assessment and task-assessment. The results suggest that the self-assessment approach requires minimum assessment workload from workers during expertise profiling. By comparison, the task-assessment approach achieved the highest response rate and accuracy. The proposed approach requires less assessment workload, while achieving the response rate and accuracy similar to the task-assessment approach.

          Related collections

          Most cited references 9

          • Record: found
          • Abstract: not found
          • Conference Proceedings: not found

          Labeling images with a computer game

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Linked Data: Evolving the Web into a Global Data Space

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Human Computation

               Edith Law,  Luis Ahn (2011)
                Bookmark

                Author and article information

                Contributors
                Conference
                May 2013
                May 2013
                : 1-10
                Affiliations
                Digital Enterprise

                Research Institute

                National University of Ireland

                Galway
                Article
                10.14236/ewic/SOHUMAN2013.2
                © Umair ul Hassan et al. Published by BCS Learning and Development Ltd. 2nd International Workshop on Social Media for Crowdsourcing and Human Computation (SoHuman 2013), Paris, France

                This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

                2nd International Workshop on Social Media for Crowdsourcing and Human Computation (SoHuman 2013)
                SOHUMAN
                2
                Paris, France
                1 May 2013
                Electronic Workshops in Computing (eWiC)
                Social Media for Crowdsourcing and Human Computation (SoHuman 2013)
                Product
                Product Information: 1477-9358BCS Learning & Development
                Self URI (journal page): https://ewic.bcs.org/
                Categories
                Electronic Workshops in Computing

                Comments

                Comment on this article