Blog
About

112
views
0
recommends
+1 Recommend
1 collections
    4
    shares
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      How good is this conference? Evaluating conference reviewing and selectivity

      ,

      The 26th BCS Conference on Human Computer Interaction (HCI)

      Human Computer Interaction

      12 - 14 September 2012

      Peer review, selectivity, reviewer feedback, statistical modelling, audience participation

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Peer reviewing of papers is the mainstay of modern academic publishing but it has well known problems. In this paper, we take a statistical modelling view to show a particular problem in the use of selectivity measures to indicate the quality of a conference. One key problem with the process of conference reviewing is the failure to make a useful feedback loop between the referees of the papers accepted at the conference and their importance, acceptance and relevance to the audience. In addition, we make some new criticisms of selectivity as a measure of quality.

          This paper is literally a work in progress because the 2012 BCS HCI itself conference will be used to close the feedback loop by making the connection between the reviews provided on papers and your (audience) perceptions of the papers. At the conference, participants will generate the results of this work.

          Related collections

          Most cited references 1

          • Record: found
          • Abstract: found
          • Article: not found

          Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments.

          People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.
            Bookmark

            Author and article information

            Contributors
            Conference
            September 2012
            September 2012
            : 410-415
            Affiliations
            Department of Computing Science

            University of Swansea, Wales
            Department of Computer Science

            University of York, England
            Article
            10.14236/ewic/HCI2012.3
            © Harold Thimbleby et al. Published by BCS Learning and Development Ltd. The 26th BCS Conference on Human Computer Interaction, Birmingham, UK

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            The 26th BCS Conference on Human Computer Interaction
            HCI
            26
            Birmingham, UK
            12 - 14 September 2012
            Electronic Workshops in Computing (eWiC)
            Human Computer Interaction
            Product
            Product Information: 1477-9358BCS Learning & Development
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Comments

            Comment on this article