+1 Recommend
1 collections
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      An Evaluation of Quality Checklist Proposals - A participant-observer case study

      , , ,

      13th International Conference on Evaluation and Assessment in Software Engineering (EASE) (EASE)

      Evaluation and Assessment in Software Engineering (EASE)

      20 - 21 April 2009

      case study, systematic literature review, primary study quality, quality checklists, guidelines evaluation

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          Background: A recent set of guidelines for software engineering systematic literature reviews (SLRs) includes a list of quality criteria obtained from the literature. The guidelines suggest that the list can be used to construct a tailored set of questions to evaluate the quality of primary studies. Aim: This paper aims to evaluate whether the list of quality criteria help researchers construct tailored quality checklists. Method: We undertook a participant-observer case study to investigate the list of quality criteria. The “case” in this study was the planning stage of a systematic literature review on unit testing. Results: The checklists in our SLR guidelines do not provide sufficient help with the construction of a quality checklist for a specific SLR either for novices or for experienced researchers. However, the checklists are reasonably complete and lead to the use of a common terminology for quality questions selected for a specific systematic literature review. Conclusions: The guidelines document should be amended to include a much shorter generic checklist. Researchers might find it useful to adopt a team-based process for quality checklist construction and provide suggestions for answering quality checklist questions.

          Related collections

          Author and article information

          April 2009
          April 2009
          : 1-10
          School of Computing and Mathematics, Keele University, Keele, Staffordshire ST5 5BG, UK
          Department of Computer Science, Durham University, Durham, Science Laboratories

          South Road, Durham City, DH1 3LE, UK
          © Barbara A. Kitchenham et al. Published by BCS Learning and Development Ltd. 13th International Conference on Evaluation and Assessment in Software Engineering (EASE), Durham University, UK

          This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit

          13th International Conference on Evaluation and Assessment in Software Engineering (EASE)
          Durham University, UK
          20 - 21 April 2009
          Electronic Workshops in Computing (eWiC)
          Evaluation and Assessment in Software Engineering (EASE)
          Product Information: 1477-9358BCS Learning & Development
          Self URI (journal page):
          Electronic Workshops in Computing


          Comment on this article