Blog
About

150
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      A follow-up empirical evaluation of evidence based software engineering by undergraduate students

      ,

      12th International Conference on Evaluation and Assessment in Software Engineering (EASE) (EASE)

      Evaluation and Assessment in Software Engineering (EASE)

      26 - 27 June 2008

      EBSE, evidence based software engineering, empirical methods, Systematic Literature Review, SLR, evaluation of software methods and tools, decision making, teaching, Requirements Management Tools, RMTs, Telelogic DOORs®, Borland Caliber® Analyst, Compuware Optimal TraceTM , GODA ARTS

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Context: Evidence Based Software Engineering (EBSE) has recently been proposed as a methodology to help practitioners improve their technology adoption decisions given their particular circumstances. Formally, Systematic Literature Reviews (SLRs) are a part of EBSE. There has been a noticeable take up of SLRs by researchers, but little has been published on whether, and then how, the EBSE methodology has been applied in full.

          Objectives: To empirically evaluate the use of EBSE by undergraduate students. To consider how insights into the students’ performance with EBSE can be applied to software practitioners’ expected performance with EBSE. To gain insights into the use of Supplementary EBSE Guidelines and associated Assessment Schemes.

          Method: 37 final-year undergraduate students completed a coursework assessment that required them to use EBSE to evaluate one or more of four Requirements Management Tools (RMTs): Telelogic DOORs®, Borland’s Caliber® Analyst, Compuware Optimal Trace TM, and GODA ARTS. Students were provided with a range of EBSE resources, including a set of Supplementary EBSE Guidelines, to assist them with their EBSE evaluation. An Assessment Scheme was development to assess the degree to which students followed the Supplementary Guidelines. A feedback questionnaire, completed by 12 of the 37 students, complemented the Assessment Scheme.

          Results: 78% of students chose to evaluate a RMT that is currently a market leader. 62% of students subsequently recommended the adoption of their chosen RTM. Some students made a recommendation because the Guidelines indicated they should, rather than making a recommendation because the evidence ‘entailed’ that recommendation. Only 8% of students intentionally made no recommendation, and this seems to be on the basis of the poor quality of evidence available on the chosen RMT(s). All 12 students who completed the feedback questionnaire reported that this was the hardest or nearly the hardest assessment that the student had ever done! 67% of these 12 students reported that they had not given themselves sufficient time to complete the evaluation, and 83% reported that they had to balance their time on this evaluation with other commitments. The12 students found EBSE steps 1 and 4 to be the easiest and EBSE steps 2 and 3 to be the hardest, and they generally reported that they had received sufficient support from the EBSE resources made available to them.

          Conclusion: This study presents independent, empirical evidence of the full use of EBSE, to complement the growing body of published SLRs. We believe that the findings on students’ use of EBSE are relevant to understanding professional practitioners’ prospective use of EBSE. Both students and professionals would: find EBSE very challenging, in particular the SLRs; be subject to constraints and trade-offs; may not be able to find relevant and rigorous evidence; and may make errors in their critical thinking whilst conducting the evaluation. Our findings should benefit researchers, practitioners and educators.

          Related collections

          Author and article information

          Contributors
          Conference
          June 2008
          June 2008
          : 1-10
          Affiliations
          School of Computer Science

          University of Hertfordshire

          College Lane Campus

          Hertfordshire AL10 9AB, U.K.
          Article
          10.14236/ewic/EASE2008.9
          © Austen Rainer et al. Published by BCS Learning and Development Ltd. 12th International Conference on Evaluation and Assessment in Software Engineering (EASE)

          This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

          12th International Conference on Evaluation and Assessment in Software Engineering (EASE)
          EASE
          12
          University of Bari, Italy
          26 - 27 June 2008
          Electronic Workshops in Computing (eWiC)
          Evaluation and Assessment in Software Engineering (EASE)
          Product
          Product Information: 1477-9358BCS Learning & Development
          Self URI (journal page): https://ewic.bcs.org/
          Categories
          Electronic Workshops in Computing

          Comments

          Comment on this article