Assessing the Quality and Cleaning of a Software Project Dataset: An Experience Report

Abstract

OBJECTIVE – The aim is to report upon an assessment of the impact noise has on the predictive accuracy by comparing noise handling techniques.

METHOD – We describe the process of cleaning a large software management dataset comprising initially of more than 10,000 projects. The data quality is mainly assessed through feedback from the data provider and manual inspection of the data. Three methods of noise correction (polishing, noise elimination and robust algorithms) are compared with each other assessing their accuracy. The noise detection was undertaken by using a regression tree model.

RESULTS – Three noise correction methods are compared and different results in their accuracy where noted.

CONCLUSIONS – The results demonstrated that polishing improves classification accuracy compared to noise elimination and robust algorithms approaches.

Contributors

Gernot Liebchen

Bheki Twala

Martin Shepperd

Michelle Cartwright

Conference

Publication date: April 2006

Publication date (Print): April 2006

Pages: 1-7

Affiliations

Brunel University, UK

Article

DOI: 10.14236/ewic/EASE2006.14

Copyright statement: © Gernot Liebchen et al. Published by BCS Learning and Development Ltd. 10th International Conference on Evaluation and Assessment in Software Engineering (EASE), Keele University, UK

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 10th International Conference on Evaluation and Assessment in Software Engineering (EASE)

Conference acronym: EASE

Conference number: 10

Conference location: Keele University, UK

Conference date: 10 - 11 April 2006

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Evaluation and Assessment in Software Engineering (EASE)

Product

Product Information: 1477-9358BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EASE2006.14

Self URI (journal page): https://ewic.bcs.org/

Categories

Series title: Electronic Workshops in Computing