Hyperparameter Optimization to Improve Bug Prediction Accuracy

Osman, Haidar; Ghafari, Mohammad; Nierstrasz, Oscar Marius (February 2017). Hyperparameter Optimization to Improve Bug Prediction Accuracy. In: IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE). Klagenfurt, Austria. 21-21 Feb. 2017. 10.1109/MALTESQUE.2017.7882014

[img] Text
07882014.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (208kB) | Request a copy

Bug prediction is a technique that strives to identify where defects will appear in a software system. Bug prediction employs machine learning to predict defects in software entities based on software metrics. These machine learning models usually have adjustable parameters, called hyperparameters, that need to be tuned for the prediction problem at hand. However, most studies in the literature keep the model hyperparameters set to the default values provided by the used machine learning frameworks. In this paper we investigate whether optimizing the hyperparameters of a machine learning model improves its prediction power. We study two machine learning algorithms: k-nearest neighbours (IBK) and support vector machines (SVM). We carry out experiments on five open source Java systems. Our results show that (i) models differ in their sensitivity to their hyperparameters, (ii) tuning hyperparameters gives at least as accurate models for SVM and significantly more accurate models for IBK, and (iii) most of the default values are changed during the tuning phase. Based on these findings we recommend tuning hyperparameters as a necessary step before using a machine learning model in bug prediction.

Item Type:

Conference or Workshop Item (Paper)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF)
08 Faculty of Science > Institute of Computer Science (INF) > Software Composition Group (SCG)

UniBE Contributor:

Osman, Haidar; Ghafari, Mohammad and Nierstrasz, Oscar Marius

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

Language:

English

Submitter:

Oscar Marius Nierstrasz-Margiotta

Date Deposited:

16 May 2018 08:35

Last Modified:

16 May 2018 08:35

Publisher DOI:

10.1109/MALTESQUE.2017.7882014

Uncontrolled Keywords:

scg-pub snf-asa2 scg17 jb17

BORIS DOI:

10.7892/boris.113140

URI:

https://boris.unibe.ch/id/eprint/113140

Actions (login required)

Edit item Edit item
Provide Feedback