Hyperparameter Optimization to Improve Bug Prediction Accuracy

Osman, Haidar; Ghafari, Mohammad; Nierstrasz, Oscar Marius (February 2017). Hyperparameter Optimization to Improve Bug Prediction Accuracy. In: IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE). Klagenfurt, Austria. 21-21 Feb. 2017. 10.1109/MALTESQUE.2017.7882014

Text 07882014.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (208kB)

Bug prediction is a technique that strives to identify where defects will appear in a software system. Bug prediction employs machine learning to predict defects in software entities based on software metrics. These machine learning models usually have adjustable parameters, called hyperparameters, that need to be tuned for the prediction problem at hand. However, most studies in the literature keep the model hyperparameters set to the default values provided by the used machine learning frameworks. In this paper we investigate whether optimizing the hyperparameters of a machine learning model improves its prediction power. We study two machine learning algorithms: k-nearest neighbours (IBK) and support vector machines (SVM). We carry out experiments on five open source Java systems. Our results show that (i) models differ in their sensitivity to their hyperparameters, (ii) tuning hyperparameters gives at least as accurate models for SVM and significantly more accurate models for IBK, and (iii) most of the default values are changed during the tuning phase. Based on these findings we recommend tuning hyperparameters as a necessary step before using a machine learning model in bug prediction.

Interest & Impact

Downloads

1 since deposited on 16 May 2018
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item