Bug Report Triaging Using Machine Learning

5 min readAug 6, 2021

What is a Bug?

A software bug can be defined as a fault or flaw in a computer program that results in wrong or unexpected output. It can arise due to human error, bad coding practices, miscommunication and many more reasons. It can impact the functioning, safety, and security of computer operating systems. The process of detecting and removing these faults and flaws is called debugging.

A software bug goes through a life cycle, which consists of the following stages:

Fig. Life Cycle of a Bug [Ref: https://www.geeksforgeeks.org/bug-life-cycle-in-software-development/]

What is a Bug Report?

A bug report is a technical document that describes the symptoms of a bug. It contains detailed information about a single bug. Whenever a bug arises, all relevant information regarding it is filled in the bug report document. It is then submitted to a repository often a bug tracking tool such as Bugzilla, Jira, Trac, Mantis etc. The manager can assign the bug to a developer using the bug report. The developer uses the bug report to understand, recreate and resolve the bug.

What is Bug Report Triaging?

Bug Triaging is a process of classifying the bug reports on the basis of severity, efforts, duplication, or any other such factor.

Types of Bug Report Triaging:

Bug Severity Prediction
Effort Estimation for Bug Fix
Assigning appropriate developer to resolve the bug
Duplicate Bug Tagging
Valid/Invalid Bug Report Identification

Advantages of Bug Report Triaging

As the software gets bigger and complex, software bugs also become complex and inevitable. These bugs cause inconvenience to users by causing malfunctions thus, decreasing the quality of the software until it is fixed. Bug triaging helps the development team to knows which defects and bugs are needed to be worked on first. It also helps to ensure the necessary information is recorded on the bug report.

Furthermore, being well aware, when customers contact the support team about a known bug, the team is able to respond intelligently, they’re able to commit as to when the bug will be fixed.

The process also plays a role in backlog grooming, as triaging bugs keeps your backlog in better shape. Without triaging bugs, the team could miss out on critical items and be negligent of the priority on which they need to work on.

Assigning appropriate Developer to Resolve the Bug

For a given bug report, depending on the characteristics, assign the bug to an appropriate developer for resolution.

Non-ML Approach

For a given bug report, depending on the characteristics, a bug can be assigned to an appropriate developer for resolution. However, manually assigning bugs to developers may involve human errors and bias, resulting in poor performance. Automating the process helps in eradicating such bias and reduces the time required for assignments.

Machine Learning-Based Approaches

Using better “Models” for prediction
Better “Feature Generation” techniques

Improving Features Representation for Assigning Appropriate Developer

Examples for the need for better feature representation,

The bottom figure shows two bug reports, originally triaged to different developers(they are from the test set), the first image is of a bug report in the training set.
If “bag of words” models are used to generate the features any model may misclassify bug report #2 to be assigned to the developer given in the training sample figure, as several words match with the training sample.
This is due to the poor feature generation technique, as the bag of words does not consider the sequence of words

Src: DeepTriage: Exploring the Effectiveness of Deep Learning for Bug Triaging, CoDS-COMAD’19

Effort Estimation for a Bug Report

Using Code Churn Size for Effort Prediction

To estimate the time required to fix a bug, given a bug report. Estimating the efforts required helps in several ways such as, efficient developer assignment, project release scheduling, prioritization and many more.

What is Code Churn Size?
The number of lines modified, added or removed in order to resolve a bug.

How to use code churn size in effort prediction?

For training a classifier for predicting effort, the class labels could be generated using ‘code churn size’.
Several bug tracker tools(eg. Jira) have a feature to keep track of code changes for a given bug report. Using this, Effort Extractor finds the number of lines churned for a bug.
If the number of lines churned is greater than a certain threshold, consider that sample as high effort sample. (Note: sample here means, bug report for any given bug).
Feature Extractor module pre-processes the description section of the bug report to extract features i.e, terms used in the description to generate the features to train the model.
A machine learning classifier can then be trained to learn from this <feature, label> data to classify a given bug report to the amount of effort required.
When the model is deployed, the feature extractor is fed with a bug report, for which prediction is required. Feature extractor generated the features, using which the classifier predicts the effort required.

Src: Automatic prediction of bug fixing effort measured by code churn size, SoftwareMining’16

Some Standard Datasets

Mozilla Core
Firefox
Eclipse Platform
Spark
Hadoop

To Download Datasets Visit: https://github.com/logpai/bugrepo

How to Learn More

Learning is a never-ending process, to gain further knowledge on the application of Machine Learning for Bug Reports Triaging, following approaches can be taken:

Visit Top Software Engineering Conferences such as ICSE, ASE, FSE, TOSEM, TSE, MSR and so on.
Read some good survey paper or literature review
Check out the reference sections of the research papers.

Acknowledgement

I would like to thank my professor Dr Pankaj Jalote.

References

[1]. DeepTriage: Exploring the Effectiveness of Deep Learning for Bug Triaging, CoDS-COMAD’19
[2]. Applying Deep Learning Based Automatic Bug Triager to Industrial Projects, FSE’17
[3]. Automatic prediction of bug fixing effort measured by code churn size, SoftwareMining’16
[4]. Bug or Not? Bug Report Classification using N-Gram IDF, ICSME’17