Definition
The percentage of merged pull requests that address defects.
To determine that a PR is addressing a defect, we look for keywords in the PR title. Specifically, we search for any word (case-insensitive) that begins with: "fix", "revert", "bug", or "repair"(thus, we would include PRs with the words "reverts" or "bugfix" but not "transfix").
Defect Rate is only calculated for merged PRs.
How to use it
It’s normal for some defects to make it through the code review process and need patching with future PRs. However, when the defect rate gets too high, it’s an indicator that your engineers might not have access to or may not be making adequate use of tools for testing changes before they get merged. When your defect rate is above 20%, you should review recent defect PRs to identify patterns. Consider:
How thorough is unit test coverage?
What tools are available to test out proposed changes on realistic data?
Are there automated tests for detecting UI regressions?
Benchmarks
The median organization has a Defect Rate of 17%. The lowest quarter of organizations have a Defect Rate of 13% or less, while the higher quarter of organizations have a Defect Rate of 25% or more.