Gamifying user feedback collection on static program analysis tools

Use of static program analysis tools can be highly beneficial in software development, but usage is hindered by usability issues. One method to better understand these issues is to gather user feedback, but it is challenging to get developers to invest effort in giving user feedback.

In this paper, we investigate whether gamification can increase user engagement in feedback collection on static analysis tools. To this end, we carried out a mixed-method study at a large software company, with initial user research, design of a feedback system with a context-sensitive gamification, deployment of the system to 900 developers over 14 days in an experiment with randomized assignment, and a final collection of qualitative user input.

We found that user engagement increased for the gamification group in terms of amount of feedback (2.47 times more feedback), but the difference is not statistically significant, and participants also expressed a mix of enthusiasm and skepticism in our follow-up interviews.