Quantifying developer productivity via code analysis
Posted by Bill Harding and Grace Bradford
Published March 26, 2018
GitClear was built in part to quantify developer contributions. A common question we hear is "How can we measure the productivity of our development team?”
Boiling down developer productivity to a single number could be a holy grail for managers – especially non-technical managers. So, how can we help them find clarity on the quantity and nature of work that’s being done?
Better measurement with Line Impact
Would you use an ineffective measurement tool when something more accurate was avaiable?
Some teams use "lines of code written" as a benchmark for productivity scoring. This is problematic for many reasons, which we've discussed in-depth in our previous article on counting lines of code. We recommend checking out that content if you haven't read it yet.
There are tons of factors that inflate or diminish a dev's code line count, making it a tall order to use lines of code to measure a developer’s impact. Our data suggests that about 95% of all changed lines are noise from the standpoint of whether they result in a lasting impact to the code base. But we’ve determined it can be done -- with sufficient flexibility and attention to detail.
It took our team about 30 months of concerted pre-launch effort to build the system of checks and balances that digests all of the factors involved in coding, and consolidates them into a single metric.
We call this metric Line Impact, and we believe that it’s the most accurate and comprehensive solution for teams who want to quantify the impact being made over time by their development team.
Much of our development time has gone toward fine tuning our tools so that individual teams can adjust the scalars we apply to lines in different code categories, file types, and so on. We’ve worked hard to ensure that our default values are sensible for general use cases – this isn’t a tool you’ll need to spend hours configuring before you begin to draw profound insights. But we also appreciate that reasonable CTOs can come to very different conclusions of how much impact is made by writing a test, writing a line of CSS, or updating a file annotation.
Making the most of GitClear
Although Line Impact isn't the One Ring to rule them all, it does offer us unprecedented clarity to identify developers who:
- • Make a high volume of contributions
- • Make contributions to high value files
- • Make contributions that persist over time
Our experience suggests that these three attributes correlate strongly with developers who add the most value to a team. When we talk to a new customer, our first question is “do these Line Impact values align with your intuition about the team’s most productive developers?” Usually we’re pretty close.
Despite the high correlation between Line Impact and human-judged developer productivity, Line Impact alone can’t tell you whether a developer is working on the most important tasks. To assess whether a developer is getting not only a lot of work done, but the most important work done, we still need to rely on the judgement of humankind (sorry, robots).
Fortunately, we have Project Managers, Lead Developers, and CTOs who can make sure that their team is working toward goals that matter most to the organization. In addition to measuring Line Impact, we’ve invested heavily in tools that allow managers to pinpoint what their team is working on during a given day, month or year.
Line Impact lets you visualize how much work is getting done, while the rest of GitClear focuses on illustrating what kind of work is getting done. And developer productivity lies at the intersection of these two measurements.