AI Coding Tools Attract Top Performers – But Do They Create Them?
Featuring cohort data retrieved from Cursor, Github Copilot & Claude Code, this research includes high-fidelity measurement (taken from AI provider APIs) that reveals a significant productivity gap associated with AI use.
The new data lets us compare developers whose weekly AI use ranged from "Non-User" to "Power User." Contrary to our expectations, the difference in productivity data between the two sides amounts to a chasm:
Start reading to learn:
- How substantial was the difference in durable code output between developers who used AI throughout the day, vs those without measurable AI use?
- How much do developers that use AI most cause their team to become mired in code review?
- Which negative side effect was 9x more likely to occur among developers that use AI most?
Abstract
The first quarter of data from GitClear’s AI usage measurement is in, and it defies conventional wisdom. Developers who use AI throughout the day aren't just 10% faster—empirical data shows them authoring 4x to 10x more work than “AI non-users” during the weeks their AI use is highest.
However, our research suggests this isn't another story about the triumph of AI tools and agents. Moreso, this story is about who is using the tools and what company policies must prospective AI developers contend with?
Constructed from AI provider APIs, the new data lets us compare developers whose weekly AI use ranged from "Non-User" to "Power User." Contrary to our expectations, the difference in productivity data between the two sides amounts to a chasm.
Weekly “Commit Count” is far from an end-all productivity metric – but it mirrors the trend found across other proxies for developer output
Finding such an extreme gap implies the existence of something akin to “dark matter” for developer productivity. There must be much more than “enlightened AI use” to explain a difference of this magnitude. In the sections to follow, we’ll present and dissect data to understand “how can the ‘highest engagement’ cohorts average 5x more progress across almost every metric GitClear tested?”
In other words, what’s the source of the “dark matter productivity” enjoyed by the “Power User” cohort? If factored out, how large of a difference remains? That calculation can help executives hone their intuition for “what quantifiable improvement can we expect to observe, if our team succeeds at maximizing AI benefits while minimizing AI side effects?” As past AI research has demonstrated, the latter objective requires careful attention to combatting the code churn and code block duplication that accompanies contemporary AI use?
Venn Diagram of Developer Cohorts by AI Use
Dissect factors contributing to productivity multiples in the AI Agent era.