New Research Debunks 8 Myths About AI Coding Tools

Myth 1: Developers Spend Most of Their Day Writing Code

They don’t. Studies put the figure at 11–18% of a typical workday. A 2025 study co-authored by Houck landed at 14%.

The rest goes to design, meetings, code review, coordination, and administrative work. If AI accelerates only the code-writing slice, the ceiling on productivity gains is much lower than vendor decks suggest. The paper found a typical organization is seeing roughly a 7.8% increase in code throughput from AI tools—a real number, but not the 10x the headlines promise.

Myth 2: More Code Output Means More Productivity

Bill Gates said it decades ago: measuring software productivity by lines of code is like measuring airplane progress by weight. The paper makes the same point with fresher evidence.

Volume metrics reward the wrong behaviors and inflate review burden—which is already a bottleneck. For one internal AI coding agent examined in the research, only about half of pull requests were ultimately accepted. Fifteen percent were abandoned. Another 15% sat waiting on a human reviewer.

Pull request throughput is a better signal, but even that needs context. The paper points to the DX AI Measurement Framework as a more honest approach.

Myth 3: AI Coding Tools Reliably Speed Things Up

The research is genuinely mixed. Some studies show large gains. Others show neutral effects. One study of experienced open-source developers found AI tools increased implementation time by 18% on average.

The variance isn’t random—it tracks with task familiarity, developer experience, motivation, and problem-solving style. Even prompt wording matters: one study found semantically equivalent rewrites produced different code 46% of the time and changed correctness in 28% of cases.

Myth 4: The “10x Developer” Is Real

The paper pushes back on this directly. Productivity gains measured on isolated, toy-sized tasks rarely survive contact with real codebases and real teammates.

Prior research also suggests that much of the variance between developers is a property of the task, not the person. The 10x narrative may say more about task selection than human capability.

Myth 5: Most Developers Trust AI Tools

Adoption is high. Trust is not. The paper found that 80% of developers use AI tools—but only 29% trust their accuracy.

That gap matters. A tool people use but don’t trust tends to generate extra verification work, not saved time.

Myth 6: AI Is Going to Replace Programmers

Only 10% of developers in one Microsoft survey expressed concern about AI taking their jobs. Most describe AI as expanding their capacity—freeing time for architecture, mentorship, and brainstorming rather than routine lookups.

The replacement narrative appears to be more of a media story than a developer concern.

Myth 7: AI Benefits Everyone Equally

It doesn’t. The paper identifies what the authors call a “competence penalty”: developers—particularly women and older engineers—receive harsher evaluations for AI-assisted work, even when the output is identical to non-assisted work.

Adoption isn’t just a technical challenge. It’s a social and organizational one.

Myth 8: This Is an Individual Productivity Problem

This may be the paper’s sharpest point. Nearly all existing research studies a single developer paired with a single tool, placing the burden of productivity on the individual.

Historically, real productivity gains have come from systematic changes at the organizational level. AI may be the first technology where organizations have spent millions on licenses without a clear plan for extracting value from them. As Butler framed it: stop asking “how much code did AI write?” and start asking where AI is actually relieving friction—and where it’s just shifting pressure downstream.

What This Means If You’re Evaluating AI Coding Tools

The paper isn’t anti-AI. It’s anti-sloppy-measurement. The practical read for engineering leaders and tool evaluators:

• Don’t benchmark on code volume. Measure completed work, review burden, and developer experience together.
• Expect variance. Gains on familiar tasks won’t automatically transfer to complex, collaborative work.
• Treat adoption as a systems problem. Individual licenses without organizational change tend to produce individual results.
• Watch for hidden costs. Low trust, review bottlenecks, and competence penalties can quietly offset throughput gains.

The tools may be real. The ROI depends almost entirely on how you deploy AI coding tools.