Blog / Other resources

How to keep code quality as AI agents write more of the code

AI agents write more of the code. Learn how to keep code quality: measure rework and reverts next to delivered work, by engineer, assistant and agent.

Why quality can drop when agents write more code

  • More code, same reviewers. Agents open more pull requests than people can review with care.
  • Code that passes but does not hold. Tests pass, and the change comes back as rework weeks later.
  • Output looks better than it is. If you count only pull requests, rework looks like extra output.

4 quality signals to track

Signal What it shows How to measure it
Rework Work that did not hold Share of delivered work fixed again within 30 days
Reverted changes Work that failed Share of merged pull requests that were reverted
Defects after release Work that reached users with a problem Bugs linked to the change that caused them
Review depth If people still check the work Review comments and changes requested per pull request

Read each signal by source: engineer, AI-assisted engineer and agent.

How to keep quality in 4 steps

  1. Measure output and quality together. Never report delivered work without rework next to it.
  2. Compare by source of work. If agents produce more rework for one type of work, keep people on it.
  3. Keep human review where it matters. Review all agent changes to security, payments and data. Use AI review for the rest.
  4. Check the trend every month. Quality must hold as the share of agent work goes up.

See the AI quality tax for what rework costs.

Common mistakes

  • Counting pull requests as output. Agents make many small changes, and rework looks like progress.
  • Measuring quality once. Quality changes as agents take more work.
  • Blaming people for agent rework. Track the source of each change, so you fix the process, not the person.

How Pensero measures quality for humans and agents

The Delivery Point is one unit of finished work, scored by size and difficulty, the same way for engineers, AI-assisted engineers and agents. Pensero reads rework and reverts next to it, and traces each change to the pull request and ticket behind it.

Sources

Frequently asked questions

Does AI-generated code have lower quality?

It depends on the type of work. Measure rework and reverts by source, and you see where agents hold quality and where they do not.

Which metrics show code quality with AI agents?

Rework, reverted changes, defects after release and review depth, read next to delivered work.

Should every agent change get a human review?

Keep human review on high-risk code, such as security, payments and data. Use AI review for low-risk changes, and check the rework.

Which tool measures quality for humans and agents?

Pensero measures the delivered work and the quality of engineers, AI-assisted engineers and AI agents on one scale, the Delivery Point.