One of the questions I find most interesting when working with AI agents is not how much code they can generate, but how much work we are willing to let them do without intervening.
The easy answer would be that autonomy should increase as models improve. If an agent writes better tests, understands larger repositories, and makes fewer mistakes, it seems reasonable to supervise it less and less.
But I think an important variable is missing from that reasoning: the context in which we are introducing the change.
We do not need the same level of confidence to build a prototype as we do to modify a system with years of history, real users, and data we cannot afford to lose. Nor do we need the same evidence to change the look of a button as we do to change a billing rule.
So autonomy is probably not a property of the agent.
It is an engineering decision.
And it depends both on what the agent is capable of doing and on the consequences of getting it wrong.
An MVP or POC is not production
There is a distinction that often disappears when we talk about AI.
Building a proof of concept is not the same as modifying a system that has been running for five years.
A proof of concept tries to answer one question:
Can we technically do this?
In that context, the code can be disposable, the architecture provisional, and test coverage minimal. It may even be perfectly reasonable to build almost everything through prompting if all we need is an answer.
An MVP pursues something different: validating a product hypothesis at the lowest possible cost.
There will already be users, and we will need a certain level of quality, but learning remains the priority. We can accept reversible decisions and even knowingly take on debt if that helps us discover sooner whether the problem really deserves a solution.
An established product plays by different rules.
It has users, data, contracts, integrations, past incidents, and decisions that perhaps nobody documented. It also contains behaviors that someone depends on, even if no specification mentions them.
When we change that kind of system, the question starts to shift.
It is no longer enough to ask:
Can AI implement this feature?
We also have to ask:
Can we introduce this change without breaking existing expectations, constraints, or relevant behaviors?
The agent’s technical capability may be exactly the same in both cases.
The autonomy we grant it probably should not be.
In greenfield we can make assumptions. In legacy we have to discover
When we build something from scratch, we have a considerable advantage: much of its behavior can still be decided.
If an agent proposes a reasonable structure, meets our criteria, and the decision is easy to reverse, we can probably move forward without too much investigation.
Legacy systems are different.
The code contains knowledge, but it does not contain all the system’s knowledge. Some of it lives with users, in incidents, old tickets, databases, operational processes, and exceptions that accumulated over the years.
Sometimes we find a seemingly absurd condition and our first instinct is to remove it.
Then we discover it exists because something happened in 2021 that nobody wants to experience again.
The design may become cleaner when we remove it, while production stops behaving the way someone expected.
These situations seem especially relevant when we work with agents. A model may analyze everything in front of it perfectly and still have no access to the reason things ended up that way.
That is why I think an important skill for developers working with AI will be distinguishing when they can make assumptions and when they need to investigate.
Our prior experience becomes more valuable precisely here. Having seen systems fail, knowing a domain’s invariants, or knowing which parts tend to hide exceptions helps us decide where we can delegate and where we should pause before changing anything.
The point is not to distrust the agent.
It is to recognize when the system knows things that neither we nor the agent know yet.
The developer of the future should not be an auditor of generated code
There is also an opposite extreme that does not seem very sustainable.
If our agents can produce ten times more code but we need to read every line exhaustively before accepting it, much of the gain disappears. The bottleneck has simply moved.
That does not mean we should stop reviewing.
It means we need more ways to gain confidence than manually reading every implementation detail.
A mature engineering practice should try to turn some of its implicit rules into explicit checks.
Tests are the obvious example, but not the only one. We can rely on the compiler, types, static analysis, contracts, architectural checks, observability, acceptance scenarios, or domain-specific evaluations.
We can even use independent agents to review some properties of work produced by other agents.
The interesting question is not how many tools we accumulate, but which important properties we can check automatically.
If we never allow a particular dependency between two modules, perhaps that rule should not live only in the head of someone who knows the architecture.
If an operation needs to be idempotent, we can test that property.
If certain data must never be left in an inconsistent state, we can try to express that invariant in a way the system can check.
Every rule we can move out of our memory and turn into an automated check increases the space in which we can delegate with some confidence.
That lets us expand agents’ autonomy without increasing our supervision at the same rate.
But there is an important limit:
Guardrails can only protect us from what we have thought to check.
Green tests do not necessarily mean the software is correct
This problem becomes especially interesting when the same AI can participate in every part of the process.
It can help us write the specification, generate the implementation, produce the tests, and later review the result.
At first glance, that seems like an extraordinarily complete cycle.
It can also be an extraordinarily coherent cycle built around a mistaken interpretation.
If the specification starts from an incorrect understanding of the problem and the tests derive from that same interpretation, the implementation can satisfy every criterion we defined perfectly.
The tests will be green.
The software will still be wrong.
This is not really a problem unique to AI. Human teams can also spend months building exactly what they specified, only to discover later that it was not what they needed.
AI can simply reduce the time it takes to reach the same mistake.
That is why we need sources of feedback that are sufficiently independent from the generation process.
Sometimes that will be another agent with different context. At other times, it will be an acceptance test written in advance, a domain constraint, production telemetry, or a conversation with a user.
And there will be situations where the only way to discover whether something is right is to put a small version in front of reality and see what happens.
Internal consistency is necessary, but not always sufficient.
Autonomy should be proportional to risk
Perhaps a more useful way to think about all this is to abandon the idea of finding a universal level of autonomy for agents.
It does not make much sense to decide that in an organization “agents can implement complete features” or that “all generated code must be reviewed line by line.”
Context changes too much.
A disposable prototype can allow enormous autonomy.
A small, reversible change to an internal application might as well.
Changing access authorization for sensitive data, an irreversible migration, or a rule that affects billing belongs in a different category.
In those cases, we probably want more context, more independent evidence, better rollback mechanisms, and perhaps more human involvement too.
The interesting thing is that this does not necessarily mean using less AI.
We can use it specifically to increase our safeguards: explore scenarios, generate edge cases, review invariants, compare alternatives, or look for side effects we had not considered.
The point is to adjust how we delegate to match the risk we are taking.
Conclusion
As agents become capable of doing more work on their own, it will be tempting to treat their level of autonomy as a measure of maturity.
I do not think that is a good metric.
A system where an agent can work for hours without human intervention is not necessarily more advanced than one with several checkpoints.
The relevant question is what risk we are taking and what evidence we have to trust the result.
In some contexts, that evidence will be minimal. In others, we will need tests, invariants, observability, independent review, or real user feedback before we feel comfortable.
The agent’s capabilities matter, but they do not decide on their own how much we should delegate.
That decision is still ours.
And perhaps that is an important part of the work we still have as engineers: designing systems where we can grant more and more autonomy without having to grant, at the same time, more and more blind trust.