AI Needs an Org Chart Before It Needs More Autonomy

I did not set out to build an AI company inside my company. I started with a simpler goal: reduce the routine work that still depended on me.

One agent became a few. Then the work split naturally. Strategy was different from finance. Writing was different from publishing. Building software was different from testing it.

Eventually, I had 21 defined AI roles covering strategy, finance, marketing, sales, customer success, delivery, product, engineering, design, quality assurance, security, research and data.

That was when the problem changed.

The question was no longer, “Can the AI do the task?” It became, “Who is accountable when it does?”

Companies are moving from copilots that wait for instructions to agents that can plan, call tools, access systems and take actions. The more capable the agent becomes, the less useful it is to think of it as just another software feature.

A capable agent needs an operating model.

Give agents jobs, not just prompts

A prompt describes what you want now. A job describes what someone owns repeatedly.

I found that each AI role needed a clear mandate, scope, inputs, outputs and escalation point. A writing agent should not approve a client deliverable. An engineering agent should not decide that its own code is production-ready. A research agent should not install what it discovers.

This sounds obvious with human teams. It becomes easy to ignore with AI because the same underlying model may be capable of doing all three.

Capability is not authority.

Do not let AI mark its own homework

The second lesson was to separate the builder, reviewer and acceptor.

If the same agent writes the code, tests the code and declares it ready, you have created the machine equivalent of an employee approving his own audit.

So I split those responsibilities. The builder builds. A separate testing role checks the result. A separate quality role reviews the evidence. Final acceptance of important work still comes back to a human.

If those answers are unclear, adding more autonomous agents will not make the organization more mature. It will simply make the ambiguity run faster.

NIST’s AI Risk Management Framework follows a similar lifecycle discipline through its Govern, Map, Measure and Manage functions. The lesson for me is simple: one control does not make an AI system trustworthy. Risk has to be managed across the lifecycle.

This matters even more when multiple agents work together. One weak assumption can pass from agent to agent and emerge at the end looking like consensus.

Autonomy should follow reversibility

The third lesson was that autonomy should not be treated as a maturity score.

The better question is: what happens if this action is wrong?

If an agent drafts an internal note badly, the cost is low. If it changes permissions, deploys production code, sends a client communication or commits money, the cost can be much higher.

Singapore’s updated Model AI Governance Framework for Agentic AI makes this distinction concrete. One case study tiers actions by severity, reversibility and the feasibility of human oversight. Low-risk, reversible actions can be automated. Moderate-risk actions require human approval. High-risk, difficult-to-reverse actions may be blocked entirely.

That is close to how I now think about autonomy.

I no longer ask, “How much can I automate?” I ask, “What is the most authority I can safely delegate for this class of action?”

Inventory the system around the agent

The fourth lesson came when I started documenting the agents themselves.

An agent is not only its prompt or model. It may inherit tools, connectors, credentials, scheduled jobs, databases and external services. Its real risk surface is everything it can reach.

That pushed me to maintain an internal agent bill of materials: an inventory of agents and the capabilities around them.

The uncomfortable part was not discovering how many agents I had. It was seeing how quickly permissions and dependencies accumulated around them.

Human involvement is not necessarily technical debt. Sometimes it is the control architecture.

OWASP’s 2026 State of Agentic AI Security and Governance points to the same lifecycle problem. Securing autonomous systems requires looking beyond model behavior to tooling, access, data flows and operational controls.

For technology leaders, the inventory question is becoming as important as the model question.

If you cannot answer which agents exist, what they can access, what tools they inherit and who approves high-impact actions, you do not yet have an autonomous workforce. You have an expanding accountability surface.

Full autonomy is not the finish line

Today, all 21 of my AI roles are real and used in live work, but I do not pretend they operate as a completely unattended company.

Some workflows are automated. Much of the orchestration is still interactive. Important outward actions still require approval.

I once would have described that as incomplete. Now I see it differently.

Human involvement is not necessarily technical debt. Sometimes it is the control architecture.

The goal should not be to remove humans from every workflow. It should be to know exactly where humans add judgment, accountability and authority, and remove them from the places where they do not.

For leaders moving from copilots to multi-agent systems, I would start with four questions:

• Does every agent have a defined owner, mandate and boundary?

• Are building, reviewing and accepting separated for important work?

• Is autonomy limited according to impact and reversibility?

• Can you inventory every agent, tool, permission and external connection?

If those answers are unclear, adding more autonomous agents will not make the organization more mature.

It will simply make the ambiguity run faster.

The views and opinions expressed in this article are those of the author and do not necessarily reflect those of CDOTrends. Image credit: iStockphoto/WANAN YOSSINGKUM