At an executive roundtable we hosted in August, a leader who had spent years moving his company into the cloud said something that quieted the room: we have moved a lot of things to the cloud over the years, but that does not mean it is safe. As his team turns AI loose on those systems, everything that quietly accumulated there behaves like the old shared S drive, where people put things wherever they wanted. It is not just what you do with AI next, he said. It is what has already been done. As you point AI at that legacy, it is going to get exposed.

That exchange is why this post exists. The on-prem versus cloud debate is a decade old, but agents just changed the question underneath it.

Agents changed the question

Chat-style AI reads. Agents act. They retrieve documents, touch customer records, call tools, and produce new outputs, which means the sovereignty question is no longer only where your data is stored, but where reasoning, retrieval, and action are allowed to execute.

The roundtable produced a perfect example. One company asked us about letting an AI agent loose on their CRM to update records, until someone did the math out loud: that is 5,000 customer records the agent could touch, and one bad run could quietly change any of them. What safeguards exist before we say go? Another leader worried about the reverse flow: you are feeding these models with prompts, files, and context about your business, and at some level they are learning from you. A third raised the insider question that on-prem alone does not solve: once your company’s knowledge is easily searchable by AI, how do you detect the one person who downloads all of it?

The question is no longer where your data sits. It is where your AI is allowed to act.

What the market is actually doing

The numbers back the room’s instincts. According to Broadcom’s Private Cloud Outlook 2026, the share of enterprises running production AI inference primarily in the public cloud fell from 56 percent to 41 percent in a single year, with a majority now running or planning production inference on private infrastructure. This is not a wholesale retreat from cloud. It is workload-by-workload sorting, the same rebalancing we wrote about in why more organizations are bringing workloads home, now accelerated because agents raise the stakes of getting placement wrong.

And the honest counterpoint still applies: aging local infrastructure can throttle AI just as surely as a bad cloud contract, a case we made in your on-prem servers are the real bottleneck to AI. Agents do not settle that argument for one side. They force a sharper sorting rule.

Three questions that decide where an agent lives

1. Blast radius: What can this agent touch, and what is the worst thing it could do there? An agent that drafts marketing copy can live anywhere. An agent with write access to 5,000 CRM records, your ERP, or your financials belongs where you control permissions, logging, and the kill switch, whatever infrastructure that is.
2. Context sensitivity: Agents run on context, and context is your business: pricing logic, customer histories, tribal knowledge. If the prompts and files an agent consumes would hurt you in a competitor’s hands, keep that context in an environment with contractual no-training guarantees, or on infrastructure you own. Public tools on personal accounts fail this test automatically.
3. Economics and rhythm: Agent inference is a steady, predictable, always-on workload, exactly the profile where owned or private capacity amortizes well, while burst experimentation still favors cloud flexibility. Watch the pricing fine print too, because promotional API rates now expire on a schedule, and an always-on agent fleet feels a price change immediately.

One more honest note from the roundtable: a participant experimenting with a local agent platform on a spare desktop admitted the learning curve is real. Plan on serious hours before a self-hosted agent stack feels routine, which is exactly why the environment decision should follow the workload, not the hobbyist enthusiasm.

What this looks like in practice

The pattern across our client work is consistently hybrid. One manufacturer chose a custom AI build inside its existing infrastructure specifically to answer privacy concerns, keeping sensitive data in the environment it already trusted while still getting frontier capability. Another organization is expanding a locally hosted AI knowledge tool from one user to a multi-user, permission-based system, growing governance with the footprint. And one technology leader told us plainly that his team’s recent AI projects all run locally because the data was not suitable for an environment they could not verify.

None of them picked a side in the on-prem versus cloud debate. They sorted workloads: sensitive context and high-blast-radius agents on infrastructure they control, commodity tasks on the cheapest reliable API, and a routing layer in between so no single provider decision is permanent. If you are earlier in that journey, our guides on running LLMs locally and choosing a cloud LLM provider cover each half of the map.

Sort the workloads, not the ideology

The gap between AI aspiration and AI that works often comes down to this: aspiration argues about cloud versus on-prem, while AI that works asks what each agent touches, what it consumes, and what it costs to run all day. That is IT Intelligence doing its job, and it is the diagnosis we run before anything gets built: map the agents you want, score their blast radius and context sensitivity, and place each one deliberately.

Your agents are coming either way. Decide where they live before they decide for you, and put AI to work for your people on ground you control.

Frequently Asked Questions

An IT leader at a mid-market manufacturer told us he was racing to get his company’s first AI policy to a working version within the week. He had a borrowed template from his managed service provider open in one window and a blank page in the other. Then his infrastructure partner asked a simple question: do you have any data privacy requirements around AI, like no training on your data, no retention, no PII? His answer was honest. Not yet.

That moment plays out in mid-market companies everywhere. Your people adopted AI first, and the rules are trying to catch up. An AI acceptable use policy is how you close that gap, but only if you write one that people can actually follow.

Why most AI policies fail before they launch

Two failure modes kill most first attempts. The first is silence. At a town hall we heard about, an employee asked leadership directly what the company policy on AI was, and the answer was that one did not exist yet. In one of our operations workshops, a leader admitted the same thing plainly: no governance policy, nothing. Employees in that vacuum make up their own rules, and the most common question they carry is whether it is even okay to use AI at all.

The second failure mode is overcorrection. One leader we work with borrowed a policy from a friend at a cybersecurity firm, and it was so restrictive it effectively banned AI outright. A policy that reads like a ban does not stop AI use. It just stops honest AI use, and pushes everything underground where you cannot see it.

A policy that reads like a ban does not stop AI use. It just stops honest AI use.

The numbers say most companies are still stuck between those two failures. Only about 15 percent of organizations have updated their acceptable use policies to cover AI specifically, according to research compiled by JumpCloud, even as unsanctioned AI use shows up in nearly every workforce.

Decide what your policy is for

At a recent executive roundtable we hosted, the sharpest question of the session was deceptively simple: is this policy a structure for the organization to operate with AI, or a defensive legal and HR document? The answer shapes everything. A defensive policy protects the company from its people. An operating policy protects the company through its people, by giving them a safe, clear way to work.

Write the operating version. It should answer, in plain language, the questions your employees are already asking: which tools are approved, what data can go where, and what to do when they want something new.

The six sections your AI acceptable use policy needs

Keep it to one page if you can. Here is the structure we help clients build:

1. Approved tools: Name the sanctioned tools and accounts. If AI use must run through IT approval on enterprise licenses rather than personal accounts, say exactly that.
2. Data rules: Spell out what can never enter an AI tool, such as customer records, financials, and regulated data, and where protected work can happen safely.
3. Verification duty: If you get output from AI, you verify it before it ships. Define what verification means for high-stakes work like quotes, contracts, and customer communication.
4. Accountability: Every AI workflow and agent rolls up to a named human owner. When an agent produces a bad quote, the accountability trail should already exist.
5. The request path: Give people a fast, judgment-free way to propose new tools. This turns shadow AI into a pipeline instead of a threat.
6. Ownership and review: Name who signs the policy, who manages it, and when it gets revisited. A policy nobody owns goes stale in a quarter.

Roll it out like you mean it

Publishing a PDF is not a rollout. Two moves make the difference. First, put adoption metrics on managers, not just individuals. In one client rollout, training participation jumped significantly the moment completion became a management team metric rather than an individual one. Second, pair the policy with real training so people understand the why behind each rule; the gap between publishing rules and building skills is covered well in our piece on the hidden cost of not training your team on AI.

Your policy also does not live alone. It is the front door of a broader governance posture, and our guide on what every executive needs to know about AI governance covers the structures behind it. And once rules exist, adoption becomes the next battle, so keep getting your team to actually use approved AI on your radar.

From policy to activation

The gap between AI aspiration and AI that works is rarely about the technology. It is about whether your people know the safe path and trust it enough to take it. Across our client work we have seen what happens when they do: leaders standardizing on governed platforms so company information stops flowing through personal accounts, teams coordinating so three people are not quietly building the same agent, and manager-led rollouts that measurably lifted training completion.

That is Company Intelligence working alongside Security Intelligence: rules that amplify your people instead of restraining them. Diagnose where AI is already carrying weight in your business, write the one-page policy that makes it safe, and stand behind it in production.

Put AI to work for your people, with rules they can actually follow.

Frequently Asked Questions

 That’s $29k per employee. Per year.

In a recent Augusto leadership meeting, the point was made that our client roundtables generate a massive amount of intelligence worth sharing across the company. Nobody in the room treated that as a nice sentiment. We treated it as an operating requirement, because we had already built the system that makes it true: every conversation we have is captured, mined, and turned into knowledge the whole company can use.

Most companies run the opposite way. Their best thinking happens out loud, in meetings, and then most of it disappears. The decisions, the commitments, the client saying something that belongs in a case study, the early warning sign in a customer’s tone. It all happened, and a week later almost none of it exists anywhere. That gap between what gets said and what gets kept is the whole argument for AI meeting intelligence.

The Evaporation Problem

The research on this is brutal. Without follow-up notes, 70 percent of decisions made in meetings are forgotten within 24 hours, and 47 percent of action items discussed are never captured at all. This is not a discipline problem. People speak faster than anyone can reliably record, and the person taking notes is also trying to participate.

Now put a price on it. The average employee spends 392 hours a year in meetings, at a cost of roughly $29,000 per employee per year, and executives average around 23 hours a week. Estimates of what unproductive meetings cost US companies run from $37 billion to $399 billion annually. Your company is already paying full price for these conversations. The only question is whether you keep what you paid for.

What Meeting Intelligence Actually Means

Most leaders hear this topic and picture an AI note-taker that emails a summary nobody reads. That is the shallow version. AI meeting intelligence means the conversation becomes structured, searchable business data. We know because we built this for ourselves before offering it to clients.

The capture layer comes first, and ours is standardized: every call at Augusto is recorded with Fathom, our company-wide AI note taker. We like it because it does the unglamorous part flawlessly. It joins every meeting automatically, produces an accurate transcript with speaker labels and timestamps, and delivers a summary with action items before anyone has left the call. Just as important for what comes next, every moment carries a deep link, so any quote or decision can be traced back to the exact second it was spoken in the recording.

From there, every call flows into a system we call our Second Brain, and this is where Fathom’s clean transcripts become compounding value. AI reads each Fathom transcript and extracts the action items, with owners. The system identifies ROI signals when clients describe measurable outcomes in their own words, detects relationship signals such as appreciation, concern, or renewal risk, and maps attendees, their connections, and trending topics across every engagement.
Before a client meeting, anyone on our team can pull a prep brief built from the entire history of the relationship. When we write a case study, the client quotes are real and verbatim, because the system kept them. This is the same agent workflow pattern we apply to any document-heavy process, pointed at conversations instead. It even feeds our marketing: the proof points in our content pipeline, described in how Augusto automates content creation, come out of these meetings.

What Changes When Conversations Stop Evaporating

The first shift is follow-through. Teams that adopt AI-generated meeting summaries see action-item completion rates rise from the 50 to 60 percent range to 85 to 95 percent, largely because a searchable record kills the follow-up meetings scheduled to clarify what the last meeting said.

The second shift is institutional memory. When a veteran team member leaves, their client context does not leave with them, because two years of their conversations are searchable. When someone new joins an account, they read the relationship’s history instead of asking five people to repeat it. And leadership stops flying blind between quarterly reviews, because concern and satisfaction signals surface from calls as they happen, not months later in a churned account. None of this requires anyone to change how they run a meeting. The meetings you already hold become an asset instead of an expense, which is the cleanest ROI case we know. Our guide on measuring AI ROI before you invest shows how to baseline it.

How to Start

The pattern is simple. Record by default, with clear consent and sensible exceptions. Pipe transcripts into a system that extracts action items, signals, and people, rather than a folder of files nobody opens. Connect the output to where work actually happens, your CRM, your project tool, your team channels. Keep a human reviewing anything that drives a decision. Start with client-facing calls, where the evaporating value is highest, and expand from there.

Your meetings already contain the intelligence. The only thing missing is the system that keeps it. If you want to see what a Second Brain would look like on your own calls, start a conversation with our team.