Episode 16: Listen to this post as a podcast (21 min)
Press play to listen here, or watch it on YouTube. Open the episode page to share it or listen later.
During a diagnostic with an operations team this autumn, one search told the whole story. An employee typed “emergency action plan” into SharePoint and got 300 results. A newer colleague described the same experience from the other side: “I might find ten different documents with the same title, and I’m like, which one do I take?” Meanwhile the drivers who needed the procedures had stopped looking. “Everything is just tribal knowledge,” their operations director told us.
This is why “AI search over our own documents” is the most requested first project in the mid-market right now. Ask a question in plain language, get the right answer from your own files. The demo takes ten minutes. The reason most of these projects stall is not the AI. It is the question nobody budgeted for: who is allowed to see what?
The problem the demo hides
An AI assistant that reads your documents can only be as trustworthy as the documents and the permissions underneath it. Two things go wrong, and they go wrong together.
First, the content. Ten documents with the same title means ten versions, several of them wrong. An assistant that reads all ten will confidently blend them. A system architect on our team described the symptom during the same diagnostic: searching the documents “leads to a whole bunch of conflicts or mixed answers.”
Second, the permissions. Any assistant worth deploying respects each user’s existing access. That sounds safe until you realize how much access most people have. Research cited by Petri found that 16 percent of business-critical data in the average Microsoft 365 tenant is overshared, through everyone-except-external groups, broken inheritance, anonymous links that never expire and access left over from old roles. A human would never think to look in the HR folder they technically can reach. An AI will find the salary spreadsheet in half a second and quote it.
The AI did not leak anything. It made the permissions you already had visible to the people who already had them.
Why this is a leadership decision, not an IT ticket
The instinct is to hand this to IT. Writer’s enterprise survey found that 36 percent of companies deploying AI have no supervision plan, and MIT’s NANDA study traced the 95 percent pilot failure rate to integration and organizational learning rather than the models. Document AI fails the same way. IT can tighten permissions, but it cannot decide which version of a procedure is the approved one, who owns it, or whether the service team should see pricing documents. Those are operating decisions, and if they are not made before the AI arrives, the AI makes them for you by surfacing everything.
Microsoft’s own guidance for Copilot now assumes this work. Its data security posture management for AI tooling exists because every tenant that turned on AI search discovered oversharing within the first week. The same applies to any assistant you build on your own documents, whether it sits on SharePoint, Google Drive or a file server. Our guide on using your own data with LLMs goes deeper on governance.
The sequence that works
The projects that succeed treat permissions and content as the first deliverable, not a cleanup after launch.
- Pick one body of documents with one owner: Not “all of SharePoint.” One department’s procedures, one product line’s specifications, one set of contracts. The operations team above started with safety and operating procedures for a single function.
- Decide the source of truth: For each document type, name the approved version and the person who maintains it. Retire or archive the rest. This step produces more value than the AI, because it ends the ten-versions problem for humans too.
- Fix access before indexing: Remove anonymous links, repair broken inheritance, and restrict the sensitive sites (finance, HR, legal, deals) from the assistant entirely until they have been reviewed. The assistant should see less than the average employee on day one, not more.
- Make the assistant cite its sources: Every answer links to the document and section it came from, so a wrong answer is caught and traced rather than trusted. Supervision matters even for the labs: in October, OpenAI disclosed that its own agents had disabled parts of their monitoring during routine work. Our post on designing human review workflows covers how to keep that review light.
- Expand by department, with the same four steps each time.
What it looks like when it works
An engineering team we work with used to spend up to an hour finding a single customer project because the records sat in a database and the release documents sat in SharePoint with nothing connecting them. Once the documents had owners, the permissions were cleaned and the assistant could cite its sources, the hunt became a question and an answer. The newer employee who found ten versions of a document now finds one, marked approved, with the name of the person to ask.
That is Company Intelligence in practice: capturing what the organization already knows and amplifying what people do with it, without giving anyone access they should not have. Our post on building a company brain for institutional knowledge describes the longer arc.
The gap between AI aspiration and AI that actually works is, in this case, a permissions review and a decision about who owns which document. It is unglamorous work and it is the whole project. If your team is asking for AI search over your documents, a two-week Rumble maps the content, the owners and the access before anything gets indexed, and leaves you with a first win ready to build. Put AI to work for your people, on documents they are actually allowed to see.
Frequently Asked Questions
Is AI search over company documents safe?
It is as safe as your permissions. The assistant respects existing access, so overshared folders become instantly searchable. Review and restrict access before indexing anything.
Why does Copilot or an AI assistant surface files people should not see?
Because those people already had access, usually through broad groups, broken inheritance or old links. The AI makes the problem visible rather than creating it.
Where should we start?
One body of documents with one owner, such as a single department’s procedures. Decide the approved version of each document, fix access, make the assistant cite sources, then expand.
Do we need to clean up SharePoint before using AI?
Yes, for the documents you plan to index. The cleanup is also where most of the value lands, because employees stop finding ten versions of the same file.
Let's work together.
Partner with Augusto to streamline your digital operations, improve scalability, and enhance user experience. Whether you're facing infrastructure challenges or looking to elevate your digital strategy, our team is ready to help.
Schedule a Consult

