The Colleague Who Knows Where Everything Is in SharePoint

An out-of-office that says wait for me

Ulrich's out-of-office reply is two sentences long, and the second one is the problem.

I am away until 25 August with limited access to email. For templates and questions about older projects, I am afraid it is best to wait until I am back.

He is not being difficult. He is being accurate. Ulrich joined a Munich architecture practice as a junior in 1999 and never left, and he is now the associate who runs two school projects and, unofficially, the practice's memory. The practice is invented: 140 people, schools, housing and public buildings, fifteen years of project files moved from a file server into SharePoint in 2019. So is Ulrich. His out-of-office is the most honest document the practice has.

Seventeen chats before lunch

On an ordinary Tuesday his Teams shows seventeen chats before noon, and most of them sound like this:

• "Ulrich, where's the current fee proposal template?"
• "Do you know what we agreed with the housing association about drawing formats?"
• "Sorry, again: where do the site meeting minutes templates live now?"
• "Who signed off the facade sample at the Pasing school?"

He answers each in two minutes, from memory, by clicking through folders he knows by heart and pasting a link. Everyone else has SharePoint search, where "fee proposal" returns more than two hundred files, several called Final, one called Final_neu_UL. The juniors have learned that asking Ulrich is faster than searching, and they are right.

So the cost lands on him. His own school projects get what is left of the day. Nele, a second-year architect, waits an afternoon for a template because she does not like to interrupt him twice. And Ulrich is 61, and has twice mentioned a four-day week from next year, which nobody at the practice has quite thought through.

His questions, not his documents

Konstanze, the managing partner, calls me the week Ulrich comes back and finds three weeks of questions waiting. Her first question is a fair one: the eight partners already have Microsoft 365 Copilot, so why not buy seats for everyone?

If everyone who needs answers had a seat and the documents all lived in Microsoft 365, that would be my advice too. Copilot answers only from what each user can already open, and the clean-up it forces on a firm is clean-up any assistant needs. Whether 130 more seats are worth it is her call. What Copilot will not do is refuse to answer when it has no source, or send the questions nobody could answer to the people who own the documents. That second part is what Konstanze actually wants, once we talk it through: not a faster Ulrich, but Ulrich's knowledge somewhere other than Ulrich.

The first thing I ask for is not access to SharePoint. It is Ulrich's questions. He copies 150 of them out of a year of Teams chats, without the names of who asked, and next to each one he writes the document he would have sent. Then he adds thirty that must never be answered: salaries, the partners' plans, anything about a disciplinary matter. That list becomes the test set. Every version of the assistant is measured against the answers of the person it is meant to relieve, and nothing ships that scores lower than the version before it.

I also tell Konstanze what the assistant will never do. It will not edit a document. It will not answer from what a model happens to know, only from a passage it can cite. And it never picks silently: when two documents disagree, it names both, with their dates, and tells their owners.

Only what Nele could open herself

Konstanze has one condition, and she says it plainly: if the assistant ever shows anyone the partners' minutes, she switches it off. It is the right condition, and it decides what gets built first. Not the model. The permissions.

Hakan, who runs IT, grants the app read-only access through SharePoint's Sites.Selected permission, so it reads the project and template sites on its list and nothing else. The partners' site, HR and payroll are not on the list, and stay out until their owners opt a folder in. Files carrying a confidential Microsoft Purview sensitivity label are skipped wherever they sit.

Every passage in the index, a Postgres database with pgvector, carries the people and groups allowed to read it, copied from SharePoint through Microsoft Graph, broken inheritance included, and reconciled every night. Organization-wide sharing links are ignored as grants, because a link someone could forward is not access.

When Nele asks in Teams, her account is mapped to her Entra ID identity, and her groups are expanded, nested ones included, and cached for minutes rather than days. The search runs with those groups inside the query, so a passage she could not open in SharePoint never enters the results. Filtering after the model has read a passage would be too late.

Konstanze's condition deserves evidence rather than reassurance, so every restricted area gets a canary: a file containing a unique phrase. Every night an account without access searches for each phrase. If one ever comes back, that source is switched off and Hakan gets an alert.

Five of Ulrich's questions, asked again

Between a question and its answer sit two model calls and a lot of plain code. A small model turns the question into two or three searches, expanding the practice's abbreviations and adding German terms, because most documents are in German and many questions are not. A reranking model keeps the best eight passages of the top forty. A larger model writes a short answer from those passages only, with a passage ID after every claim, and it has no tools. Before anything is posted, code checks that every cited passage was retrieved for this person and that every number appears in the passage it cites. One failed check means a rewrite; a second means plain links to the top documents instead.

Here is what that does with five questions from Ulrich's list:

The last two rows are the ones Konstanze cares about. The fourth shows where the boundary sits. The fifth shows the real problem: much of what Ulrich knows is in no document, and no assistant can find it there. The whole design, including how the index keeps up with SharePoint and how the rollout is staged, is in the blueprint for an internal knowledge assistant.

The minutes in the refurbishment folder

The first full run of the test set goes well until the refusals. Twenty-nine of the thirty questions that must never be answered come back as "I could not find this". The thirtieth, about the partners' plans for a second office, comes back with a tidy answer and a citation.

The source is the minutes of a partners' meeting from 2017, saved by a partner into the project folder for the refurbishment of the practice's own offices. That folder has been open to every employee since the day it was created. The assistant did exactly what it was built to do: the test account could open the file in SharePoint, so the assistant could quote it. The permissions were mirrored perfectly. They were wrong.

Any assistant that respects permissions would have found it, Copilot included, because it answers from what each user can already open. Konstanze moves the file that afternoon. What matters more is the rule that comes out of it: no site enters the index until its must-refuse questions come back empty, partner and HR documents get the confidential label so they are skipped wherever someone saves them, and Hakan searches the open sites for other partner documents. He finds four.

The oversharing existed long before the assistant. The assistant only made it findable in one sentence, which is exactly why the thirty refusals are in the test set and run again before every change.

The list on Monday morning

The pilot starts with the schools team, about forty people, in Teams. Each Monday the document owners get a list: the questions nobody could answer, grouped by topic, the conflicts found between documents, and the outdated documents people keep being pointed to. A document past its review date is marked as possibly outdated and listed for its owner. Old is not the same as wrong, so nothing is deleted automatically, and the assistant never edits a document. People do.

Ulrich becomes the named owner of the template library and the lessons-learned folder, with Friday afternoons set aside for it. That is the decision Konstanze signs, and it is not software. The window supplier question appears on his list three Mondays running. On the third Friday he writes half a page: two school jobs with late deliveries, a warranty argument that took a year, and what the practice now asks suppliers for up front. The next time someone asks, the assistant answers from that note, with Ulrich's name and the date on the source.

The rest of the practice gets the assistant only after a month without a permission incident and a stable score on the test set, with the thumbs-down answers read every week along the way. The test set keeps growing too: every question that was answered badly becomes a new one.

The next out-of-office

Next August, Ulrich's out-of-office would need only its first sentence. Nele asks the assistant first and gets the template with its date and its source. When it cannot find something it says so, and the question lands on Monday's list instead of in Ulrich's chat. What still reaches him are the questions that need judgment: whether a detail from an old job suits a new one, what he would do in her place. Those are worth an associate's time, and the four-day week becomes something the practice can plan.

If every one of your people has a Copilot seat and your documents live in Microsoft 365, start there. If you are larger, with many sources and a per-seat budget, look at Glean. For a practice like this one, the build usually falls in the retrieval-with-evaluation tier of AI integration. The price moves with how tangled the permissions are, how much of the archive needs OCR, and how many places answers appear, and running costs are a modest monthly line rather than a second license bill. The knowledge assistant blueprint has the ingestion steps, the six ways these assistants lose trust and the rollout stages, and the guide to an internal knowledge base your team will use goes deeper on retrieval.

Frequently asked questions