The prototype

Acme Assistant: one place for employees to ask, and to get things done.

A working prototype of the production system. Acme, its people, and its policies are invented, and the HR system, policy sources, and sign-in are stand-ins for the real ones. Ask it a policy question and it answers from the policy, with the source, and only from documents you are allowed to see. Ask about parental leave and it works out what applies to you in your country. Ask for time off and it plans the dates, drafts the request, and gets it to your manager, who approves it in the same conversation. Say your laptop keeps freezing and it offers the fix first, logs a ticket only if that does not work, and hands the thread to a person when you ask. Every reply shows its work.

Watch the two-minute walkthrough below, then try it yourself, live. Four things to try, as an engineer, a manager, someone in HR, or someone in IT support:

01 "Can I carry over unused PTO?" 02 "How much maternity leave do I get?" from India, then from the US 03 "I want to book a vacation," then approve it as the manager 04 "My laptop is not working, it keeps freezing," then say the steps did not help

Watch: a walkthrough of all four use cases, captured from the running assistant. Narration by Gemini (Zephyr). Captions are under the CC button.

Now try it yourself, live →

Opens in a new tab. Free hosting, so the first visit after a quiet spell can take up to a minute to wake.

How it was built, and the decisions behind it, is chapter 05.

Architecture

How it is built.

Every message takes the same path. An employee asks in plain words. An orchestrator works out who is asking, what they are allowed to see, and what they want, then hands the turn to one of five agents. That agent does the job with Claude and the company systems, and the guardrails stay on the whole time.

How the assistant works, in five steps: an employee asks, the orchestrator works out who is asking, what they may see and what they want, and routes to one of five agents (policy search, time off, parental leave, manager, help), which uses Claude and the company systems, with access checks, a yes gate, traces and evals always on. An employee asks in plain words, where they already work 1 The orchestrator works out who is asking, what they may see, and what they want, then picks one agent 2 ONE OF FIVE AGENTS 3 Policy search cites the policy, or abstains Time-off agent plans the dates, drafts the request Parental leave agent what applies in your country Manager agent approves or denies, one write Help agent the article first, then a ticket, then a person Claude and your systems the model reasons; the HR system and the policies stay the truth 4 5 Always on Sees only what you may see Writes only after your yes Shows its steps under every answer 67 evals run on every change 1 ask · 2 the orchestrator picks · 3 one of five agents · 4 the model and your systems · 5 guardrails, always on

The orchestrator decides cheaply first. If a draft or a decision is waiting, "yes" and "cancel" belong to it. Then a short list of readable rules covers the phrasings the demo knows. Only when no rule matches does a small, fast model label the message, and if it is unsure the message goes to policy search, the one path that can only read. The decision and its reason are the first line of the trace under every answer.

Five agents, each with only its own tools:

Policy search answers questions from the policy library. The access check runs before the search, so the answer comes only from documents the asker may see, with the source. When nothing they can see answers the question, it says so instead of guessing.

The time-off agent plans and files vacation. It runs a Claude tool loop over nine HR tools: balance, holidays, the team calendar, blackout and notice rules. A deterministic planner scores the candidate windows and offers three. The agent drafts the request and files it only after the employee says yes.

The parental leave agent answers a question that lives in more than one document. It resolves who is asking, reads the global policy, notices it defers to a country supplement, reads that, reconciles the two, checks eligibility against tenure, and only then answers with citations. The answer can become a leave request in the manager's queue.

The manager agent runs in the manager's own thread: what is waiting, what coverage looks like, and a drafted approve or deny. It is scripted rather than model-driven. The decision is a single write, confirmed through the same yes gate, and then the employee is told.

The help agent takes "my laptop is not working" and answers with the knowledge base article first, filtered to what the asker may see. If the fix fails, it drafts a ticket that carries what the employee already tried, and logs it only after a yes. On the support side it opens a ticket, names the cases like it from the last two weeks, drafts the article from the resolved ones, and publishes it after a yes, so the next employee gets the fix instead of a ticket. When the employee asks for a person, the thread becomes a live chat with the ticket attached, and the support agent gets a suggested reply drafted from the ticket and the article, never sent without them.

Each use case below has its own chapter. If you are here to decide rather than to read, start with the business case.

Start here

Which one to build first.

Which of these is worth building first? Put in how many people work at your company and the four sort themselves by what they'd save in a year. One of them sits at the bottom with no number next to it, on purpose. The page also covers how to run the same exercise on the tools you're looking at, what the AI costs to run, and the three things that make this kind of math wrong.

Business case Prioritization Unit economics
Read →
Use case 01

Ask: policy search.

The demo starts here. Ask about carryover and get the policy with its source and the line "6 documents apply to you." Ask for the executive compensation bands and get a clean miss, because you don't have access to that document. Switch persona and the same question answers. One assistant, and the access check happens before the search.

Policy search agent Permissions Personas
Read →
Use case 02

Parental leave: agentic RAG.

Ananya in Bengaluru asks how much maternity leave she gets: 26 weeks, reconciled from the global policy and the India supplement, eligibility checked against her tenure. Sarah in California asks the same words and gets 16 weeks with FMLA running alongside. Then the answer becomes a leave request in her manager's queue. Retrieval as the reasoning.

Parental leave agent Agentic RAG India and US
Read →
Use case 03

Time off: an agent that plans.

"I want to book a vacation" and the assistant reads the balance and the policy, scores the best windows against holidays and team coverage, drafts the request, and waits for a yes. The manager sees it in his own thread, asks for coverage, approves in chat, and the decision lands back with the employee.

Time-off agent Manager agent Tool use Human gate
Read →
Use case 04

Get help: the article, the ticket, the person.

"My laptop is not working" gets the knowledge base article first. When the fix fails, the assistant drafts a ticket with what Alex already tried and logs it after a yes. Sam in IT opens it, sees five tickets like it in two weeks and the cause from the resolved ones, and publishes the fix as an article so the next employee never files a ticket. When Alex asks for a person, the thread becomes a live chat with Sam, the ticket attached and a suggested reply drafted for him.

Help agent Self-service Recurring issues Live help
Read →
Chapter 04

Evaluating it.

How I knew it worked, instead of hoping. What "good" had to mean before the build, the layers measured separately, precision and recall on retrieval, an optional model judge for groundedness, and 67 cases that run on every change. The suite itself is on the page, case by case.

Evals Precision and recall No-leak
Read →
Chapter 05

How this was built.

I built this site and the demo with Claude Code to learn the tool and find out what "loop engineering" really means by doing it. The process, the decisions behind the voice, the hosting, the model, and the code, and the prompts to run the same loop yourself.

Build journal Claude Code Loop engineering
Read →
What I learned

Build the working version yourself, then hand it over.

Each chapter has its own lessons. These held across all of them, and they are the reasons behind the decisions you can see on this page.

  1. I wrote the promise before the code, and the promise was fiction for a week.

    This page said the assistant recognized a harassment report and handed it to a person. Nothing in the code did that. I caught it building a table where every guardrail has to name the test that proves it, and there was no test to name. Writing the claim first is useful, because it shows you what is missing. But a claim on a page is a promise, and it ends one of two ways. I built it: a rule that stops before retrieval, names the ethics line and the EAP, offers a person, and writes no ticket and no record of what was said. Why it matters: in a real deployment that gap is not a copy error. It is someone’s account of being harassed sitting in a ticket table that IT support can read, found later by the wrong person. The demo you show and the system you ship have to be the same thing, or the demo is just a story about software.

  2. Sixty-seven tests were green and it was still wrong.

    Twice in one day. The stop rule missed “I want to report a safety concern,” because it matched the word harassment rather than the meaning. Then it read “How do I submit an expense report?” as a yes and opened a chat with HR, because the confirm check matched the word submit anywhere in a sentence. Neither appeared in the suite. Both appeared within thirty seconds of typing into the demo. Evals prove the cases you thought of. Using the thing finds the ones you did not, and then those become cases, which is how the suite got to sixty seven. Why it matters: a green suite is a floor, not a warranty. A team that ships on green alone finds its gaps the way I found mine, except in production, with someone’s pay or privacy attached, and with a customer reading the transcript back to them.

  3. The build time collapsed. The spec still matters.

    I started at the code on purpose. I wanted to experiment, so I began with prompts and built the working version as I went, deciding the rules on the fly as each use case took shape: which writes are irreversible, what a contractor hears when they ask about parental leave, who may approve what. Then I wrote the spec from what I had built, and the spec was where those rules became visible as product decisions instead of lines of code. In the old days a spec was the norm, and the document was how you made sure your own thinking was clear before anyone built. The principle has not changed; the order I learned it in has. Prompt and build to learn, then turn what the prompts taught you into the spec, because even when a model does the building, the clarity has to come from somewhere. The next use case starts from the spec.

  4. The rough cut is mine now. Marketing gets a working version to polish.

    The walkthrough video above would have been a request to marketing and a place in a queue. I made it in an afternoon, from stills and a script. The free voice on my Mac sounded like a machine. ElevenLabs did the job until its free tier ran out mid-project, and Gemini finished it for a few cents. Swapping the voice was one flag, because the narration, the frames and the assembly are a script rather than a file somebody exported. The lesson is not about voices. For something this size I can build the version that works, and the cycles a marketing team spends on a first draft go to polishing a thing that already exists.

  5. A running prototype is how you find out what a team would need.

    The same holds for the agents. With a prototype running I can try what a small team building this for real would have to have: which tools, which gates, where the traces go, what the evals must catch. I am finding that out now, on a working system, instead of writing it into a plan and hoping.

  6. Build the conversation first. Hook it up later.

    The first version was the conversation flows and the use cases on one surface, connected to nothing. Then the search behind it, then a real identity source in Google Drive. The next passes stand in for the systems an enterprise already runs, a ticketing system, a work tracker, team chat, to prove the connectors one at a time. Each pass makes it less of a demo and more of a working version.

  7. There are two ways to stop the assistant from doing something it should not. I used the second.

    The first is to tell the model in its instructions: do not describe documents this employee is not allowed to see. That is a prompt. The model usually obeys, but not always. A clever question, a long conversation, or a bad day for the model, and it can slip. You cannot prove it will never happen. The second is to make the forbidden thing impossible. Before the search even runs, the code removes every document Alex is not allowed to see. The model never receives them. It cannot describe the executive pay bands because they were never in front of it. The demo does this for every rule that matters. A time-off request is only written to the HR system if the code sees a yes in the latest message; the model cannot submit on its own. A manager's approval only goes through if the code confirms that person is the employee's manager; the model saying "Jordan approved" does nothing by itself. The prompt still carries those instructions too, but as a backup, not as the thing I rely on. Advice is something the model can choose to ignore. A rule is something it cannot get around.

  8. The fourth use case started from the spec. The front door was already built.

    Get help is the first use case I wrote before I built. Three flows as transcripts: try the article, then the ticket; the support agent sees the pattern and publishes the fix; a person in the same thread. The build reused what the first three had left behind: the orchestrator, the access rule that removes documents before the model sees them, the yes gate on every write, the trace under every reply, and the eval runner. The new work was the three flows, the seed tickets, and the two screens. The same gate that protects a time-off request protects a ticket, an article, and a suggested reply to a support agent. Twelve new cases took the suite from 46 to 58; the stop rule later took it to 67.

Hiring a product executive for enterprise AI? I'm exploring CPO and VP Product roles where I can own strategy, product and commercialization.

For the same instincts applied to your org, see the advisory work.