Who can commit to running your AI pilot? A worked example


November 8, 2026 | Read time: 9 mins.

Kevin Donovan · Architects' Hub
​
Forwarded this email? Subscribe to Architects’ Hub for the next issue.

Hey ,
​
In this edition of Architects' Hub:

  • What the service lead heard when the sponsor said, “We should roll this out.”
  • A commitment record filled in for one example customer-service queue, PLUS a blank copy for yours.
  • How to take a conditional decision back to the sponsor who wants a date.
  • Community Corner: Santiago Jesus Lasso Valenciano on the work that has to come off the queue.

Let’s get into it.👇

________

🧭

________

​

The chair arrived. Where’s the desk?

Imagine you’re the architect in this demo. A customer wants to know where her order is. The agent opens the assistant, which checks the order history and shipment record, then drafts a reply. The agent reads it, adds a personal line and hits send. The sponsor watches the whole thing take less time than it usually takes to find the tracking number.

“We should roll this out.”

The service lead has a question. Yesterday she handled an order for a desk and a chair. The chair arrived. The desk was still on a loading dock. Both were on the same order, which the order system called shipped. What would the assistant tell that customer? And who picks up the case if the agent can’t tell which record to trust?

I picked this hypothetical case for November because the service lead and the sponsor can both be right. The demo gives the sponsor a good reason to want the tool. The service lead knows what happens when a customer writes back, “Then where’s my desk?” We’ll follow this same assistant through the month of November. Today we’re deciding what the demo gives us grounds to commit to.

The demo can finish while all 4 commitments are still open.

What would have to be true on Tuesday morning?

I’d ask the sponsor to stay for 10 more minutes. Take tomorrow’s shift and work backward. An agent gets the split-shipment case at 9:10. She knows enough to hold the draft. Now what?

The operating owner, here the service lead, can tell us who takes that exception. She can also tell us what that person is already doing at 9:10 am. If the proposal assumes the senior agent is always available, ask who covers his queue while he investigates. That’s a staffing decision with a cost.

The resource owner can accept that cost. Ask what budget pays for licenses, integration support and the senior agent’s time once the project closes. “The pilot is funded” answers a different question. If the budget holder is willing to fund one queue for a month, record exactly that. It’s useful information for the next decision.

The data owner knows what shipped means in each system. Can the team read shipment lines, how fresh are they, and who tells the service lead when a status changes meaning? Permission to read a field doesn’t settle whether an agent can rely on it in a reply.

The decision owner sets the scope and can stop use when a condition fails. One person might hold 2 of these roles. That’s fine. Ask the person for both commitments. A sponsor who also holds the budget can authorize spending without having the authority to grant access to shipment data.

This is where I’d spend the next architecture hour. Another pass through the happy path would give us a nicer demo. Following one awkward order tells us whose agreement we need.

________

🧭

________

​

The commitment record, filled in

Here’s the page I’d take back to the sponsor. The roles, dates and proposed targets below belong to the example. Nothing has been measured or approved just because it appears on the page.

1. Outcome. Customers with delayed orders get an accurate reply within 2 hours, with less agent time spent looking things up. That’s the proposed target. Before the test, take a sample of today’s queue: how long until a useful reply, how many minutes of agent work, and how often does a second agent have to correct the answer? A quick acknowledgment doesn’t count as a useful reply.

2. Decision. The head of customer operations records: not yet, until exceptions capacity and ongoing data access and support are agreed. The decision covers one delayed-order queue. Agents check every draft. No automatic sends, credits or refunds. Come back on November 20 to decide whether to run that scope, change it, investigate a remaining question or stop.

3. Resource owner. The head of customer operations will put the run costs to the budget holder. The license and integration estimates exist. Nobody has committed the hours for exception handling. Leave the funding line open until the budget holder accepts an amount, a period and the staff time it buys.

4. Operating owner. The service team lead is willing to take the queue once she has a named backup for split shipments, a support route with an agreed response time and permission to retire an existing reporting task. Put the task on the page. If someone still needs the report, they need to be in that conversation too.

5. Conditions. The data owner confirms which order and shipment fields the team may read, how late an update can be, and who answers when the records disagree. The service lead and budget holder agree who covers exceptions on each shift. Before go-live, the delivery and support leads walk one bad draft through escalation and back to the normal manual process. Ask each owner for a date they accept. November 20 is the sponsor’s review date, not a deadline that everyone else has somehow agreed to.

6. Missing evidence. We don’t know how often a split shipment produces a misleading draft. We also don’t know how much work the assistant saves when its builders aren’t sitting beside the agents. Those questions need different tests.

A checked box records the decision. The other 5 fields tell the next person what they can act on.

How I’d use the next 2 weeks

First, test the answer. Ask the exceptions handler to select 50 historical cases the team is permitted to use. Include split shipments, stale updates and straightforward orders. Keep the categories separate: 40 easy answers can hide a pattern in the 10 difficult ones. For each draft, record what the customer would have been told, what actually happened, and whether the agent spotted the difference.

Agree the stop rule before looking at results. For example, the service lead might say that a draft promising delivery of an unshipped item, missed by the reviewing agent, rules out that case type for now. That finding could narrow the scope to single shipments. Fifty cases can expose a failure worth acting on; a clean sample can’t promise it will never happen.

Then, test the shift. For one working week, keep agent review and the agreed support route in place. Let agents use that route instead of having an engineer coach them through every case. Count the agent’s minutes, the exceptions handler’s minutes and any build-team interventions. Compare similar cases with the baseline. If the assistant saves the agent 4 minutes but costs a senior colleague 6, the queue has bought speed at one desk by borrowing time from another.

Before starting, ask the resource owner what saving would justify the proposed coverage. On November 20, show the answer alongside the quality findings. If either owner can’t yet agree to a test or to coverage, record that. Time passing isn’t evidence.

Copy this for your own pilot

Give it 20 minutes. A first pass is useful even if half the page is blank.

  1. Outcome: Which customer or team benefits? How will we compare time, effort and quality with today?
  2. Decision: Who can choose run, expand, not yet until, or stop? For which scope? When do we revisit it?
  3. Resource owner: Who has accepted which costs and staff time, for how long? What is still a proposal?
  4. Operating owner: Who takes the queue? What support, coverage or retired work does their acceptance depend on?
  5. Conditions: What must be true? Who can agree to each condition, by what date, and where will we see their acceptance?
  6. Missing evidence: What don’t we know? Who will test it, and what result would change the decision?

When the sponsor wants a start date

Sponsor: “The demo worked. Can we start next month?”

Architect: “I’d like customers to get that faster answer too. We can test one delayed-order queue. The service lead needs cover for split shipments, and the data team hasn’t agreed to support the shipment feed. Who can commit the hours for cover?”

Sponsor: “Bring the operations budget holder in. What do you need from me on the 20th?”

Architect: “A decision on that one queue. We’ll bring the quality results, the time it took on shift and the owners’ answers. If split shipments are the only sticking point, you can decide whether single shipments alone are worth running.”

The sponsor now has a meeting worth keeping. There’s a person to invite, a choice to make and a reason for the date.

________

🧭

________

​

Community Corner

​Santiago Jesus Lasso Valenciano hears a question that belongs on the operating owner’s line: “What does my team stop doing?” His practical addition is to retire the old task in writing. Otherwise the new tool goes live and the old report is still due on Friday.

​Michael D. Stark changed how I wrote Tuesday’s post. In his distinction, a proof of concept can succeed by teaching you enough to stop. A pilot tests a practical uncertainty on the way to an intended rollout. His example was training: could front-line staff use the new workflow with release notes, or would they need dedicated time to learn it? Those are different requests to a service lead.

​Jordan Corn gave the sponsor a smaller next step. Even after a successful pilot, he builds out support, procedures and other parts of the “whole product” as the next decision requires them. For our example, that means working out how to support one queue before asking what it would cost to support every queue.

________

🧭

________

​

Where to start Monday

Pick a pilot with a rollout meeting on the calendar. Fill in the 6 headings before the meeting. Find the first blank where you’ve written someone else’s name beside a commitment they haven’t made. Take that line to them and ask what they’d need to say yes.

Next Sunday we’ll follow the same assistant onto an ordinary Tuesday shift, when the people who built it have gone back to their own jobs.

If you’re the architect being asked to make a pilot real, forward your filled-in record to the person who has to sign it. If you’re the leader who owns that decision, reply with 2 lines: the workflow and the missing commitment. I’ll tell you whether the Compass diagnostic fits or whether a shorter conversation would do.

Until next Sunday,

Kevin


Work with me

Hey ,
I run Compass Architecture Advisory. I help enterprise and technology leaders turn AI ambition and architecture into decisions they can fund, govern and run.
Right now that’s a 4-week AI Architecture and Adoption Diagnostic. If there’s a decision like that in front of you, reply to this email with 2 lines about it and I’ll tell you whether it fits.
​

________

________

Kevin Donovan​
I write Architects' Hub every Sunday for senior architects and technology leaders.
You're getting this because you subscribed at architects-hub.kit.com.
Know someone who’d find this useful? Forward it to them.

​

Korsvejen 9, Viborg, Viborg 8800
​Unsubscribe · Preferences​

Architects' Hub

A Sunday newsletter for senior and enterprise architects. One decision, one practical model, one tool you can use Monday. Written by Kevin Donovan. Join 2,500+ architects. One email every Sunday.

Read more from Architects' Hub
Three doors slightly ajar in a row. Through them, a telephone, a stack of documents with a magnifying glass, and an arrow curving back on itself. A single coin rests on the floor in front of the doors.

Nov. 22 2026 | Read time: 11 mins. Kevin Donovan · Architects' HubForwarded this email? Subscribe to Architects’ Hub for the next issue. The room went quiet Hey , Two Sundays ago, our hypothetical pilot case got its commitment record. Last Sunday it got a walkthrough, and the operating owner signed with 3 tests on record. The AI assistant now drafts delayed-order replies for 3 teams, and the drafts are good. This week someone proposed letting it do more than just draft… In this edition The...

Five simple figures in a ring around a small central box, each holding one object: a magnifying glass, a signpost, a wrench, a mop, a headset. One figure carries a leaning stack of boxes.

Nov 15, 2026 | Read time: 11 mins. Kevin Donovan · Architects' HubForwarded this email? Subscribe to Architects’ Hub for the next issue. Tuesday, 10:40 am. Hey ,Last Sunday we left a hypothetical customer-service assistant with a commitment record and unanswered questions about capacity, data and support. This week, move the story forward 3 weeks. The team has agreed to a bounded trial on 1 queue. Agents still review every draft. Wider rollout is waiting. I want to spend this edition with the...

A hand-drawn ledger has columns Visible, Committed, Acted and Receipt. Three rows contain circled date markers, with one receipt clipped to the first row. A pencil rests below the sheet.

Nov 1, 2026 | Read time: 10 mins. Kevin Donovan · Architects' HubForwarded this email? Subscribe to Architects’ Hub for the next issue. Where the weeks went Hey Reader, A warning arrives early. Permission arrives late. The recovery gets called heroic. Mike Carroll opens his article on decision latency with a composite plant-floor scene: a machine shows signs of trouble, but the person who can approve a controlled shutdown isn’t available. The line keeps running. Hours later, the bearing...