AI Engineering 6 min read

Three AI agents we built and what broke

Three kinds of agent connected to real business systems, with the clients’ names left out. What each one does, what it touches, and what we found and fixed once real work went through it.

Founder and Head of AI Engineering

Most of what gets published about AI agents is a list of what agents could do. This is three kinds of agent we built for Australian businesses: what each one does, and the faults we found and fixed once real work went through it.

I've left out the clients' names and identifying details, so some of this is deliberately general. It comes from our run logs and the notes we keep on every fix.

A question tool on a company's own data

What it does. Staff ask a question in plain English inside their AI assistant and get an answer from their own business data, without anyone building a report.

What it touches. A synced copy of the company's data, pulled together from its CRM and other systems. People sign in with their existing company login.

How it's being used. A small team uses it.

The question tool, and where each fault sat
How the question tool works, with the five faults markedThe CRM and the company's other systems feed an overnight copy of the data. The question tool reads that copy and answers staff inside their AI assistant, and every lookup is written to a run log. Fault 1 sits on the assistant's standard CRM connector, which it used instead of the tool. Fault 2, the missing lookup, sits on the question tool. Fault 3, yesterday's data, sits on the overnight copy. Fault 4, the lookups that errored, sits on the run log. Fault 5, uneven use, sits with the staff. CRM Other systems Overnight copyrefreshed once a day Question toolreads the copy Staffin their AI assistant Run logevery lookup, failures too the assistant's standard CRM connector 1 2 3 4 5

Numbers match the list below. The dashed line is the route the assistant took on its own before we told it to try the tool first.

What we found and fixed

  1. The assistant reached for the wrong tool. In the first demo we asked about a live deal and the assistant used its standard CRM connector instead of ours. One instruction in the organisation's settings fixed it: try this tool first, and fall back only if the data isn't there.
  2. A missing lookup. The same test showed a deal stage with no name mapped to it. Reading the code wouldn't have found it. A real question about a real deal did, which is why we test with the client's own questions.
  3. Yesterday's data. The copy refreshes overnight, so a deal created that morning isn't there yet. That's by design for a tool that looks across every system, but people expect a chat box to be live. Tell people that before they start.
  4. Some lookups errored. We could count exactly how many, and which ones, because every lookup is logged, failures included.
  5. Uneven use. A few people ran most of the lookups. Giving a team access to a tool is a different job from getting them to use it.

A morning briefing written from yesterday's activity

What it does. Each morning it reads the previous day's activity, finds what connects, and writes a short briefing for the people running the business: what changed, why it matters and what to do about it. We have built this more than once, and the faults below come from two of those builds.

What it touches. CRM activity, read through a synced copy of the data, and email delivery at the other end.

The morning briefing, and where each fault sat
How the morning briefing works, with the six faults markedCRM activity is synced hourly into a copy. Early each morning the briefing agent reads the copy, writes the briefing and emails it to the people running the business. If the data is late, the agent waits and alerts a person. Fault 6, the bulk change that stalled the sync, sits on the synced copy. Faults 2 and 4, sample data and running before the data arrived, sit on the link between the copy and the agent. Faults 3 and 5, too much in one request and writing for the wrong reader, sit on the agent. Fault 1, the empty recipient list, sits on the email. CRM activityyesterday's work Synced copyupdated hourly Briefing agentearly each morning Emailrecipient list Leaders Hold and alertif the data is late 6 2 4 3 5 1

Numbers match the list below. The hold and alert step is one of the fixes: the briefing now waits for the sync to finish.

What we found and fixed

  1. An empty recipient list. On one build, most of the first scheduled runs failed, and the run that worked had no recipients set. It's why we check the inbox, and the run log after it, before we call an agent live.
  2. Sample data left in place. The same build still pointed at the sample data used while it was being built, so the briefing described a made-up business with complete confidence. Checking which data an agent reads is now part of going live.
  3. Too much in one request. One step sent a whole day's inbound messages in a single call with a capped reply. We split the work into fixed-size batches.
  4. It ran before the data arrived. The briefing ran early in the morning and the data sync finished hours later. The freshness check was loose enough to pass, so a briefing went out on a nearly empty day. It now retries until the sync has finished, and holds and alerts a person if it runs too late.
  5. Written for the wrong reader. On the other build, the people reading it told us the new format had lost substance. The cause was one line in the prompt, which described the reader as staff on a phone, so it wrote staff instructions and flat lists for the people running the business. We changed the line, and the sections went back to closing on a finding.
  6. One bulk change in the CRM stalled the sync. The CRM re-stamped a huge number of emails at once. The hourly job hit its time limit, saved no progress and read the same slice again each hour. Its alert repeated hourly too, and an alert that fires every hour is easy to tune out. The job now saves its place as it goes.

Planning an agent that runs while nobody's watching?

Tell Matt what it needs to do each morning. He will talk you through what we check before one goes live, and who hears about it on the day it doesn't run.

Book a call with Matt

A call note writer for a team that records its client calls

What it does. A team records its client calls with a call recording tool. The agent turns each recording into a structured note and attaches it to the right CRM record. When it isn't sure which record a call belongs to, the note waits for a person.

What it touches. The call recording tool and the CRM, plus a daily digest we added later.

The call note writer, and where each fault sat
How the call note writer works, with the four faults markedRecordings come from the call recording tool to the note writer, which writes a structured note. If it is sure which record the call belongs to, the note is attached to the CRM record. If not, the call is held for a person to place, and held calls go into a daily digest. Fault 3, endless retries, sits on the link from the recording tool. Faults 1 and 2, the template the API would not accept and thinking that used up the budget, sit on the note writer. Fault 4, held calls with no queue, sits on the review step. Call recordingone per client call Note writerwrites the note Sure whichrecord? CRM recordnote attached Held for reviewa person places it Daily digestof every held call yes no 3 1 2 4

Numbers match the list below. The daily digest is one of the fixes, added so held calls get seen.

What we found and fixed

  1. A note template the API wouldn't accept. One template was more complex than the API's limits on structured output allow, so calls using it failed. We split it into two requests and added a test that covers every template.
  2. Thinking used up the budget. On longer calls, the model spent its whole token allowance thinking and returned no note. We lowered the effort, raised the limit and allowed one retry.
  3. Endless retries. A recording that failed tried again on every cycle. Now it backs off to hourly.
  4. Held calls needed a queue. Many calls were held for review, because the recording titles didn't carry enough to tell which client or deal a call belonged to. Holding them was correct. So that someone sees what's waiting, we built a daily digest of held calls.

What the three have in common

Almost none of these faults came from the model's judgement. They came from everything around it: the recipient list, the sync, the request format, the retry and the alert. So these are the things we check before any agent goes live. How we test the model's own answers is in how we test an AI agent before it goes near a customer.

  • Silence looks like a quiet day. A note that doesn't attach and a briefing sent to nobody don't announce themselves. Decide what "it didn't run" looks like, and who hears about it.
  • An alert nobody reads is the same as none. A failure that alerts every hour ends up as easy to miss as one that says nothing.
  • Green tests aren't delivery. Check the thing a person actually receives.
  • Log every run. Much of what's in this post came out of a log.
  • A person's check needs a queue someone reads. A review step nobody knows about is the same as no review step.

If you're talking to someone about building one of these, ask to see what their last agent did on the day it failed, and how they found out. The answer tells you more than the demo.

Frequently asked questions

What goes wrong with AI agents once they are live? In these builds, mostly the parts around the model: a recipient list left empty, a data sync that finished after the agent had already run, a request format the model’s API refused, and alerts that repeated often enough to be tuned out.

Should an AI agent update your CRM on its own? Only when it is sure, and with someone able to see everything it was not sure about. The call note agent above holds anything it cannot place. The caution was right. What it needed was a daily list of held calls, so someone sees what is waiting.

Sources

Read next

Neighbourhood builds AI agents on Claude for Australian businesses, as part of our AI engineering work.

Got a job your team keeps working around?

Tell Matt what it is. He will talk you through whether it needs an agent at all.

Book a call with Matt
Neighbourhood

Neighbourhood is a HubSpot Diamond Partner in Brisbane. We build AI systems and the revenue operations they run on, for businesses across Australia and New Zealand.