Ask for a system that is running in production, talk to the engineer who built it, and get the operating cost in writing before you get a proposal.

Almost every firm in this market can now produce a convincing deck, because the deck is the easiest part to generate.

What separates them is whether anything they built is still working six months later, and whether they can tell you what broke along the way.

We are an Australian firm that does this work, so read this accordingly.

We have also lost pitches to people who answered these questions better than we did, which is roughly how we ended up with the list.

The four questions that sort the field quickly

1. What have you got in production, and can I speak to whoever built it?

Not a case study. A system, running now, with a person attached. The tell is whether you are put in front of an engineer or in front of an account manager. Most AI work fails at the integration and evaluation layer, which account managers cannot discuss.

If every example is a pilot, a prototype or a demo, you are buying a first attempt.

2. What did you get wrong on it?

The best question in the set, because it cannot be prepared convincingly by someone who has not done the work.

An honest answer sounds specific and slightly boring: the data was worse than we thought, the model was confidently wrong about a category we had not tested, an integration rate-limited us, the client's team stopped using it because the approval step was too slow. A firm with real production experience has a stock of these. A firm without one will answer with something that is secretly a strength.

3. What will this cost to run each month at my volume?

AI systems charge per unit of work, so the operating cost scales with use. A partner who cannot model this at your actual volumes has not run one at volume. Get the number in writing and get it at your volume, not per action.

4. If we part ways, what do I keep and can I run it?

Whose accounts does it run in, where do the prompts live, whose API keys, which repository, and is there documentation a stranger could use. Ownership and operability are different, and under Australian law the copyright default runs against you: a contractor generally owns copyright in commissioned code unless the contract assigns it. That is the reverse of the American position, so American-sourced advice on this is wrong here. We covered it in who owns the AI you paid someone to build.

Australian specifics worth insisting on

Some of this genuinely cannot be answered by an overseas firm, or by a local one working from overseas material.

They should know which framework applies to you. There is no AI Act in force in Australia. What applies is the Privacy Act and the APPs, the automated decision-making disclosure commencing 10 December 2026, ASD's agentic AI guidance from 1 May 2026, the National AI Centre's Guidance for AI Adoption, and APRA's CPS 234 and CPS 230 if you are regulated or supply someone who is. A partner who answers "AI compliance" with GDPR or the EU AI Act is telling you where their material came from. We mapped this in which Australian AI framework applies to you.

Data residency should be a specific answer. Which region, named. Ap-southeast-2 is the Sydney region and a partner who has thought about this will say so without being asked twice.

They should be able to answer a buyer's security questionnaire with you. If you sell to government, financial services or larger enterprise, that questionnaire is coming. See the AI security questionnaire your buyer's CTO is about to send.

Reading the proposal

Three things that predict how the project will go.

Does the scope include evaluation? If there is no line for building a test set from your real cases, nobody has planned to find out whether it works. This is the most common omission in AI proposals and the most expensive.

Is the data work priced, or assumed? Data readiness is part of the build. A proposal that treats it as a prerequisite you will handle is a proposal with a variation in it.

Does it start narrow? A first phase covering one process with defined cases is a partner planning to prove something. A first phase covering four departments is a partner planning to bill something. ASD's agentic AI guidance recommends deploying incrementally starting with clearly defined low-risk tasks, for security reasons that happen to align with commercial ones.

Partner badges, and what they are worth

Certifications tell you a firm has done the training and met some volume of work. They are a floor, and they say nothing about whether a firm can build a custom system on top of a platform. Plenty of highly certified partners configure software very well and have never shipped a custom AI system.

Use them to filter out the bottom, then ignore them and go back to question one.

Questions that sound good and are not

"How many AI projects have you delivered?" Volume without production survival is meaningless. Ask how many are still running.

"What is your AI methodology?" Everyone has a diagram now. Ask for the system instead.

"Which models do you use?" Fine as a detail and useless as a filter. The model is the least durable decision in any build and it will change.

What we do, so you can hold us to it

We put the engineer who built the thing in the meeting. Everything runs in the client's accounts, with prompts in the client's repository. We publish real prices, because vague pricing wastes both sides' time. And we answer the "what did you get wrong" question with specifics, because we have plenty.

In short

Ask what is in production and meet the person who built it. Ask what went wrong on it. Get the monthly run cost at your volume in writing. Confirm what you keep and whether you can operate it.

Check they know Australian obligations rather than European ones, make sure evaluation and data work are priced, and be suspicious of a first phase that covers your whole business.

Frequently asked questions

What should I ask an AI implementation partner? What is running in production, whether you can speak to the engineer who built it, what went wrong on that build, what it will cost per month at your volume, and what you would keep and be able to operate if the relationship ended.

Do AI partner certifications matter? They establish a floor, showing a firm has completed training and delivered some volume of work. They do not indicate whether a firm can build custom systems, so use them to filter and then assess production experience directly.

What Australian regulations should an AI partner know? The Privacy Act and the Australian Privacy Principles, the automated decision-making disclosure commencing 10 December 2026, ASD's Careful adoption of agentic AI services guidance from 1 May 2026, the National AI Centre's Guidance for AI Adoption, and APRA CPS 234 and CPS 230 for regulated entities and their material service providers.

Should an AI project start small? Yes. A first phase covering one process with clearly defined cases lets you measure the result before committing further, and it matches ASD's recommendation to deploy agentic systems incrementally beginning with low-risk tasks.

Sources

Putting this list to a few firms? Put it to us too. Talk to us.