AI staff performance comes down to one question. Is the work moving faster and cleaner than before, with you still making the calls that matter? If yes, they're helping. If you spend more time checking their work than they save you, they're not, and something in the setup needs fixing. Judge them like any new hire in the office. Not on how busy they look. On whether the job moves.
This is for operators. That means the contractor business that runs the clients, the subbies and its own operatives. I'll go through how I'd check each of the three AI colleagues in Plan@Job, what to look at, and what stays your decision however good they get.
What does helping actually look like?
Activity is not a result. A pile of sent messages means nothing if the jobs stuck on Monday are still stuck on Friday.
Helping looks like this. Briefs reach you complete enough to price. Quotes and proposed dates come back without you having to remember to chase them. Deposits and overdue payments are ready for you to approve, not sitting in your head. And none of it creates new mess, like confused clients or subbies getting the wrong message.
Here's the part people hate hearing. If your process was a mess before, AI staff won't fix it. They'll just show you the mess faster.
Write down how it runs now, before you judge anything
You can't say something improved if you never wrote down how it was. Before you switch anything on, or this week if you already have, keep a rough note for a week or two:
- How many job briefs came in missing something you needed to price
- How many subbie quotes you had to chase, and how long you waited
- How many proposed dates you were still waiting on
- How many deposits or overdue invoices you chased late, or forgot
- How long your morning takes before you know what's going on
A notebook or a basic spreadsheet is enough. This doesn't need to be clever. It needs to be honest.
I run the back office from Tangier and do the UK site visits myself. I can't stand over anyone's shoulder, human or AI. So my rule is simple. I look at what was stuck at the start of the week and whether it's still stuck at the end. That one check tells me more than any activity count ever has.
How to check each AI colleague
They do different jobs, so judge them on different things. Don't mark Sophie down for something that was never her job.
Sophie Bennett, AI Enquiries Coordinator
Sophie checks existing job briefs for missing information and asks for clarification. If she still doesn't have what she needs after two attempts, she hands it to a person. She does not turn every email in your inbox into a job, so don't judge her on that.
What to check: are the briefs reaching your pricing stage complete? Are her questions the ones you would have asked? And when she hands off, does someone actually pick it up?
Illustrative example, not a real job: a brief comes in for a bathroom refit with no access details and no photos of the existing room. Sophie asks the client for both. After the second attempt there's still no reply, so it goes to a person. If someone in your office phones the client that afternoon, the system worked. If it sits for a week and you blame the AI, that's on you.
Daniel Reed, AI Operations Manager
Daniel supports eligible sharing of jobs with your existing vetted network, follows up on quotes and proposed dates, and puts together a morning brief.
What to check: are quotes coming back without you personally chasing? Are proposed dates landing in time to plan the week? Do you actually read the morning brief and act on it? If you skip it every day, find out why. Either it isn't telling you what you need, or you haven't built the habit. Both are fixable, but they're different fixes.
What he doesn't do is award the job or set the price. Those are yours.
Maya Collins, AI Finance Coordinator
Maya prepares deposits and overdue payment actions for your approval. Finance runs on fixed workflows, so you're not judging her creativity. You're judging whether the right actions are ready at the right time.
What to check: are the actions there when you need them? Do they match your own records? And how long do they sit before you approve them? If they sit for days, the bottleneck is not Maya.
What still needs your decision
This part doesn't change, however well the AI staff perform. A person decides:
- Who gets awarded the work
- What price goes to the client
- Whether a payment action is approved
- First contact with a new client, which normally needs your approval
- Anything outside the job brief or the agreed scope
If you catch yourself wishing the AI would just make these calls, that's not a performance problem. That's the line. Your name is on the job and your margin is in the spread. You protect it.
Don't blame the wrong tool
Plan@Job has three different AI things, and people mix them up when they're judging what's working.
AI staff are the background colleagues above. Ask Plan@Job is you asking for something in conversation and acting on it yourself. The MCP connection links your own compatible external AI client to your account, set up by following the current connection guide.
Judge them separately. If something went wrong in a conversation you started, that's not Sophie. And paying for your own external AI subscription doesn't power the hosted staff. They're separate things.
Signs it isn't working, and what to fix
Most of the problems I'd expect come from the setup around the AI, not the AI itself.
If Sophie keeps asking about things that don't matter for your work, get clear on what a complete brief means for your jobs. Then make sure whoever takes enquiries asks for it up front.
If quotes still don't come back after Daniel's follow-ups, look at your network. A follow-up can't make a subbie want the work. If the same people never respond, that's a network problem, and you've just found it.
If approvals pile up, it's you. Put the approval queue in your morning routine, same as checking the van.
If a client or subbie seems confused by a message, stop. Read the thread, sort it out with them yourself, and raise it with the Plan@Job team so it gets looked at.
A simple monthly review
Once a month, sit down with your before notes and do this:
- Compare how many incomplete briefs reach pricing now against before.
- Count how many quote and date chases you still did yourself.
- Check how fast you cleared payment approvals, and whether any were wrong.
- Ask whoever picks up Sophie's handoffs whether they're getting dealt with quickly.
- Pick one thing to change next month. One. Not five.
If two or three of those are clearly better and nothing new has broken, they're helping. If nothing has changed, look at your process before you look at the AI.
If you want to compare notes with other operators running jobs this way, that's what we do in Contractor Club.
My verdict: AI staff earn their place the same way a good office hire does. Jobs move, nothing slips, and you spend your time on the decisions only you should make. If you're on Plan@Job and haven't set them up properly, start with how the AI staff work. If you're not on it yet, see how it runs for contractors in the UK.
FAQ
How long before I can judge AI staff performance properly?
Long enough to see a normal run of enquiries, quotes and payments go through. For most operators that's a few weeks, not a few days. Judge too early and you're judging teething problems, not the system.
Can AI staff award jobs or move money without me?
No. Daniel follows up quotes and proposed dates, but he doesn't award work or set prices. Maya prepares deposits and overdue payment actions for your approval. She doesn't move money.
Does my own AI subscription make the AI staff better?
No. The hosted AI staff run separately from any external AI subscription you pay for. Your own AI client connects through the MCP connection for your own use, and that's a different thing.