AI SDR Reply Handling: What an Outbound Agent Should Do With Each Reply

Sending is the easy half of outbound automation. The replies are where an agent either earns its place or quietly embarrasses you, because a reply is the first moment a real person has spent attention on your company and expects something back that reads like a human understood them.

The articles that rank for AI reply handling agree on most of the basics. They sort replies into five to eight intent buckets, argue that speed matters, draw a line where a human takes over, and treat unsubscribes as a compliance job. Good. All of that is in here too.

What none of them deal with is that the labels are not equally dangerous to get wrong.

An agent that misreads an out-of-office as a soft no loses you a few weeks. An agent that misreads "please remove me" as an objection, then drafts a friendly rebuttal, has emailed someone who opted out. That gap should shape how you build the classifier, what each label is allowed to do, and where you put the human. This page is about that gap. It follows our MCP tool permission matrix for outbound agents, which covers what the agent can touch in general. This one is narrower. It decides which of those tools the agent may use once a specific reply lands.

Pause first, classify second

The moment any human-written reply arrives from an address, every scheduled message to that address stops. Before the model reads a word of it.

This sounds fussy. It prevents a failure that has nothing to do with intelligence: a prospect writes back on Monday, the classifier stalls on an expired Gmail token, and step three of the sequence fires on Tuesday as if nobody had said anything. A pause is dumb and cheap, and that is the point. Put it in the tool layer, outside the model, and let classification take as long as it needs.

Six labels, and what each one unlocks

You can split replies finer than this. We would not, because the person reading the log next quarter will not remember the difference between "soft positive" and "curious." Six is enough if every label maps to a different set of allowed actions.

Read down the right-hand column. The only labels where the agent acts without a person are the ones where the correct action is to do less, like stopping or waiting. Every label that results in words going to a prospect stays a draft.

Rank the mistakes, then bias toward the cheap ones

Every classifier is wrong sometimes. The useful design question is which wrong answers you can live with, and the honest ranking looks roughly like this, from worst to mildest.

Opt-out read as anything else. This is the expensive one. Under CAN-SPAM an opt-out has to be honored within 10 business days and each violating email can carry a penalty of up to $53,088, and our cold email compliance guide covers GDPR and CASL on top of that. Opt-outs also rarely say "unsubscribe." They say "not for us, please don't follow up" or "take me off this," or they just reply "stop" in lowercase from a phone.

Interested read as not interested or out of office. Nobody gets fined. You just lose the one reply the whole campaign existed to produce, and you will probably never find out, because a closed sequence does not complain.

Everything else. An objection filed as interested gets a human looking at it sooner than necessary. An out-of-office filed as not interested ends a sequence early. Annoying, recoverable.

So bias it on purpose. When the model is torn between opt-out and any other label, it picks opt-out. A suppressed address that was secretly interested costs you one awkward manual follow-up from a person, if that person reads the log and decides the reply really was a yes. When it is torn between interested and anything except opt-out, it picks interested, because that label routes to a human anyway. Ask the model for its label and a one-line reason, and send anything it marks low-confidence to a person. That one-line reason is also how you debug it later.

Referrals

A referral is the warmest cold email you will ever send, so do not let the agent waste it by enrolling the new name in step one of the same sequence the referrer just declined.

Speed, without letting the agent send

Every guide on this topic says to answer interested replies fast. Fine. The usual conclusion, though, is that the agent should therefore reply on its own, and we disagree.

Speed comes from the draft already existing. If the agent has written the reply, attached the Calendly or Google Calendar link, and pinged one Slack channel with the prospect's message above the draft, a person can approve it from a phone in the time it takes to read it. That covers almost all of the speed benefit and none of the risk of an agent improvising pricing to a buyer who just showed intent. Mira books meetings through Calendly or Google Calendar, and the process we run for clients ends with the meeting on your calendar, and this reply is the step right before it.

There is also a reason specific to how we get paid. A qualified meeting, for us, is one where the prospect actually shows up. A reply that was rushed out by an agent and booked a confused buyer onto a calendar is worth nothing on that definition. The agent's job ends at attendance. Booking is the middle.

Out-of-office replies are free data

They usually contain a return date, often a named colleague, and sometimes the news that the person has left the company. Have the agent extract all of it. A return date reschedules the sequence. A departure means the contact record is stale and should be updated before anyone else on your team mails it. A named colleague is a referral that nobody made on purpose, and it should be treated with the same restraint as a real one: researched and drafted, then approved by a person.

If you want the timing logic for when to resume, our guide to follow-up emails after no response works for returning travelers as well as silent ones.

What to log

For every reply, keep the raw text, the label, the model's reason, the action taken, and who approved it if anyone did. Then read a sample every week, starting with the replies labeled not interested. That is the bucket where good replies go to die silently, and it is the bucket nobody opens.

Mira runs on OpenClaw from a Mac mini in San Francisco, which means rules like these live in config files somebody has to maintain. Keep yours short enough that somebody actually reads them.