Modern Code Consulting
Book a call

Automation · 12 steps

Cold email reply analysis: positive reply rate and the copy behind every reply

Every campaign's numbers and every reply go in. The positive reply rate, what changed since last time, and the lines of copy behind the replies come out.

Built by Matt Cruz, Modern Code Consulting

The problem

A sending tool counts replies but it does not read them. Automatic replies inflate the count, and a reply the tool scores as negative can be the lead that books a call.

The bigger miss is the copy. When a pattern shows up in the replies (a merge field that read wrong, a question about price the email never answered) nobody ties it back to the exact line that caused it.

What I built

I built this review to read every reply in full. Two Python scripts pull the campaign numbers and the live copy from Instantly, net out automatic replies and work out what changed since the last snapshot.

A Claude Code skill classifies each reply, works out the positive reply rate against a 1% line and sets each reply pattern beside the live line of copy behind it, with a suggested tweak. It does not change the live campaign.

How it works, step by step

A Claude Code skill reads the replies and writes the review. Two Python scripts pull the numbers and the live copy.

Stage 1 · Pull from Instantly

  1. Pull campaigns, steps, replies

    Substituted

    Asks Instantly for every campaign's totals, the per-email and per-variant rows, and every email received in reply.

    Instantly API

Stage 2 · Clean the numbers

  1. Net out automatic replies

    Ran

    Builds one row per campaign: unique replies minus automatic ones, leads actually emailed, bounce rate and status.

    instantly_retro.py

  2. Compare with the last snapshot

    Ran

    Loads the newest snapshot dated before today and works out what moved since: emails, real replies, opportunities and bounces.

    instantly_retro.py

  3. Split by variant, strip quotes

    Ran

    Splits each campaign by email and variant, drops Instantly's junk rows, cuts the quoted original out of each reply and writes the two files.

    instantly_retro.py

Stage 3 · Read every reply

  1. Read and classify every reply

    Ran

    Reads all 30 messages in full and records, for each: which email it answers, any mistake that cost credibility, what it says about the offer, and its class.

    cold-email-retro skill

  2. Compute the positive reply rate

    Ran

    Works out reply rate, positive reply rate against the 1% line, first-email sends and positives per variant, and automatic replies per 1,000 emails before and after the last snapshot.

    cold-email-retro skill

Stage 4 · Check the live copy

  1. Pull the live copy per variant

    Substituted

    Reads each campaign's sequence and prints every variant as plain text with its merge variables, so findings can point at the live wording.

    instantly_campaign.py

  2. Tie reply patterns to the copy

    Ran

    Sets each pattern from the replies beside the live line that caused it, and writes a tweak for each without changing anything.

    cold-email-retro skill

Stage 5 · Confirm, log, report

  1. Confirm calls and closes

    Substituted

    Asks the owner for the three numbers Instantly cannot know: sales calls booked, closes and revenue from cold email.

    The owner

  2. Log the snapshot to the tracker

    Not run

    Appends one row per campaign to the tracker's ColdEmail tab, once per snapshot date.

    Google Sheets

  3. Draft the community post

    Not run

    Reads the owner's last retro post, audits the tweaks it promised, and drafts the next post in the owner's own voice for them to paste.

    cold-email-retro skill

  4. Report numbers and tweaks

    Ran

    Hands the owner the numbers, the reply patterns, the recommended tweaks, the two counts that disagree, and what was not done.

    cold-email-retro skill

The recorded run

This is a real run of the automation on an invented business: a payroll service for restaurant groups. Use the tabs inside it to move between the overview, how it's built, the step-by-step run and the result.

What ran for real: Invented sender, campaigns, replies, copy and earlier snapshot. Instantly and Google Sheets were never called: the pull and the live copy are invented responses read by the real, unmodified scripts, the owner's answer was written in beforehand, and the sheet log and community post were left out. The parsing, deltas and quote stripping are the real instantly_retro.py run through its own main function; the reading, the rates and the findings were done in a Claude Code session following the skill.

Open the run full screen(opens in a new tab)

The run is laid out for a desktop screen.

More automations

All automations

Want something like this running in your business?

Book a free 15-minute call. Tell me where the hours and the leads are leaking, and I'll tell you what I'd automate first and what it would cost.

Book a free automation audit