Price a cold-email test honestly: 2,000 sends per variant costs a few hundred dollars in list, tooling and hours — but the verdict it produces decides a sequence rewrite, a new ICP, a quarter of a rep's calendar. Sizing the test at 200 sends doesn't save money; it moves the cost out of the budget line and into the decision, where it's ten times larger and nobody sees it. The thresholds are not opinions: ~1500–2000 sends per variant for reply rate, 100–500 for subject-line opens, 100–200 targeted visitors for a landing-page demand read, 1–3 target CPA plus 48–72 hours before an ad leaves the learning phase. Below the threshold, "winner" and "noise" are indistinguishable — and you will act on whichever one you were hoping for. Fifteen years of B2B practice, compressed into one file your agent can actually read; install cost is zero, and one campaign it stops you from killing on noise pays for it many times over. https://axelfreeman.github.io/marketing-mindset/ — npx skills add axelfreeman/marketing-mindset — What's the cheapest test in your setup right now, and what decision is quietly riding on it?
How do you account for the diminishing returns on test size, though? I've seen cases where 2500 sends per variant didn't add much more confidence than 1500.
You mention 1500-2000 sends per variant for a reliable reply rate - have you seen similar thresholds for email warm-up sequences, or do cold-email tests require a different approach?
How do you determine the optimal number of sends per variant for reply rate, is it solely based on industry benchmarks or also on specific business goals?