A/B test subject lines

I need to run an A/B test on this upcoming promo to see which subject line drives opens. Send the campaign to our April list with two subject lines: “Spring launch — 20% off for three days” and “Limited: 20% off starts Thursday.” Randomize recipients evenly, hold statistical confidence at 95%, let the winner after 48 hours auto-send to the remaining subscribers, and pause if unsubscribe rate exceeds 1% during the test. Confirm the test settings and scheduled send time before launching.

Before I run this subject-line split, make the test easier for a busy reviewer: surface the projected sample size needed for a 95% confidence win, show estimated additional opens each variant must generate to be decisive, highlight any words that may trigger spam filters (like ‘Free’ or excessive punctuation), and flag if our send window conflicts with other mailings to the same list in the last 48 hours.

I sent both subject lines to a small seed group and the second variant got higher opens, but I’m not sure if that’s noise or a true advantage. The list is 12,000 with mixed engagement, and past tests swung after 72 hours. Should I let the 48-hour rule run, increase sample size, or pick the seed winner now and roll out to avoid missing the promo window? Tell me the likely cause and the best next step.

Across campaigns we waste time running subject-line splits that never reach significance because lists skew between active and dormant users and we start tests during irregular send windows. How should I change the habit — sample selection, timing, or test length — so we get decisive results without delaying launches? Recommend one operational change and how to enforce it.