How to deduplicate leads across multiple outbound mailboxes

Spreadsheet row uniqueness is not contact uniqueness

Two files can contain `Maya@Example.com` and ` maya@example.com ` as different text while they resolve to the same mailbox in normal operational handling. If separate imports route those rows to different sender accounts, a naive system can contact the same person twice. Normalize email addresses consistently, create a shared recipient key, and perform duplicate checks against the central contact set rather than only inside the current CSV. Deduplication is a system property, not an Excel cleanup step.

Define the key and its limits explicitly

For most outbound operations, a normalized email address is the clearest technical key for delivery state. It does not prove identity across a person's old and new jobs, aliases, or multiple addresses, so keep a separate person/company model if the business needs one. Avoid fuzzy matching names as an automatic send blocker unless the false-positive cost is understood. The deliverability guardrail should be deterministic: the same normalized mailbox should not be independently active in two conflicting sequences.

Reserve or lock recipient state before enqueueing

A race can occur when two workers import or schedule the same address at nearly the same time. Checking “not currently active” and then inserting two jobs is possible unless the database enforces uniqueness or the operation uses a transaction/compare-and-set. Add a campaign-state reservation keyed by recipient and policy scope. If the second worker loses the race, it should report a duplicate and skip scheduling rather than quietly creating another message from a different mailbox.

Re-check immediately before send because state can change

Even a correctly deduplicated queue can become stale after a human reply, hard bounce, or manual suppression. At execution time, read the latest central recipient state and verify the job is still the active eligible touch. This also protects against restored queue snapshots or delayed jobs. The final send gate should be able to answer: has this address replied, bounced permanently, been suppressed, or received another qualifying touch since this job was created?

Keep cross-mailbox campaign ownership visible

If Mailbox A owns the current sequence, store that ownership so Mailbox B's scheduler can explain why it skipped the same contact. Visibility matters when a solo founder later reviews a list and wonders why seventeen rows were not assigned. Silent deduplication looks like data loss; explicit state looks like coordination. Store the owning campaign, last sent timestamp, and next eligible action so operational decisions can be audited.

Handle re-engagement as a policy, not as a duplicate bug

Sometimes contacting the same address again months later is intentional. Define a re-engagement window and eligibility rule separately from duplicate prevention. A contact that completed a sequence six months ago is not the same state as a contact currently in an active sequence or one that hard-bounced permanently. Encoding these states prevents teams from weakening deduplication just because they need legitimate future outreach.

Test duplicates with deliberate race cases

QA the system using mixed casing, whitespace, repeated CSV imports, two workers scheduling simultaneously, and a reply arriving between queue creation and send time. Confirm that only one eligible message leaves and that subsequent attempts record a clear skip reason. These tests are more valuable than counting unique spreadsheet rows because they exercise the moments where real outbound systems create duplicates: import boundaries, concurrent workers, retries, and stale scheduled jobs.

Reserve the normalized address before queue creation

Deduplication is safest when it happens before separate mailbox workers create their own copies of campaign state. Normalize the email address, look up global suppression, then acquire a reservation or unique database constraint for the active campaign window. Only after that should a mailbox-specific job be created. If two imports race, one succeeds and the other receives an explicit duplicate result instead of silently producing two sends. This is stronger than running a spreadsheet UNIQUE function because the conflict is enforced at the point where concurrency occurs. Keep the original source rows for audit, but make the sendable identity a shared object with one current owner.

Test the race, not just the clean import

A dedupe system that works on a static CSV can still fail under real timing. Create deliberate tests where the same address appears in two uploads, two campaigns start within seconds, one worker crashes after reserving a contact, and a reply arrives while another worker is about to send. Verify that no path emits two messages without an explicit re-engagement decision. The test should also confirm that expired reservations can recover safely without losing suppression or reply state. These failure cases matter because duplicate outreach is often created by scheduler concurrency rather than obvious duplicate rows. A small sender can avoid an embarrassing and reputation-damaging class of errors with a few database-level invariants.

Normalize carefully but never invent addresses

Email deduplication should lowercase the domain and apply only normalization rules you can defend. Do not remove dots, plus-tags or other local-part characters globally because provider-specific semantics differ and a transformed address can become a different mailbox. The safest unique key is usually the canonical address as supplied after conservative whitespace and case handling, paired with explicit aliases only when your own data proves they are equivalent. Over-aggressive normalization can merge two legitimate recipients just as under-normalization can create duplicates. Keep the original source value beside the canonical key so suspicious merges can be audited.

Make duplicate prevention visible in reporting

Count duplicate rows prevented at import, duplicate send attempts blocked at queue time and cross-mailbox conflicts caught at the final send-time check. These are positive operational metrics. They reveal whether the problem is messy source data, race conditions or stale campaign ownership. If most duplicates are caught only at the final boundary, move the control earlier so workers do less wasted work. If imports are clean but queue conflicts rise, investigate concurrency. Measuring prevented duplicates turns a hidden safeguard into a signal that the outbound system can improve.

Field checklist

• Normalize email addresses into one stable technical recipient key.
• Check duplicates against central state, not only the current CSV.
• Use database uniqueness/transactions to prevent scheduling races.
• Store active campaign/mailbox ownership with the recipient.
• Re-check reply, bounce, suppression, and prior-touch state at execution time.
• Define re-engagement separately from duplicate prevention.