Someone unsubscribes. Three weeks later they get the newsletter again.
They didn't opt back in, and nobody made a mistake. The unsubscribe was written to one record, and the send used a different one. Same person, two records, two consent states, and the system did exactly what it was told.
That's the version of this problem that gets escalated. The rest of it is quieter and more expensive.
Duplicates aren't a hygiene failure
The standard advice treats duplicates as sloppiness — bad imports, undisciplined reps, buy a dedupe tool. That framing is comfortable and it's wrong, which is why disciplined teams keep accumulating duplicates anyway.
The actual cause is that your two systems disagree about what a person is.
The MAP usually thinks a person is an email address. One address, one record, and it will refuse to create a second. The CRM thinks a person is a record ID, and it maintains two separate object types that can both describe the same human at the same time. Add multiple business units, each with its own uniqueness scope, and the same person can legitimately exist five times without a single validation rule being broken.
Neither system is misconfigured. They're answering different questions. Every duplicate is that disagreement made visible.
Once you see it that way, the fix stops being "clean up" and starts being "decide, then enforce" — which is a different project with a different owner.
The ones that come back
Here's the pattern that makes people give up.
A duplicate gets merged in the CRM. The loser record is gone, the survivor has the history, everyone moves on. Two days later it's back.
The MAP never heard about the merge. It still holds its own copy of the loser, and on the next sync cycle it does what an integration is supposed to do: it sees a record that doesn't exist downstream and creates it.
Most teams fix the same duplicate three or four times before someone works out that the sync is undoing the cleanup. We've seen instances where a monthly dedupe job and a bidirectional sync had been quietly fighting each other for over a year, each one confident it was working.
Merging on one side of a sync without a matching operation on the other isn't cleanup. It's a delay.
What it actually costs
Scoring stops working. Engagement splits across records. One copy has the webinar and the pricing page visit, the other has three emails. Neither crosses the threshold, so a lead that should have been routed sits below the line on both records. The scoring model isn't wrong. It's being fed halves.
Two reps call the same person. Different owners, different records, no shared history. The prospect notices even when the reps don't.
Attribution counts one person twice. Two records, two first-touch sources, two entries in the funnel. Any channel that tends to generate duplicates looks better than it is, which means budget follows the duplicates.
Consent fragments. The opening example. Preferences live on records, not on people, so an opt-out only stops the sends that use that record. This is the one that turns a data quality issue into a compliance conversation.
The asymmetry nobody plans for
Duplicates are recoverable. Bad merges are not.
A duplicate is annoying and reversible — the data is still there, just split. A merge destroys the loser. Activity history, campaign membership, original source, whatever was stored on the record that lost: gone, in most systems permanently, and the audit trail that would let you reconstruct it went with it.
Now consider what happens when the match was wrong. Two people at the same company with the same first initial and a shared info@ address. A parent and child with the same name. Two records that a fuzzy matching rule scored at 91% and merged automatically. You've destroyed two histories and created a consent record that belongs to nobody.
So the correct bias is toward under-merging. Leave the ambiguous pairs alone, route them to a human, accept that some duplicates survive.
This is the opposite of how most dedupe tools ship. Default settings are tuned to demo well, which means matching aggressively and reporting a large number of records cleaned. That number is the wrong metric. The right one is how many merges you'd have to reverse if you could.
Write the survivorship policy first
Before any tool runs, decide what survives. Not record-level — field-level. "Keep the oldest record" is not a policy, because the oldest record often has the stalest job title and the newest one has the current phone number.
Go field by field and answer three questions:
Which system owns this field? Job title probably belongs to whatever the person last told you. Lifecycle stage probably belongs to the CRM. Pick one, write it down.
What wins on conflict? Most recent, most complete, or a specific source. Different answers for different fields, and the differences are the point.
What can never be lost? This is the short list that matters. Consent and unsubscribe status: the most restrictive value always wins, regardless of recency. If either record says no, the merged record says no. Original source and first-touch timestamp, if attribution is going to survive the cleanup. Anything with a legal retention requirement.
That last question is worth more than the other two combined. Everything else is a preference. That list is what makes a merge safe to run unattended.
Order of operations
The sequence matters more than the tooling, and getting it wrong is how a cleanup becomes an incident.
Pause the sync. Not after. First. Merging live while an integration is watching produces the zombie pattern, and at volume it produces it faster than you can fix it.
Take a backup you have actually tested restoring. An export you've never opened is a hope, not a backup.
Merge in the system of record, then reconcile the other side. Pick a direction. Bidirectional cleanup during an active sync is how records get resurrected from both ends at once.
Start with the exact matches. Same email, same domain, no ambiguity. That's usually the bulk of the volume and close to none of the risk. Getting the easy tranche done builds the confidence to look properly at the hard one.
Route the fuzzy matches to a human. Yes, it's slow. It's slower than reversing a bad merge, which you can't do.
Turn the sync back on and watch the first cycle. Not the summary. The actual record counts on both sides, before and after.
Preventing the next batch
Cleanup without prevention buys you about a quarter.
Normalise at capture. Trim whitespace, lowercase email addresses, standardise country and state values at the point of entry rather than in a nightly job. A surprising share of duplicate pairs differ only by a trailing space or a capital letter.
Reduce the number of doors. Every form, every list import, every integration that can create records is a place duplicates enter. Route them through one path with one set of rules. If a tool needs its own write access, it needs to respect the same matching logic.
Gate list imports. Most bulk duplicate events trace back to a spreadsheet someone uploaded before a conference. A required dedupe check before import costs one person ten minutes and saves the quarterly cleanup.
Watch the rate, not just the count. The useful number isn't how many duplicates exist — it's how many were created this week. A flat count tells you nothing. A rate tells you when something changed, and it's usually a new tool or a new form that nobody mentioned.
The part that's hard to justify
None of this makes a quarter. It's the kind of work that gets deferred every planning cycle because the cost is spread thin: a slightly wrong number here, a lead that didn't route there, a complaint that got handled quietly.
The bill arrives all at once, though, and usually at the worst time — during a migration, an audit, or the week someone asks why the same person is in the funnel twice on a board slide. By then the cleanup is entangled with a deadline, and the ambiguous pairs get merged in a hurry by whoever is available.
The cheap version of this project is the one you do when nothing is on fire.
Deciding what a person is across your CRM, MAP, and CDP — identifier hierarchy, merge rules, and the monitoring that catches syncs recreating what you just cleaned up — is data and integrations work. If you're not sure how many duplicates you're carrying or where they're coming from, a stack audit will tell you, and the findings are yours either way.