Contact Deduplication

Pull contacts from the CRM (customer relationship management system), detect likely duplicate records, and compile a merge/change list to a file for a human to review and approve.

When to use

• A CRM data-quality sweep is requested or scheduled.
• Import or sync jobs are suspected of creating duplicate contact records.
• Before a bulk email or outreach campaign, to avoid contacting the same person twice.
• A human asks for a duplicate report or cleanup pass on the contacts database.

Tools

• http-get: reads contact records from the CRM's REST API.
• shell-execute: runs a small script to normalize fields and detect duplicate/near-duplicate contacts.
• write-file: writes the merge/change list artifact for human review.
• send-message: notifies a human when the report is ready, when an Integration is missing, or when authorization fails.

Playbook

1. Confirm scope with the human or task brief: which CRM object (contacts), which fields matter for matching (email, phone, name, company), and any date range or list filter.
2. Call the CRM REST API with http-get (e.g. GET https://api.hubapi.com/crm/v3/objects/contacts) to pull the contact list. Authentication is injected by the org's Integration row for this host — never ask for, echo, or hardcode credentials. If the call returns 401/403 or no Integration row exists, stop and message a human via send-message asking them to create the Integration, naming the service and required scopes.
3. Paginate through all results with repeated http-get calls until the full contact set is retrieved.
4. Use write-file to save the raw pulled contact data to a working file (e.g. JSON or CSV) so downstream processing has a stable input.
5. Use shell-execute to run a small script that normalizes fields (lowercase emails, strip phone formatting, trim whitespace) and flags likely duplicates by exact email match, exact phone match, or close name+company similarity.
6. Read the script output and group flagged records into duplicate clusters, deciding a suggested "primary" record per cluster based on completeness or recency.
7. Compile the findings into a clear merge/change list: for each cluster, list the record IDs involved, the suggested primary, and the fields that would be merged or overwritten. This system cannot update or delete CRM records directly (no PATCH/PUT/DELETE tool exists) — every change must be routed to a human for manual execution or approval.
8. Use write-file to save the merge/change list as the final artifact, including a summary count of duplicates found and records affected.
9. Use send-message to notify the requesting human that the deduplication report is ready, where the file is, and that no records were altered — approval and merging is a manual next step.

Failure modes

• 401/403 on the CRM API -> no Integration configured for that host; stop and escalate via send-message naming the service and scopes needed.
• Empty result set from the pull -> verify the endpoint, object type, and filter parameters before assuming there are no contacts.
• Rate limit (429) from the CRM -> back off and retry with delay; note the rate limit behavior via update-memory if it recurs.
• CRM page or API schema has changed and fields are missing/renamed -> fall back to browser-search to confirm current API documentation before retrying.

Done when

• A merge/change list file exists (via write-file) enumerating every duplicate cluster, the suggested primary record, and the fields to merge.
• No CRM records were modified — all proposed changes are queued for human approval.
• The requesting human has been notified via send-message that the report is ready and where to find it.