Tackling duplicate records in Salesforce can be a real challenge. Fortunately, platform expert Gijs Hovens has prepared this guide to help you plan your approach to Salesforce duplicate management, with relevant solutions based on your circumstances.
Are you working with Salesforce? No doubt you have encountered duplicate records in your Salesforce organization. For example, a salesperson from your org calls a lead who turns out to be a client. Since this person had two records (a lead record and a contact record), the salesperson did not know about the lead already being a customer.
Not only personal experience confirms a duplicate problem in CRMs. According to Gartner, bad data is the number one cause of CRM system failure. Forrester states: ‘Insight is the engine of today’s businesses. The value of a company can be measured by the performance of its data.’
Fortunately, there are solutions to combat duplicate data. The most accessible is the built-in Duplicate Management feature of Salesforce. We will explore this feature and find out when you will need to look for more sophisticated solutions.
Browse our free candidate search to start connecting with certified Salesforce Admins. We can help you find the next Salesforce expert for your business today.
What is Salesforce Duplicate Management?
Salesforce Duplicate Management is a core feature of Salesforce included in Essentials, Professional, Enterprise, Performance, Unlimited, and Developer Editions. It consists of three elements: matching rules, duplicate rules, and duplicate jobs.
Matching rule
Matching rules are about finding duplicates: you define how to match records as a duplicate. You can match records within an object or across two objects. A matching rule compares fields in a pair of records with different matching algorithms. Salesforce comes with three standard rules: one for business accounts, one for contacts and leads, and one for person accounts.
Duplicate rule
A duplicate rule determines what happens when a matching rule has identified duplicate records during a create or edit action.
Duplicate rules can trigger two outcomes:
- An alert and/or report for the user creating a duplicate
- The blocking of the creation or editing of a duplicate record
Duplicate job
Recently, Salesforce has improved Duplicate Management with Duplicate Job functionality for users on ‘Performance’ and ‘Unlimited’ editions of Salesforce. With a Duplicate Job, you’ll find duplicates in existing data. A job runs one matching rule at a time. The Duplicate Job produces a list of Duplicate Record Sets. From this list, you compare and merge duplicate records.
Now you know what Salesforce Duplicate Management does, it’s time to get to know its limitations.
Know the limitations of Duplicate Management in Salesforce
Finding Duplicate Records
- Is not suited to Large Data Volumes
Everyone knows that working with Salesforce can be frustrating when dealing with large data volumes. This is especially true of jobs of any kind. Salesforce is well aware of this problem and warns:
“In an org with many records, duplicate jobs can fail.”
Other solutions, such as Duplicate Check, will handle millions of records.
- Can’t do a cross-object duplicate job
Chances are pretty big that your new lead is already present in your Salesforce organization but as a Contact. To make sure you don’t work with duplicate data, cross object matching is imperative. Salesforce can do alerting on manual insert for all objects, but jobs are limited to finding duplicates within objects.
- Jobs are limited to more expensive Salesforce editions
Running duplicate jobs is essential to merge duplicates in your existing database. Salesforce has limited this feature to the Performance and Unlimited editions.
- Limited matching algorithms
Finding duplicates is a sensitive business. You don’t want to miss any duplicates, but you don’t want false positives either. To fine-tune your matching, you need precise algorithms. Salesforce Duplicate Management offers Exact and Fuzzy matching algorithms. More specialized solutions like Duplicate Check offer up to 20 algorithms for specific fields (for example, email, URL, and large text algorithms).
In addition to limited matching algorithms, Salesforce Duplicate Management also lacks a weighting method. A matching email address should count heavier than a matching first name, for example.
Merging Duplicate Records
- Blocks creation of new records
It is important to consider that the alerting and reporting of a duplicate upon creation or edit only works for manual actions. 99% of all records are added in imports or through APIs. In these cases, Salesforce will block the creation of a new record. You will lose valuable information in this process, for example, information stored in previously empty fields and campaign data.
- Does not work for all objects
Duplicate Jobs do not allow the comparing and merging of records on custom objects. Pretty important if you ask me.
- Only performs manual merges
In the list of duplicate groups, Salesforce enables you to merge record groups. The only way to go about this is to open and review each duplicate group manually. Duplicate Check for Salesforce allows you to quick merge groups without opening them and even to automatically merge all groups in the list.
- Only merges a maximum three records at a time
Chances are pretty big that your job has resulted in a list of duplicate groups that includes groups of four or more duplicates of the same record. Salesforce Duplicate Management has a limitation on merging a maximum of three records at a time. This will slow you down in processing the list.
- Does not automatically process new records
Well, blocking is also an automatic action, but according to us, blocking is not desirable. If your marketing automation processes run all the time, you cannot wait for a manual review of duplicates. In that case, the fully automatic checking and merging of new duplicate records saves you time, does not delete valuable data, and makes sure your marketing and sales machine keeps on running 24/7.
With these limitations in mind, decide if Salesforce Duplicate Management suits your needs or if you require another solution. If you need another solution, read on.
How to start looking for solutions?
We don’t recommend to try deduplication solutions randomly. You’ll get the best solution when you follow a thorough process:
- Step 1: Data analysis
Not all solutions can handle all objects and entry points. The first step is to list all the objects you need to deduplicate. Also, note if you only need deduplication within those objects or also cross-object (leads with contacts, for example) and how the records for those objects enter your system (import, manual, API, web to lead). Mark custom objects.
- Step 2: Process analysis
Outline your current and ideal process in dealing with duplicates. Are you looking for a one-time cleanup (not recommended), periodical batch cleanups (good), or prevention of duplicates (better)? Or even a combination (best) of those? Which people/roles should perform which deduplication functions? Should some parts of the process be automated?
Check with your compliance department if there are restrictions in processing personal data with external suppliers (GDPR).
- Step 3: Supplier selection & testing
Search the Salesforce AppExchange and Google for solutions. Match them against your requirements from step 1 and step 2. Select the top options for a test drive. Almost all deduplication solutions offer a free trial. Don’t just test drive the app, test-drive support as well.
- Step 4: Implementation
We advise you to configure your solution in a sandbox environment first. Deleting, merging, or blocking duplicates is final and cannot be undone. For the same reason, we advise you only to automate merging or blocking when you are fully confident your matching rules don’t turn up too many false positives.
- Step 5: Maintenance
We cannot overstate the importance of having a deduplication process in place instead of a one-time cleanup. Good quality data requires continual action.
TL;DR
In short, Salesforce offers a deduplication solution out of the box: Salesforce Duplicate Management. This solution offers basic finding and merging options for the most important objects (leads, contacts, accounts). Salesforce will look for duplicates only when editing or creating a record. Recently Salesforce has added the ability to run a job to find duplicates in an organization. This feature is only available in the Performance and Unlimited editions.
When you require large data volumes, deduplication of custom objects, automation, or a general less time-intensive way of managing duplicates, it is time to look for 3rd party solutions. Follow our process to guarantee you select the best solution for your situation.
You are very welcome to try Duplicate Check for Salesforce. Step 1 is to install our free version from the AppExchange. Step 2 is to get a free trial for our paid plans.
About the author
Gijs Hovens is a marketing manager at Plauti, a Salesforce AppExchange Program Partner based in the Netherlands. His specializations are B2B, inbound marketing, and product management.
Plauti is helping Salesforce users achieve more revenue and cost reductions by improving data quality with the highly-rated Duplicate Check and Record Validation apps.
