Tip #627: Smart duplicate detection

Woot-woot! Our video channel now has the official URL: https://www.youtube.com/CRMTipOfTheDay! (Hint: take a minute and subscribe). To celebrate this achievement, our video dude, Derik, has gone MIA. After 38 straight weeks of videos, can’t hold it against him.

So today’s tip is not a video but something that has been on my todo list for quite some time. In fact, the original code spans back to CRM 2011.

DVD RewinderDuplicate detection has always been a pet hate of mine – it’s a cool feature but the interactive dialog has usefulness level of a DVD Rewinder.

But the most annoying part is not even the dialog, it’s the fact that it pops up after all data entry is done, party is over and everyone’s gone home. Imagine frustration of the users who duly enter all contact data including marital status, fax number, blood type and assistance name, only to find out that the contact already exists.

There is a smarter way to do duplicate detection – by making it proactive and real-time. Enter RetrieveDuplicate message – we can send it to the server when attributes change to see if the record is a potential duplicate as user enters the data. How do we know what attributes to monitor? duplicaterule and duplicaterulecondition entities to the rescue, they define the attributes taking part in the active duplicate detection rules.

The solution steps (for CRM2015+):

  1. Download and install the unmanaged solution. Because it’s unmanaged, to remove it, you’d need to nuke by hand the resources it installs (there are only 4 of them)
  2. Create a copy of a contact form – no need to experiment on the good ones.
  3. At the very top, add a section and insert web resource alex_DDNotification.html. Name the control DDNotification.
  4. Add alex_xrmsvctoolkit.0.2.js and alex_duplicatedetection.js scripts to the form.
  5. Add call to AlexDuplicates.OnLoad to the form onload event.
  6. Make sure some duplicate detection rules are activated.
  7. As the form is loaded, code will retrieve duplicate rules from the server and attach onchange handlers to the form fields that are part of the rules.
  8. When any of these fields change, code will send RetrieveDuplicateRequest to the server to get the potential list of duplicates.
  9. If duplicates are found, warning section will be displayed listing the potential duplicates in the web resource area. User can continue as normal but now they are fully aware that record is a potential duplicate.

Smart Duplicate Detection in Action

Since users are now notified well in advance, you can go to Settings > Data Management > Duplicate Detection Settings and switch off duplicate detection on create and update.

This solution has been ported from CRM 2011 so before you pull out pitchforks and apply spießrutenlaufen:

  • It still uses XrmSvcToolkit. Yep, works fine.
  • It still sends custom-grown soap and fetch to the server.
  • It was not aware back then that jQuery on form script is a no-no (separate topic!)
  • Should be rewritten to use web api or at least SDK.Soap.js
  • Should be wrapped up as a single html web resource that combines both code and visualization.
  • Should deploy some timers to cater for fast operators
  • Contains small parts, not suitable for children, does not drive or operates heavy machinery.

BUT you get the idea!

Special thanks go to Oleksandr Klymenko for the initial implementation, and some good ideas and suggestions.

7 thoughts on “Tip #627: Smart duplicate detection

  1. Tom Phillips says:

    Nice! One of my biggest annoyances with duplicate detection is that if fires on-create and on-save with many customers I speak to only want the warning on-create. This ultimately means the feature doesn’t get used or the rules are so specific (e.g. Same first three letters of company name + exact match postcode) that other duplicates will slip through the net.

  2. Graham says:


    I have followed the instructions (currently on CRM 2016 Rollup 1) and loved the idea of this solution, however, when I open a record, it shows as duplicates detected, but does not show the possible duplicate records?

  3. Emily says:


    Thank you for your post !
    Unfortunately it does not work for me on Dynamics 365 on-premises, following your instructions. What could I do wrong please ? The duplicate detection rules are all activated…

  4. Dnyaneshwar Erande says:

    Did anyone found any resolution to get this solution working for Dynamics 365 version?
    Thank you.

  5. Fatma says:

    Where is the solution?

Leave a Reply

Your email address will not be published. Required fields are marked *