DeDuplication

Deduplication helps to avoid indexing duplicate records in the search engine.

The Deduplication is an optional parameter (query) that can be passed in the API request of Parse and Index. Once Deduplication is passed in the request parameter of Parse and Index, it will avoid indexing duplicate records in the search engine.

In Deduplication, you need to set checkDuplicate = true, only then duplicate records will be checked. You can set delicacy checks on field level as well as on output level. By Default if it is on the output level and if no fields are set, then the same output is generated for any record is considered as a duplicate record.

You can check the output with the help of md5 checksum hash code. The other way is to set the fields or the combination of fields to avoid duplicate records. You can set the updated parameter to true to update the records in the existing ID. By default, it is false, means the duplicate records will not be updated and will keep the old value.

"deDuplication":
    {
      "checkDuplicate":true,
      "fields":["Email","FullName"],
      "update":true
    }