xConnect Contact & Interaction Trimmer
Sitecore 10.1 introduced an official tool to remove old and corrupt data from xDB called the xConnect Purging Tool included as part of the xConnect Data Tools. The purging tool is a background job in xConnect that silently removes contacts and interactions older than a specified date. The purging tool job can be triggered by calling an API with an authorization token obtained from your Identity Server or by using the xConnect DevEx CLI plugin.
The purging tool is incredibly useful to help limit the growth of your xDB shards and to keep the shards small to avoid needlessly or prematurely scaling them - A task often feared by maintenance teams for its risk and complexity. While the tool is incredibly useful, the triggering of the job must currently be performed as a manual process on an ad-hoc basis. Ideally, the triggering of the purging job would be automated and pushed to the back of mind, giving us one less maintenance task to worry about.
There are many ways you could automate the triggering of the purging tool depending on your hosting environment, but my preferred “Sitecore standard” way of doing it is to include a foundational Helix project in the solution that leverages Sitecore’s built in command scheduler. The helix project is available here for all to use:
The code repository includes a Unicorn configuration that defines a custom Command template. This Command template has an additional name/value field that allows specifying parameter values. The most important value to set is the "CutoffDays" parameter that controls the interval after which contacts and interactions become candidates for deletion.
You can use Sitecore’s built-in task scheduler to set this command to run as often as needed. Once per day is probably a reasonable frequency.
There are some additional technical details in the readme in the repository. One small detail to call out is the cut-off interval for interactions and contacts can’t be less than 180 days by default unless you make a configuration change to xConnect directly. If you want to trim your interactions to a period of less than 180 days, then this change is easy enough to make by following Sitecore’s instructions here:
Configure the Cortex Processing Engine for task registration | Sitecore Documentation
References:
GitHub - nevnet/xConnect-TrimContacts
xConnect purging tool | Sitecore Documentation
Web API for xConnect Data Tools | Sitecore Documentation