Rules & rule sets
As best practice we recommend that you define clear naming conventions for match rules in rules & rule sets, as well as clear descriptions. The names will help you to make configurations easier, and descriptions help you to troubleshoot and share with others why you configure it in a certain way.
Rule precedence
The conditional rules and rule sets work alongside default rules. It is important to note that the ‘Ignore’ option always takes precedence over any other rule at the same level.
For example, when one rule set designates a field as a mandatory key and another designates the same field to be ignored, the Ignore rule overrides the mandatory key.
Best Practice: Avoid using ‘Ignore’ as the default rule for any fields, since it can unintentionally overwrite other rules.
It is worth noting that a matching rule does not change the input values, it only affects the way in which input values are compared.
Please find details below for each ‘field comparison’ and ‘matching control’ rules available as below:
Multi-pass
Sometimes it's not possible to match all records in "one go". This may happen, for example, because there are multiple matching criteria (e.g. for different asset classes), not all fields are populated all the time, the data quality is low, or for other reasons. To overcome these problems, Duco allows you to set up multiple passes. Duco runs a multi-pass process as follows:
- Each pass has a set of rule sets and optional roll-up rules associated with it.
- Each pass is executed in turn according to your pre-specified order.
- At the end of each pass, all 100% matches become part of the final set of results.
- Any partial matches or unmatched items (or only the unmatched items) are used as the input for the following pass.
To setup multiple passes please refer to How to setup multi-pass.
Examples of where this is useful are:
- You want to find all the one-to-one matches first, then followed by a one-to-many matches.
- To match cross-currency swaps by matching notional 1 against Notional 1 and Notional 2 against notional 2. Then try cross-match notional 1 against Notional 2 and vice versa in a second pass.
Setting up a multi-pass process requires a good understanding of rules and rule sets. You may want to read also the pages rules using natural rule language (NRL) and conditional rule sets.
As best practice when there is a large dataset to reconcile we recommend to build multi-pass in two phases:
- Phase 1: High-confidence match rules (no retained partials)
Initial passes should be configured to catch as many perfect matches as possible and very few partial matches. Then any partial matches may not be retained as it would be very few and can be scoped again for following passes.
- Phase 2: More flexible match rules (with retained partials enabled)
The later passes can be flexible as most of the volume would have been matched by now and partial matches can be retained so we can identify which rule is responsible for marking these items as partial matches.
- Sequence passes from strictest rule to most flexible to improve accuracy and efficiency.
- When using multi-pass with conditional rule sets, ensure that conditions are defined based on mandatory fields. Always apply the condition check on the correct side of the field (internal vs. external). If a match field is used, the condition will be evaluated against the match field on both sides.
- Avoid creating multiple conditional rules or rule sets that overlap, as this can cause conflicts and unnecessary complexity.
- Use structured naming conventions for passes: e.g., MWDTP_ExactMatch_TradeID or MWDTP_FuzzyMatch_DateAmt.
- Add description to each pass to clarify intent.
Analyse match rates per pass to adjust their order and fine tune your configurations. Each match in the result tells you which pass was used to create it.