When creating a recon process
The following points are important to consider when setting up a new reconciliation process, as they would influence what you need and how they should be configured:
- All the data sources and their file formats - e.g., one or multiple data sources from one side; CSV, XSLX or ISO 20022 XML formats.
- Clarify the type and granularity of data being reconciled - e.g., are they individual transactions, group of transactions, positions, cash movements, client data, etc.
- Data that reconcilers and operation teams need to see to investigate and resolve issues.
Tip: always record the purpose and configuration rationale for each process in the description field for future reference and audit readiness.
Use Data Prep to feed into reconciliation
Advantages of using Data Prep include:
- Data transformation & enrichment: Apply entity-specific calculations, mappings, and enrichments upfront so reconciliation consumes meaningful, business-ready data.
- Filtering: Exclude unwanted or irrelevant records before they reach the recon process, improving both accuracy and performance.
- Segregation of responsibilities: By separating data preparation (transformation, mapping, filtering, enrichment) from reconciliation (matching rules, exception handling), teams can maintain clearer ownership. This reduces operational complexity, simplifies audits, and ensures compliance can trace logic more easily.
- Future proofing: Input sources and formats often change. With Data Prep, you can onboard new data sources, manage switchovers, and deprecate old feeds seamlessly - without rebuilding reconciliation processes.
- Consolidating multiple data sources: Consolidate and standardize multiple data sources into a single input dataset before reconciliation to ensure consistency and reduce processing complexity.
- Deterministic data loading : Apply snapshot triggers to ensure that reconciliations always run against a consistent dataset.
What happens if you don’t use Data Prep?
- Increased complexity: Matching and transformation logic become intertwined within the reconciliation process, making it harder to test, maintain, explain, or audit.
- Reduced flexibility: Any changes in input format or data source requires modifying the core reconciliation setup, introducing unnecessary risk and deployment dependencies.
- Operational overhead: Testing and maintenance efforts grow significantly over time as the process becomes tightly coupled to input data structures.
When would you start without Data Prep
In some client scenarios-particularly where the data is clean and ready to be reconciled right away-it may be faster to set up a reconciliation process without Data Prep.
As data volume, source diversity, or transformation logic grows, it is best practice to transition to Data Prep (see here for how to change the data input of an existing process to a Data Prep process). Doing so provides scalability and long-term maintainability without disrupting existing reconciliation logic.
Select the appropriate process type based on your objective
Typically generic 2-sided reconciliation processes are used when you’re reconciling two data sets, which covers the majority of use cases.
Even when your data comes in a single file, it’s useful to use Data Prep to logically split it into 2 stable, mutually exclusive data sets (e.g. Buy and Sell) and feed them into a 2-sided process.
Generic 1-sided recon processes are useful in some scenarios.
When to use single-sided process:
- When your data comes in a single file and there are no inherent matching requirements.
- Ideal for simpler reconciliation use cases where data is self-contained and can be verified in isolation.
For threshold checks or workflow-driven controls on a single data set. Use calculated fields to identify outliers and use simple workflow rules to automatically close exceptions that meet predefined criteria.
When not to use single-side process:
While your data comes in a single file, the data that you want to reconcile within it can be split into 2 mutually exclusive sets with clear rules e.g. buy vs sell. Then the best practice is to feed the file into 2 Data Prep processes, each with opposite filtering rules to split it into 2 data inputs that feed to a 2-sided recon.
- Advantages: your 2-sided recon process will have clean data on both sides, much easier to configure and use day to day, with highly predictable behaviours.
- What if you don’t do this: wrong exception ageing and tracking may impact your work. In a single-sided recon, matches of the same 2 rows of data could be formed in different ways in different runs (i.e. A matches B partially on run X, and B matches A partially on run Y), so in a replacement rec, these could mess up the tracking, force closed exceptions and exception ageing.
- If you have complex matching rules, and / or complex and multiple workflow rules.
- When you want to connect one process to another for explaining the breaks.
No unique identifier strategy
- When no single unique ID exists in a rec we recommend to combine multiple fields (e.g., counterparty + direction + trade date + amount).
- If constructing such a composite key is not feasible, in that case, add the row number as a dedicated column in the file before loading the file into Duco.