Scenario #1. Suppose that:
- You have a two-sided process reconciling an internal trade file against a Data Prep input with all broker data consolidated
- You use a timing trigger for 11AM every day on your Data Prep process
- By 11AM today, only 3 out of your total of 5 broker files arrive, the snapshot is created and the reconciliation runs - meaning that you have unmatched trade items on the internal side of your reconciliation for the remaining 2 brokers
Challenge: How can you push the new snapshot (2 broker trade files) to the reconciliation to be processed with the outstanding unmatched trades from the other side?
Resolution: You have a three options:
- Wait for the next run (if you do not need to feed the late broker files into the reconciliation immediately and can wait until another internal trade file arrives the next day)
- Wait for the remaining 2 broker files to arrive, together with the next day’s batch of broker files
- At tomorrow’s cutoff time, a new snapshot will be created containing the two outstanding broker files from today, as well as the next day’s broker files
- Tomorrow, once the two-sided process received files for both sides, a new run will be triggered
- The unmatched items from the internal side, as well as tomorrow’s internal file will be reconciled against the consolidated broker data from today and tomorrow
- The carried over unmatched items from the internal side will be matched off against yesterday’s broker files that were included in the next day’s snapshot
- Manually submit the remaining 2 broker files (if you need to feed the late broker files into the reconciliation immediately and cannot wait until another internal trade file arrives the next day)
- Manually create a Data Prep snapshot with the 2 broker files
- Trigger a new reconciliation run by manually submitting an empty internal file on the other side OR use an ad-hoc submission window (which is set to trigger a run in the next, say, 5 minutes, and configured to run with submitted data; delete the submissions window afterwards)
- Similar to the above, but send through an empty internal file through SFTP to trigger that run (again, if you need to feed the late broker files into the reconciliation immediately and cannot wait until another internal trade file arrives the next day)
- Manually create a Data Prep snapshot with the 2 broker files
- Trigger a new reconciliation run by submitting an empty SFTP file on the internal side of your reconciliation
Note: All of the above are manual steps to rectify the problem when submissions go wrong. If you think that some of the brokers will always send their files late, then you should consider using the completion trigger instead, together with a submission window configured so that you will receive notification when some broker data is not received by a certain time.
Scenario #2. Suppose that:
- You have a Data Prep process that is automated (i.e. whenever a new snapshot is created, this will be routed to the two sided reconciliation where it will wait for the opposite-side file to arrive before a new run is triggered)
- You have uploaded data in a Pending status
- You make changes to your Data Prep process
- Duco prompts you whether you wish to discard pending data or create a new snapshot
- You choose to create a new Snapshot which essentially brings your submissions out of sync
Challenge: How do you 'invalidate' the snapshot that you have just created?
Solution: You have 3 options:
- Create a submission window for the next, say, 5 minutes, and configure it to fail the run if not all your data arrives within the specified time window (i.e. within the next 5 minutes). Providing no data is received for the opposite side of your reconciliation, this will use up the snapshot created earlier and fail the run, essentially invalidating your latest Data Prep snapshot.
- Go to your two sided reconciliation and manually run the process using your latest Data Prep snapshot and an empty file on the opposite side; then mark the run as failed.
- Submit an empty SFTP internal file. This will trigger a new run. Mark the run as failed.