What you will need to begin

Before you begin you'll need to check you have the following:

• one of our supported web browsers installed
• a secure URL to access Duco
• a user name and password to log into Duco
• permission to create and set-up matching processes.

If you don't have any of the above, your line manager or an administrator within your organisation should be able to help you obtain what you need.

You will also need some test data which you can download at the bottom of this page.

You can unzip these files on the local drive in your computer and use them when instructed to do so. If you want to use your own data, you can upload files in CSV or Excel (xlsx) format.

Logging in

To log in, type or paste the secure URL for Duco that you have been given into your web browser.

This should take you straight to the log in screen.

Enter your user name (i.e. your email address) and password.

After successfully logging in, you will see your Dashboard on Duco.

The Dashboard provides a summary and statistical overview of your latest matching processes and results. As you have not yet set up any matching processes, the Dashboard will look a little empty when you first log in. We'll get back to the Dashboard once we have a set-up and run a matching process.

 

How to set up your first matching process

 Setting up a matching process is an iterative process: The steps are:

1. Import the test data
2. Match the relevant fields
3. Run the process
4. Examine the results
5. Update matching rules, reported fields, etc.
6. Repeat from step 3 until the results are as expected.

To set up a process you need to:

• Click on Processes on the navigation bar at the top of the screen. This will take you to the Processes screen. If this is the first process that you are setting up, the screen will be empty. Otherwise it will show all the processes that you have permission to view.

Click on the Create Process button, which will take you to the following dialog box:

• Leave type as "Generic two-sided process". In Duco you can also set up a one-sided process. But for this tutorial you'll need a two-sided process.
• Select the use case field based on a pre-defined list. Full list with description of each use case can be found here.
• Enter the process name, E.g., "London Options and Futures". Use a descriptive name for your processes, not "Proc 1" or "Rec A".
• Enter a unique short code for your process. This should be a short code such as "LOF01". We suggest that you choose a code no longer than 8 characters or so.
• Enter a Data label in the singular and plural form for the items you are matching. For example, "trade" and "trades" or "position" and "positions". This label is then applied in the UI to make use more intuitive (eg. on menus).
• Leave "Mask sensitive fields" un-ticked. This is an advanced feature covered in section Masked processes.

 

 

• Finally, click Create process.

Duco will show the Settings overview screen for the matching process that you have just created. To run your process, as a minimum, you will need to configure Data Inputs and Matched fields settings.

Set up the Data Inputs

From these pages you can access all the settings available in Duco. We suggest you only set up the minimum settings to run the process for the first time. This will give you results you can analyse quickly. You can then refine the settings and run the process again until you get the results you expect. This is the methodology we're going to follow with the sample data.

Let's start with Data inputs.

• Click on Data inputs. This will take you to this screen.

• Enter a name for the data set on the left hand side and click Create. For example, call it "Internal". You can change this name at any time, if you place your mouse pointer just to the right of the name, an Edit button appears.
• Choose the input file of test data called "internal.csv", that you have downloaded previously, and click on upload.
• Similarly for the data set on the right side, enter a name (for example "Exchange").
• Choose the input file of test data called "exchange.csv", that you have downloaded previously from, and click on upload.

You should see a screen that looks as follows:

 

 

Although not a required step, if you select Exit and navigate to the Settings menu, you can select Data Inputs from the menu to review Duco's assessment of the input files.

 

 

 

 

This is what just happened: When you uploaded the file, Duco automatically recognised the file format and the data fields.

For the "Internal" data, Duco recognised that the file uses a comma (,) as a field delimiter character, the double-quote (") as the quote character, and that the first line of the file is a header line.

Similarly for the "Exchange" data, Duco recognised that the delimiter is the pipe symbol (|).

For both the "Internal" and "Exchange" data, the screen lists also the fields that the files contains. For each field the screen shows:

• the field name
• a sample of the data for that field (under the field name)
• the data type for that field (on the right). Date, Text, Integer, Decimal, etc. Note that Duco detects the data types automatically.

The Data inputs page of the Help Centre describes in detail how to set-up data inputs.

Matching fields

The next step is to match the fields from the two data sets. We can do this from the Match fields screen.

To access this screen:

• Click on the Settings menu on the top-right of the screen and choose Match fields

You will see a screen that looks like this:

You now need to match fields from the "Internal" data set with fields from the "Exchange" data set. For example, "Trade Date" with "TrdDt". Just do as follows:

• Click on "Trade Date" at the left of the screen. The field will become shaded.
• Click on "TrdDt" at the right of the screen. The field will become shaded.
• Click on the Add field button.

The result is as follows:

The two fields are now matched. This is represented by the entry in the Match field column in the middle.

The two fields have also been removed from the left and right column, as they are already matched.

• Repeat the previous few steps for the remaining fields. You can match "Value Date" with "ValDt", "Symbol" with "Ticker", "Company" with "Cpy", etc.

This is the result:

Note that, in this case, the "Exchange" field from the internal data set remains unmatched because there is not a corresponding field in the Exchange data set.

Running a process

Your process is now set-up and can be run.

• Click on Exit settings on the top left of the screen. This will take you to the Process screen, which will list your newly created process.

• Click on the process name. This will take you to the Process overview screen.

 

 

 

 

• Click on Run Process → Re-use previous data

A run will take only a few seconds. At the end, you will see something like this:

To browse your results, click on the items in the navigation bar on the left side of the screen.

The navigation bar comprises of the following items:

• All Trades
• Unmatched
• Partially Matched
• Auto-matched
• Accepted
• Errors

As you can see, in this case, most trades are partially matched. This is to be expected since this is the first run for this process. We still need to fine tune the process settings. 

The Overview screen shows some general information about the run.

In particular: 

• The Run results information box shows some stats about the run results as a graph or as a table.
• The Breaking fields box shows which fields are breaking and how frequently.
• The Recent runs box shows the result bars for the last few runs.
• The Data submission box shows some stats abut the records submitted.
• If you hover with the mouse pointer on the calendar icon on the top right of the screen you'll see some additional details about this run.

The Breaking fields box is especially useful when setting up a process. In this case it's showing that the field "Product" breaks for most records. This usually indicates that we need to add a rule associated with this field. Or perhaps that this field is not mapped correctly. You can also see that 40 results items break on the field "Company", 6 on the field "Trade Date", etc.

Let's examine the results in more detail:

• Click on Partially matched from the navigation bar on the left of the screen. This will show a result screen like the following:

• Click on the ID for the first result. You should see something like the following:

This screen shows the detail for a match result.

For this result item, all fields match with the exception of the field "Product", which is causing the break. The match score in this case is 90% because 9 out of 10 fields match.

• Go back to the match result page (click on your browser's back button)
• Have a look at other match results.

If you examine a few results, you'll notice that in most of them the "Product" field is causing the break. This is not surprising as the Breaking fields box we saw earlier already indicated this. Also the field "Company" is responsible for many breaks. We'll deal with this later. 

Looking at the result, it should also become apparent that the "Product" field in the "Internal" data set uses the values "O" and "F" to indicate whether a trade is an option or a future. Whereas the corresponding field in the "Exchange" data set uses the values "Option" and "Future". Duco reports a break because "O" and "Option" are different values and so are "F" and "Future".

To overcome this problem we need to tell Duco the value "O" should match with "Option" and that "F" should match with "Future" in the "Product" field. We can do this by adding a matching rule to our process configuration. In this case, we need to create a rule using a Text Equivalence rule and a reference table.

Using a text equivalence rule

We want to add a text equivalence rule and define a reference table. To do this, first, we need to go back to the process configuration.

• Click on the Settings drop-down menu and choose Rules & rule sets

You should see the following screen:

• Click on Add rule in the "Product" field
• Select Text equivalence from the list of field comparison types on the left.
• Enter a name for your reference table (E.g. "Futures and Options")
• Click the Save rule button

 

 

 

 

The Rules & Rule sets screen now indicates that there is a rule associated with the "Product" field.

Now we need to add entries to the reference table that we have created.

• Click on notifications bell icon at the top of the screen, this will show you a table has been created and you can click on the name of the table to take you straight to the reference data table.

 

 

 

• Alternatively, you can click on the three-lines-menu at the top right of the screen: Reference Data → Futures and Options

• Click on Insert Row
• Add entries for for "O" → "Option" and "F" → "Future"

The text equivalence rule and the reference table are now set up, Duco is primed to match "O" with "Option" and "F" with "Future" and we can try to run the process again.

• Click on Processes on the navigation bar at the top of the screen. This will take you to the Processes screen
• Click on the "London Options and Futures" process
• Click on Run process → Re-use previous data

 

Note that the result bar for the second run is mostly green, indicating more matches.

We can also see that some partial matches remain, and the statistics box shows that the "Company" field is causing most of the breaks.

• Click on Auto-matched from the side bar on the left of the screen to see the matched trades.

• Click on the match result IDs to examine the data in more detail.

As these are the matched trades, the match score is 100% and there are no breaks.

Now, let's look at the remaining partial matches:

• Go back to the Overview screen
• Click on Partially matched

• Click on the match result IDs to examine the data in more detail.

• Click on you browser's back button and look at other match results to try to understand what's causing the breaks.

As expected, the "Company" field is causing the break. You will notice that, in the the "Internal" data set, company names sometimes contain a trailing comma, whereas the "Exchange" data does not. In the absence of a specific rule, Duco reports a break each time it finds two values that are not identical. For example, "Mattel," and "Mattel".

To overcome this problem, we can change the process settings again and add one more rule.

The Rules & Rule sets page of the Help Centre describes in detail all the possible matching rules.

Adding an ignore characters rule

• Click on the Settings drop-down menu and choose Rules & Rule sets. This will take you to the Rules & Rule sets screen.
• Click on Add Rule

• Select Ignore characters from the list of field comparisons on the left.
• Add comma (,) from to the characters to strip.

• Click Save rule

 

The Rules & Rule sets screen now indicates that there is a rule associated with the "Company" field.

Now we can try to run the process again.

• Click on Overview at the top left of the screen
• Click on Run Process → Re-use previous data

The result should be the following:

As you have already done previously, you can inspect the results.

You will notice that differences in the field "Trade Date" is causing breaks. 

• Also have a look at the unmatched trades - these have failed to fulfil Duco's default requirement that items must be at least 50% similar to qualify as partial matches (this threshold can be manually adjusted under Results Control).

Running a process with new data

Once a process is set up, you normally need to run it using new data. You can do this easily from the Process overview screen.

• Click on Run process → Submit new data

• Click on Choose File to select the files that you want to upload.

Exceptions workflow

The Exceptions workflow offers users a suite of functionality geared specifically for tackling exceptions found within a reconciliation. This includes a modified interface with unique buckets that track the status of exceptions within the review lifecycle, as well as workflow tools to assist the review process itself.

This feature is not enabled on process creation, but is typically switched on after the initial build phase has been completed.

Configuring Workflow

When creating a process, the Settings Overview shows an option to select Exceptions workflow. 

To enable the Exceptions workflow you must first select a Default group. This will be the user group with responsibility for advancing exceptions through the review lifecycle, and performing actions on any new exceptions, such as applying comments or labels. Other users with access to the process, but not in the default group, will have rights to assign exceptions, apply labels and make comments. They will not, however, be able to progress exceptions through the review lifecycle.

Only one Default group can be selected per process, although a user can assign exceptions to additional groups that have been permissioned to the process. The users in groups with permission to a process should always be granted permission individually to the process. This additional step is necessary otherwise they will not be able to access the exceptions.  

The coloured buckets now identify the workflow status of your items. The three buckets are: Unreviewed, Under review and Pending. 

Any items found to be either partially matched or unmatched are regarded as exceptions that will require attention, and as such will show in the Unreviewed bucket on the left of the screen.

Clicking on the Unreviewed bucket will take you to the individual results. Here you can select on individual or multiple items, and then from the Actions menu at the top of the screen select actions to perform.

Items can be progressed from an Unreviewed status through to being Under review and then finally Pending, in more exceptional circumstances where the issue cannot be resolved in a downstream system Closed. It is also possible to capture details of the break resolution and triage using labels and comments, as well as the actual owner of the exception.

Sharing results

Duco generates static URLs for processes, runs, match results, etc. For example:

• The URL for the process LF2013 might look like the following: http://ducocube.mybank.com/processes/LF2013
• The URL for the third run of that process might be: http://ducocube.mybank.com/processes/LF2013/runs/3 
• The URL for a match result within the third run of process LF2013 might be: http://ducocube.mybank.com/processes/LF2013/runs/3/matches/1804233

You can send these links via email to your co-workers to share your results. Your co-workers will be able to just click a link to see the result, assuming they have permission to view the process.

Most people find this a very handy feature. For example, it avoids having to email screenshots, or spending time writing lengthy descriptions of the match results in an email.

You can also bookmark these URLs in your browser like any other web page.

You will notice that a process short code is part of the URL. This is the reason why the short code can not be changed and that two processes can not have the same short code: The URL needs to be unique.

There is more!

To know more about what you can do with Duco, read through the Help Centre. It provides the finer details, including explanations around setting up a process, matching rules, automated data submissions, etc. If you are an administrator, start by reading the A quick start guide to adding your first Duco users (for Administrators) and then read through the pages in the Administration section of the Help Centre. If you are a Builder and want to try your hand at a more advanced process, why not try our 102 build.