Change management control using staging copies

When your Data Prep process is in production and there are changes you want to make to the configuration, you can go through a change management process with 4-eye review controls by using staging copies. Just like how you’d do the same for reconciliation processes (see here).

Overview:

These are the steps to follow:

1. Create a new staging copy of your process.
2. In the copy, make changes and test them.
3. When ready, raise a change request.
4. A reviewer can then review the changes and approve it. Once approved, the new configuration will be applied to your process and used on data submitted after that point.

1. Create a new staging copy

Start creation of the staging copy by clicking on the Settings button. At the bottom of the list you will see Staging. 

If you are already working on the Settings, this option will be at the bottom of the left hand side panel.

Click on Create staging copy. Duco will capture the current process configuration and create a new child process using that configuration, with the Short Code:  <process code>-COPY*

You can have several independent copies of the same process at the same time.

 

 

Click on the Short Code for the staging copy, which can be found in the column named Copy, to navigate to your staging copy where you can begin making changes.

A banner in the top-right corner will indicate which copy you are currently working on. Use this dropdown to navigate between copies.

 

2. Make your changes and test

Update the configurations according to your needs. Submit data manually to test those settings, create snapshots and fine tune the settings.

Snapshots created in a Staging copy will not be sent to any downstream processes. They also won't be visible from downstream processes.

The staging copy list shows you how many changes you’ve made compared to the parent process. The percentage gives you an indication of the size of changes in this copy compared to the parent process.

 

Check out all the changes you’ve made in the Changes tab by clicking on View changes.

 

Note: the changes will be regenerated whenever there are changes made to the staging copy or the parent process.

 

3. Raise a change request

Once your configuration is finalised, raise a change request. Another user with the Config admin role for the process can then review the changes and decide whether to approve or reject the request.

 

 

 

Note: no further changes can be made to the staging copy when there’s a pending change request.

If the reviewer rejects the request, you can still continue to make changes to the staging copy and raise another Change request when ready. 

Once the reviewer approves your request, the configuration from the staging copy will be applied to the parent process.

The list of change requests can be seen in the Request history tab on the Staging page.

 

Environment segregation

If you are using our Development environments feature, you can then strengthen the change management process further by segregating environments for building, testing and running Data Prep processes in production. 

This article explains how it works in detail with reconciliation processes, which is the same for Data Prep processes.

 

1. Raise an export request

Go to General in the Settings menu and select Raise export request for the Data Prep process you would like to export to another environment.

 

2. Review and approve import request

Go to Administration > Configuration import to see the list of requests.

 

Note: you cannot yet see a PDF version of the configuration for requests of Data Prep processes.

 

If your NRL rules (e.g. Map and Transform, Filter rules) use any reference data tables, you can choose the corresponding table in the target environment in the mapping step.

 

 

If you are happy with the request, apply it to a new Data Prep process or an existing one. Either case it will create a new staging copy, which you can then further update the configuration if needed, and go through the change request steps to push it to production.

 

Best practice for managing changes to a Data Prep process and any reconciliation processes that uses it as an input

1. Export your data prep process to the target environment, and apply the import request. It will create a new staging copy for the data prep process.
2. When ready, raise a change request to roll it out to production.
3. Export the reconciliation process to the same target environment and apply the import request. Choose the data prep process that is now in the target environment in the mapping step.
4. Test the whole flow in the staging copy.