Successful ERP Implementation depends on The Critical Step of Data Cleansing

So your company is planning on completing an ERP implementation within the next few months and you think your data is clean? Okay, you are not alone.  Many companies think the same way.  But, before you flip the switch on your new ERP software package, ask yourself these questions:

·         Have you ever entered an entry for a price quote and forgotten to delete it?

·         Do you have duplicate customer or supplier accounts?

·         Do you have any blank description fields for products or generic items?

·         Do you have old product codes that are no longer in use?

·         Or do you have any critical data in Excel that is NOT in your legacy system?

If you have answered yes to any of the above, your data probably needs to be cleansed prior to go-live with your new ERP software package.  Or in other words, you need to learn the art of data cleansing.

Most of ERP software vendors purposely do not take responsibility for cleansing data before or during an ERP implementation.  It is extremely costly for them to do so, therefore they opt to leaving the data cleansing to the actual ERP users.  While the ERP software vendors do not put this high on their priority list, the ERP project team should make this an integral part of the ERP implementation.

Why do we consider data cleansing an art form? The art lies in making a smooth cutover at go-live.  The purpose is to bring enough clean, accurate, up-to-date and relevant records to function with as little of the “junk” as possible. This ensures minimal downtime at conversion, as well as reduced problems after go-live.

When It Comes to ERP Implementations and Data Migration, Garbage In Means Garbage Out

Clients often ask why we do not bring over everything available in regards to their data or how we can realistically identify the junk within their legacy system. Let’s put it this way, if you are moving to a new house, would you bring over all old furniture, newspapers, and trash to your new house? The more useless items you bring over, the harder time you will have finding the things you really need. Similarly, the more data you bring over, the more you need to sort through at conversion, the more likely to impact go-live. On the flip side, if you bring over too few records, you can lose the record totally!

  

What is Considered Junk within an ERP Implementation?

What is junk? Well, how about an item that has not been sold in the last, say five years? Or would you consider three years as an appropriate limit? How about an item that you no longer manufacture? How about a customer who has gone out of business or not bought anything from you for the last three years?  All closed sales and purchase orders? What data needs to be kept can be quite arbitrary and dependent on the industry and purpose the data provides. Forecasting needs to be considered as well. There are no hard and fast rules, but management need to firm agreement on what does or does not need to be kept, as it can span several departments.

Regulations Make an Impact on ERP Implementations and Data Migration

Different industries have different regulatory requirements and need to be considered.  Some regulations restrict the ability to change certain kinds of data records. Some govern the length of time records need to be kept, such as for auditing purposes. However, most of these might be addressed using paper records, thereby lowering the required data needed for migration and cleaning.

Who Does the Data Cleaning?

There is a general misconception that the IT team or resources (DBA) can handle all of the cleaning. While there is data cleansing software, these packages can only help identify potential areas and records to be cleaned. Someone still needs to know, for example, out of two different address records for a company, which one is the correct one.  Or someone knowledgeable needs to be able to verify that the closing balance is in fact correct. It is this data integrity and accuracy that we are seeking. In most cases, the owner to clean these records is not the IT department, but the actual functional departments or users. However, they both need to work together to ensure success.

In-System vs. Excel

There are pros and cons for both methods of data cleaning and we often use both. In-System cleaning is to correct the data in the legacy system database. This method is good especially if you intend to keep a final snapshot of the legacy system for record purposes. Downloading data and changing it in Excel is often used when there are fields that are not in the legacy system. This method is often used very close to go-live date as it represents duplicate data and if there are changes in the system, it will not be captured after download. Often it is a succession of cleaning efforts that is repeated.

Generating Clean Data

After identifying where the problem areas lie, there may be some time to actual go-live, thus data keeps being added into the legacy system. If you are the person generating the dirty data, believe me, you will be the one cleaning it up! There should be training or procedures put in place to ensure minimal cleansing is needed moving forward.