Differentiating the ERP from the Data Project

Editor’s Note: Norman Katz, president of supply chain consultancy Katzscan Inc., writes a monthly column for Supply Chain Management Review. Katz’s column appears on the third Monday of each month.

I was called on, or rather, back to, help a client company with their ERP (enterprise resource planning) project, not their first one, nor the first one that I helped them with. The first system that they had was just too big and complex, the second one that they got involved with was too small, limited in its features and functions. This third time, thankfully, the system was just the right fit.

But my client kept delaying addressing the data project, even while looking for their new ERP system and going through the selection process. I did my best to caution them that these were really two different projects that would come together, but my advice was not heeded. Instead, and much to my surprise, by the time I was asked to get involved, the new ERP system was implemented with all of the old data as-is. The data was more of a mess to clean up in the new system than my client realized and was informed by their new software vendor. But the fact that I was correct was too late.

Data projects related to ERP systems typically don’t begin until the new ERP system has been selected, but I disagree with this philosophy. I think that once a company comes to the understanding that it is going to commit to a new ERP system, the data project should begin right then and there, running in parallel and complimentary to the ERP project.

Data projects related to ERP projects are typically described as “ETL” or “extract, transform, and load” projects. But for one, I think that a crucial step is being left out. And second, I think that the data project can start on its own while the ERP project is getting underway.

The ERP project is going to be focused on business process definition, software selection, transactions, system integration, and master and transaction data. The data project will be focused on extracting the data from the legacy system, transforming the data, converting the data into a format that the new ERP system can accept, and loading and testing the data load into the new ERP system.

In my opinion, the data project extract and transform phases can and should be done independently and as soon as possible.

Extracting data from the legacy system means understanding the database, table, and data field structures not just objectively, but subjectively. Legacy systems are famous for their “creative” use of data tables and data fields, so knowing and documenting the history of the unique quirks of what data is stored where and why is critically important if this is not already done, which it typically is not. You might have to review program code to extract this information.

Getting the data out may not be as easy a task as you may think. Data stored in “memo” fields can be a bit tricky to work with. Are special characters (e.g., tabs) getting in the way of this being successful? How will you manage links to documents? 

Chances are you will just need to extract the data to text (TXT) or comma-separated value (CSV) files. Again, are there special characters such as double quotes used in descriptions (e.g., to represent the inch measurement) that are affecting how the data is being formatted?

To me, transforming the data involves taking the data from the current state of the enterprise and applying business logic to present it in the proposed future state. If your customers are not distinguishable by business versus individuals but you want them to be in your new ERP system, now is the time to transform your data to do this. The “how” will depend upon the characteristics of your current-state data. Categorizing items by attributes, grouping suppliers by product or service, associating vendors based on their type, and re-aligning customers by different sales territory geographies are all considerations as to how to take the state of the data now and represent it for the future state of the company. Think of your reporting frustrations and what would make your life better if your data was just aligned differently.

An advantage of transforming the data prior to ERP selection is that this will inspire more vision as to where the company is heading in the future-state and will help to better define the required functionality of the ERP system the company is going to select.

As the ERP project is progressing along its own path and the data project along its path through the extract and transform phases, collaborating and communicating along the way, the two projects will really begin to meet when it comes to considering how the (transformed) data will be converted into the new ERP system. The conversion phase of the data project includes the field-to-field data mapping from the transformed data to the new ERP system.

Loading the data—repeatedly—ensures that the data passes through the new ERP system’s database, table, field, and business logic, that formatting issues did not get in the way, that special characters did not cause any problems, that the data mapping was correct, that the data looks the way it should when the user-facing forms are displayed. I like to load data multiple times, and right to the very end, to ensure that my clients don’t have to manually enter any data into the new ERP system to cover any gap period.

Don’t wait to begin your data project if you know that you are going to commit to an ERP or similar project. Starting early will help to ensure that your overall software project is completed on time, and that you start your new system with both clean data and data that is designed to take your company into its new future.

About the author:

Norman Katz is president of Katzscan Inc. a supply chain technology and operations consultancy that specializes in vendor compliance, ERP, EDI, and barcode applications. Norman is the author of “Detecting and Reducing Supply Chain Fraud” (Gower/Routledge, 2012), “Successful Supply Chain Vendor Compliance” (Gower/Routledge, 2016), and “Attack, Parry, Riposte: A Fencer’s Guide To Better Business Execution” (Austin Macauley, 2020). Norman is a U.S. national and international speaker and article writer, and a foil and saber fencer and fencing instructor.

SC
MR