Is your enterprise suffering from too much data?

Editor’s Note: Norman Katz, president of supply chain consultancy Katzscan Inc., writes a monthly column for Supply Chain Management Review. Katz’s column appears on the third Monday of each month.

Not only is there too much information running through my brain (thanks to the rock band The Police for that line, taken from their song “Too Much Information” from their 1981 album Ghost In The Machine—I mostly blame social media and the internet for this these days), but apparently there’s also too much information running through most companies too. 

According to a CIO Dive article titled Enterprise data overload triggers cost overruns, spending spikes from September 25, 2023, data overload causes cost overruns and increases in spending.  After all, it does cost to collect and store all of this data.

In a survey of 500 total professionals by Wakefield Research, excessive spending due to excessive data collection was something that nearly everyone on the survey acknowledged.  Similarly, 4 out of 5 survey respondents admitted to limiting log ingestions, and approximately one-third admitted to randomly discarding data.

All of this is at a time when IOT—internet of things—initiatives are being pressed forward which only involves the collection of more data as more data collection points are added.  Advances in data collection methodology, e.g., shifting to radio frequency identification (RFID) instead of barcode label scanning for automatic identification, realistically means that more data is collected via the RFID tag than the traditional barcode scan. Even the move to 2D (two-dimensional) barcodes from 1D (one-dimensional) barcodes means that more data will be acquired per scan.

More data collection points collecting more data per collection point. Whew. What is practical to collect and how often to give you the necessary insights into your supply chain’s activities?   

To balance the costs of data collection and retention with the value of the insights that the information can provide, the right retention policy needs to be formulated for all of the different data collected. How long does the data need to be retained? (Are there regulatory or tax requirements for data retention?) Does the data need to be retained in its original state, or can it be aggregated and only the summary data retained with the detailed data eventually deleted? Is there a single source of the data, or is the data being replicated unnecessarily, e.g., for analysis purposes, across multiple software systems?

Is there so very much data in your organization that your people just don’t know what to do with it all?  If there is, then you’ve likely reached the point of diminishing returns to a factor: the more data you have, the less value it has.

Closing transactional gaps in supply chains, increasing operational effectiveness, and enhancing your competitiveness with data collection is helpful.  But doing so without a good data policy is costly, and probably confusing too.

SC
MR