Pitfalls in large data management initiatives

In my last bog I talked about the journey many organizations go through before accepting they have a problem managing their information and data resources.  Once an organization reaches this stage, the question is then “what should we do about it?” 

Complex application systems, data bases, and end user computing solutions matched with competing organizational and departmental objectives are the obstacles that must be understood and surmounted.  These challenges can lead to numerous pitfalls if not approached correctly.  Some of these pitfalls are present in any complex program of work, and some are more unique to data management initiatives. 

I have broken the pitfalls into two parts: pitfalls in planning and pitfalls in implementation.  As we examine the pitfalls, we see they are often interrelated.

Pitfalls in Planning

Throwing money at the problem

This response often occurs when an organization’s senior management is faced with a significant regulatory issue.  They want to demonstrate to regulators their commitment to solving the problem; they want to believe that dollars spent will show they are taking the matter seriously.  This often leads to seeking out consulting firms based on name recognition to solve the problem for them. This may work out, but sometimes leads to further pitfalls in planning and implementation as discussed below.     

“In theory, practice should be easy.”  – Yogi Berra

Organizations that move beyond the “throw money at the problem” approach still may be at risk finding the right consulting partner. It makes sense to looks for expertise if the organization itself does not have the theoretical expertise or staffing available to do the initial planning and implementation.  However, that expertise must be tempered with practical experience. Theory is required to help sort through complex problems.  However, theory not tempered with experience can lead to problems with implementation as we will see in the second section on implementation, as well as to additional pitfalls in planning.

Boiling the Ocean

Sometimes the selected data management guru will pull out a comprehensive blueprint representing all the areas of discipline within data management.   Along with the model may be a maturity model and roadmap.  These templates are invaluable.  The danger is to use them as a one-size-fits-all solution to be pursued without effective prioritization.  Attempting to boil the ocean often leads to many pitfalls in implementation as we shall see.

Taking the safe road

 Sometimes the organization will realize that addressing all the aspects of data management at once (boiling the ocean) is not practical.   They resist the temption to throw money at the problem.  That leaves them with still having to decide where to start.  One temptation is to find out what their peers are doing and follow their lead.  Another approach is to research best practices from a business research company.  That may be fine if the underlying data and organizational structure is consistent with the industry norm.  However, it sometimes leads to starting on the wrong problem and never solving the underlying value proposition of the initiative.

Reaching for the shiny object

In this scenario the organization might look to a well-publicized facet of data management (the shiny object).  One might hear statements such as “Let’s build a new data warehouse,” or “let’s install a master data management solution,” or “let’s pursue total data quality management”.  Once again, all of these approaches can be appropriate, but are they the optimal first, second and third steps in the organization’s journey?  Additionally it is well documented that these shiny object solutions are often costly and risky.

Pitfalls in Implementation

Beware of a team alone in a room

It is critical that that the team implementing the plan doesn’t retreat into their shared workspace (online or on premise) to work the problem until they have come up with the solution.   Minimal contact with stakeholders after the initial planning stage can lead to indifference or resentment.  In addition, there is less opportunity to take corrective action if there is no frequent feedback loop. Lastly, a failure to report on milestones can lead to unpleasant surprises at the tail end of these types of initiatives.  


Lack of support from senior management

When the implementation team is recommending changes in an organization there may be groups that are too busy to contribute the necessary resources, groups that wish to maintain the status quo because they perceive it is better for their group or team, and groups that would like to see other groups fail for political reasons.  These common obstacles are significantly lessened by having the clear visible support of senior management.  Resistance is less likely when the resistor knows that there is risk involved in not helping to move things forward.

Not setting realistic short term (sprint) milestones

The implementation team is actively engaged and by all appearances moving forward.  However, unless the organization has established realistic and tangible project milestones, it may be difficult to really know the extent and rate of progress. A flurry of activity does not necessarily equate to results. Is the team moving around or are they moving forward?  Has the team climbed another set of staircases only to feel like they are right back where they started?

Not herding the “inside the organization“ cats

Most organizations have multiple areas that are involved in managing their data on a daily basis.  These include data security analysts, data warehouse professionals, MDM teams, software developers and integrators, DBA’s, enterprise architects, and records managers.  These folks should already have been engaged in the planning process in some fashion.  They also need to be brought into the implementation process.   Without coordination, redundant efforts and counter-productive tasks can easily occur. 

Tool selection – “To a man with a hammer, everything looks like a nail”.

This is the software side of the pitfall related to herding the cats.  When an organization has multiple data related departments carrying on their day to day jobs, some of these groups have more than likely already selected software to help automate processes and improve efficiency in their area of discipline.  These tools have been selected for the specific needs of that discipline.  At the same time, the software vendors for the selected tools are probably marketing the fact that they can meet all the data related needs of the organization.  Beware.  Without a look under the hood, the organization may select a sub-optimum tool to solve a problem that it was not designed for. Software selection should be thought through in considering both the desired end state and the initial set of priorities before a decision is made.

Sending in the droid armies

This pitfall is often directly related to many of the planning problems cited above.  A large initiative is sold to senior management by senior consultants.  To implement the solution will take an army of consultants.  This army will, on occasion, turn out to be recent hires or the consultants’ B team while the senior consultants are off selling to their next client.

I have now covered the various stages of data grief resulting in the realization that something needs to be done in a comprehensive way.  (See blog 1 of this series)  In this blog I discussed how the complexity involved in trying to address the problem can lead to a number of pitfalls. In the next blog I will suggest some ways to implement a comprehensive data management solution in an effective manner.