February 20, 2001, 1:55 PM — For years, the Montana Department of Corrections was a prisoner of data quality problems. Aging IT systems perpetrated countless data entry offenses in reports that the prison system was required to submit to state and federal authorities. And while the department's IS group put in hours of manual labor to try to maintain some level of reporting integrity, overall confidence in the quality of data was nonexistent and morale in the IS group was low. The situation came to a head four years ago when the department nearly lost a coveted $1 million federal grant. The culprit: information systems that, lacking business rules and a data dictionary, failed to accurately forecast how many of a particular type of offender would be incarcerated. "We had an egregious data quality problem. Not to the point where we were losing offenders -- but we weren't able to accurately portray how many we thought we'd have over the next two to five years," says Dan Chelini, bureau chief for information services at the Helena, Mont.-based department.
With the go-ahead from the state prison's board of directors, Chelini's department mounted an aggressive campaign, from late 1997 to mid-1999, to turn around data quality as part of an overhaul of the prison system. The first step was to bring in a team from Information Impact International, a consultancy specializing in data quality, to evaluate organizational processes, acquaint the department with the concept of data stewardship and set up a methodology for data entry. Although some employees were leery at first of the new demands, they bought into the new standards once trained in basic data modeling and data cleansing techniques. A data validity officer was also appointed to rally support for the program and enforce the new rules.
The program officially launched August 2000, and the department claims to see some real results. Instead of a handful of programmers holding all of the responsibility for prisoners' information, 30 data stewards from all walks of prison life -- from probation officers and attorneys to the guy who showers prisoners when they first enter a facility -- now function as data quality gatekeepers. They are accountable for accurately entering information on prisoners, such as names, last known addresses and identifying scars and disfigurements. The Montana Department of Corrections' data quality problem has been detained. "For the first time in years, we're meeting deliverables" such as reports to federal overseers, says Data Validity Officer Lou Walters. "People are involved and excited about pushing data quality."
Although companies deal with customers, not prisoners, the increasing need for accurate data is driving many organizations -- in finance, health care, retail and other segments -- to launch formal initiatives to bolster the quality of customer information in core business systems. Until recently most organizations haven't felt a lot of urgency or enthusiasm about cleaning up dirty data; inaccurate and multiple listings of customer information were seen as trivial problems and a tolerable price of doing business. But the current trend in many industries toward data warehousing and data mining has increased the value of good data and the costs of cleaning up databases. That task is anything but trivial, and the costs, which include the direct costs of hiring people and consultants and the indirect costs of missed sales opportunities, are significant.
"Our studies in cost analysis show that between 15 percent to greater than 20 percent of a companies' operating revenue is spent doing things to get around or fix data quality issues," says Larry English, principal of Information Impact in Brentwood, Tenn. Some organizations, like the Montana Department of Corrections, are creating full-time positions around data quality and instituting homegrown methodologies to ensure that information stays consistent and is usable across different types of applications. Other companies are purchasing data cleansing services and customer identification and standardization software from companies such as Vality Technology in Boston and Innovative Systems in Pittsburgh to clean up their act.
STAGE ONE: DENIAL
