Beware: Databases Store Low Hanging Fruit

Does transaction data hold all the answers?

The dangers of focusing only on transaction data are explained in a recent article by a colleague of mine - David Bean of Attensity. Since a small proportion of available information is accessible by traditional reporting and analysis tools this is considered a 'costly irony.' We spend money to access the easily accessed even if it represents only 15% of all data. What's needed is the ability to convert free form text into things databases can understand -- entities and relationships. The article gives a good example of decoding explanations in a claims processing system into events and attributes that can be reported.

To some extent this relates to the ability of database systems -- it was always easier to load transactions and other structured data; so that's what we did.