Oops! The input is malformed!
Originally published 22 December 2005
The opportunity for insight is the basis of value for all information systems. Once an organization has insight, all things are possible. With insight come new opportunities—to make money, save money, improve goods and services, etc.
But where do we acquire insight? The answer to this question is limitless. Insight can come as easily as glancing out a window and seeing a rainbow. In contrast, it can be as difficult as investing in a dot com startup, only to discover there never was a business case in the first place.
Correlative analysis is one of the best ways to gain insight. Correlative analysis is the analysis of events occurring together. Suppose a study is about event A. While gathering information about event A, we notice that another event (which we’ll call event B) is frequently occurring. The fact that two events occur together can offer many opportunities for insight. Some typical questions are:
When using correlative analysis, there are many other opportunities for insight.
Correlative analysis has the most potential when applied to medical and healthcare organizations. When a medical or healthcare organization gathers all of its episodes of care and other encounters with patients, and then integrates and assimilates the documents associated with them, there can be very interesting and unexpected results.
Observing the natural correlations that have evolved in medicine can be very useful. No major patterns can be discerned by any particular doctor (short of alerting his/her intuition), because the doctor can only see one patient at a time. Of course, this is the patient that is immediately in front of him or her. But given many observations over a long time, medical and health patterns that would otherwise go unnoticed begin to emerge.
One challenge of correlative analysis is that a significant number of events must be reported for it to be meaningful. Examining 50 or even 500 incidences of care provide minimal value. However, 500,000 or 5,000,000 incidences of care are useful for spotting previously unseen correlations.
Dealing with data is also a common challenge with correlative analysis. Medical and healthcare data is notoriously unstructured. Medical and healthcare data is usually textual data as well. In addition, there is an extremely variable nomenclature for the same event or activity. This phenomenon was recently illustrated when a doctor told me there were at least 15 ways to describe a broken bone.
Another challenge with meaningful correlative analysis is that some terms are spelled differently. For meaningful correlative analysis, there must be a single spelling of the same term.
Still another challenge is that correlative analysis must be visual to be effective. When correlative analysis is not visual, patterns that are unclear or faint generally hide. Because of this, patterns can get lost in massive amounts of data, as well as more obvious patterns. But when visual techniques are used for correlative analysis, the chances of spotting faint and unclear patterns becomes greatly enhanced.
Fortunately, correlative analysis for the medical community is now possible. Leading medical research firms are beginning to use powerful new technology for very sophisticated correlative analysis.
It is now a reasonable expectation that today’s technology can spot important data patterns. Amazingly, many of these patterns would not have been found even a year ago.
Recent articles by Bill Inmon