We are moving toward the creation of a nation-wide interconnected electronic health information infrastructure whose primary goal is to provide better healthcare. At the same time, many regional health care organizations are only now adopting electronic health record systems. And, close by, vendors and other entities of all sorts are vying for influence over these advances.
Throughout this many-player process, it is imperative that healthcare organizations give value in return for the money they receive from the government and others and that businesses remain competitive. To understand the results of the enormous investments that are being made in the new electronic health information infrastructure, continuous measurement of the key variables is essential.
Every organization has a lot of information about its operation or has the ability to gather such information should it choose to. The problem is making good use of this information. Purely financial measures of performance are insufficient to ensure long-term improvement in healthcare. This is where Data Envelopment Analysis (DEA) can help. It's a mathematical technique that combines traditional performance ratios into a single efficiency score. That is, DEA, unlike many other quantitative methods, does not rely on a single criterion for measuring performance.
Furthermore, DEA tells you where an organizational unit can improve, based on the performance of its peers (see "A composite hospital - dual prices" below). Since it is a peer based comparison, the targets set for improvement are realistic and, therefore, more likely to be achieved. So, using DEA to compare the efficiency of a large urban teaching hospital with a small rural private practice -- apples with oranges -- would be an inappropriate use of this method.
DEA can be applied either spatially or temporally: i.e., at a single instance of time, it can be applied to compare the efficiency of distinct organizations (or systems) or, at different points of time, it can be applied to a single organization (or system). In the case of a system that’s still largely on the drawing boards, e.g., the nation-wide electronic health information infrastructure, initial comparisons need to be made among computer simulations of the different proposed solutions.
As the simple example that follows will illustrate, DEA can help you get a good overall picture of your organization's performance and where potential improvements might be made. However, as anyone reading this post already knows, the terrain in which DEA operates is very complex.
Linking Patient Records is currently being handled by different organizations in different ways. For example, in Massachusetts, MA-Share is using a federated architecture, with a shared record locator service (RLS). In California, the Mendocino HRE uses a brokered architecture with mirrored data at a central HRE. And in Indiana, IHIE is using a central data repository with standardized data.
The figure below serves to illustrate the three models. Furthermore, each of these groups employs different standards, software preferences, etc., which adds complexity when it comes to one of these groups interoperating with another.
Achieving an efficient interconnected electronic health information infrastructure requires the collaboration of individuals from many disciplines. My reference section below reflects this by citing the application of DEA to the optimization of computer networks in addition to the outputs of hospitals and physicians.
The nation-wide electronic health information system will be established by interconnecting a large number of preexisting regional systems, including many of the kind shown in the figure above, plus another large number of heretofore all-paper-record-keeping organizations. DEA methods may be applied to any and all of these, as long as you avoid comparing apples with oranges .
As a standalone system evolves, you should compare its sole performance before and after changes are made. Similarly, as individual systems join a national (and, perhaps, eventually, an international) grid, you need to compare their performances before and after they do so. The obvious question is which metrics should be tracked and included in the analyses. This is a very big question and far beyond the scope of this post. To illustrate how DEA works, however, I'll proceed with a simple example.
Consider a group of three hospitals. To simplify matters, assume that each hospital "converts" two inputs into three different outputs. The two inputs used by each hospital are
Input 1 = capital (measured by the number of hospital beds)
Input 2 = labor (measured in thousands of labor hours used during a month)
The outputs produced by each hospital are
Output 1 = hundreds of patient-days during months for patients under age 14
Output 2 = hundreds of patient-days during months for patients between 14 and 65
Output 3 = hundreds of patient-days during months for patients over 65
Illustrative inputs and outputs for these three hospitals are given in the table below.
The efficiency of hospital x = value of hospital x’s outputs / cost of hospital x’s inputs
From here, the math and theory get rather complicated.
Software solutions
Fortunately, there are computer programs -- some free, others not -- that can help you with all of this. Most of the general-purpose mathematical optimization software can be adapted to solve Data Envelopment Analysis problems. In addition, there are several DEA-specific programs that provide a variety of interesting facilities.
For a technical introduction to DEA, the following two videos
Data Envelopment Analysis 1
http://www.youtube.com/watch?v=xjECr9dveKk
Note: this video starts off referring to "the efficient frontier." For a review of this concept, you might take a look at my article “Capital Budgeting: Managing Efficient IT Project Portfolios,” which I cite in the bibliography at the bottom of this blog.
and
Data Envelopment Analysis 2
http://www.youtube.com/watch?v=eBx9iZPc4i8&feature=related
plus
a brief white paper that explains how DEA can help you get a good overall picture of your organization's performance and where potential improvements might be made http://www.banxia.com/frontier/pdf/FA_InUse.pdf
may be helpful.
Breakups or mergers
The Options For Clinical Data figure above shows three of the many different ways in which individual silos of clinical data can be distributed; i.e., broken up or merged. The decision on which architecture to adopt is based on considerations of security, privacy and many other factors.
With DEA, you can evaluate the performance of a silo and one or a combination of a few other silos. Such a comparison can indicate whether or not a breakup or merger of units needs to be considered. The scope of this kind of analysis may be limited by many factors, not the least of which is the degree of cooperation forthcoming from those who control the individual components of the overall system. Thus, sometimes breakups and mergers are not an option, even though DEA indicates there might be benefits from a breakup or merger.
If the output bundles produced individually by two hospitals (or other units) can be produced more efficiently together by a single hospital (or other unit), there is an efficiency argument in favor of merging these two units. Similarly, in some cases, breaking up an existing hospital (or other unit) into a number of smaller units would improve efficiency.
Attaining technical efficiency ensures that a unit produces the maximum output possible from a given input bundle or uses a minimum input quantity to produce a specified output level. Full economic efficiency lies in selecting the cost-minimizing input bundle when the output is exogenously determined (e.g., the number of patients treated in a unit) and in selecting the profit-maximizing input and output bundles when both are choice variables, as in the case of a business firm.
A composite hospital - dual prices
DEA analysis reports typically have a column entitled "Dual prices" that can give you great insight into Hospital 2's (or any organization's found inefficient by DEA) inefficiency.
After running a DEA analysis on the data in the table shown above, you would be inclined to create a composite hospital derived from the model that's made out of, say, 26.1 percent of the input level used by hospital number 1 and 66.1 percent of the input used by hospital number 3. If it were then found that the composite hospital uses "a" capital and "b" labor (with a or b lower than the corresponding amount required by Hospital number 2 and the other of these two variables no larger than its corresponding amount) to achieve the same level of outputs achieved by hospital 2, you could use these numbers as performance targets for hospital 2.
Bottom line: Dual prices can sometimes find a composite hospital that is superior to an inefficient hospital and where the origen of this inefficiency arises. Breakups and mergers aren't the only options.
In the news: The Johns Hopkins Health System Corporation has recently aquired Suburban Hospital in nearby Bethesda, MD, converting it into a Hopkins subsidiary. With its new association with an information technology-, medical- and management-savy institution like Johns Hopkins University, Suburban Hospital is now well positioned to participate in the development of the coming nation-wide interconnected electronic health information infrastructure.
References
This is a big topic and can only be handled by teams of experts who understand the complexities of the medical, financial, management and political issues involved! However, for anyone interested in more information on DEA (in addition to the two videos and the white paper cited above), consider these additional, albeit much more sophisticated, sources:
For examples using DEA in hospital and physician evaluation, see Chilingerian, J.A. (1994). Exploring why some physicians' hospital practices are more efficient: Taking DEA inside the hospital, in Charnes, A., Cooper, W.W., Lewin, A.Y., and Seiford, L.M. (Eds.), "Data Analysis: Theory, Methodology, and Applications, Boston: Kluwer Academic Publishers; Sherman, H.D. (1988). "Survive Organization Productivity." The Society of Management Accountants of Canada: Hamilton, Ontario.
and
Since Information Technology has a good deal to do with determining the overall efficiency of an interconnected, electronic health information infrastructure, some readers might also be interested in Medhi, D. and Ramasamy, K. (2007). "Network routing: algorithms, protocols, and architectures." San Fransisco: Morgan Kaufmann. In section 7.7, DEA is applied to the problem of finding the best topology for a computer network.
and
Comparing Performance with Data Envelopment Analysis
http://www.lindo.com/downloads/LINGO_text/Chapter14.pdf
and
A DEA tutorial
http://www.deazone.com/tutorial/index.htm
