Data Envelopment Analysis and Electronic Health Records

Evaluating the efficiency of an interconnected electronic health information infrastructure

We are moving toward the creation of a nation-wide interconnected electronic health information infrastructure whose primary goal is to provide better healthcare. At the same time, many regional health care organizations are only now adopting electronic health record systems. And, close by, vendors and other entities of all sorts are vying for influence over these advances.

Throughout this many-player process, it is imperative that healthcare organizations give value in return for the money they receive from the government and others and that businesses remain competitive. To understand the results of the enormous investments that are being made in the new electronic health information infrastructure, continuous measurement of the key variables is essential.

Every organization has a lot of information about its operation or has the ability to gather such information should it choose to. The problem is making good use of this information. Purely financial measures of performance are insufficient to ensure long-term improvement in healthcare. This is where Data Envelopment Analysis (DEA) can help. It's a mathematical technique that combines traditional performance ratios into a single efficiency score. That is, DEA, unlike many other quantitative methods, does not rely on a single criterion for measuring performance.

Furthermore, DEA tells you where an organizational unit can improve, based on the performance of its peers (see "A composite hospital - dual prices" below). Since it is a peer based comparison, the targets set for improvement are realistic and, therefore, more likely to be achieved. So, using DEA to compare the efficiency of a large urban teaching hospital with a small rural private practice -- apples with oranges -- would be an inappropriate use of this method.

DEA can be applied either spatially or temporally: i.e., at a single instance of time, it can be applied to compare the efficiency of distinct organizations (or systems) or, at different points of time, it can be applied to a single organization (or system). In the case of a system that’s still largely on the drawing boards, e.g., the nation-wide electronic health information infrastructure, initial comparisons need to be made among computer simulations of the different proposed solutions.

As the simple example that follows will illustrate, DEA can help you get a good overall picture of your organization's performance and where potential improvements might be made. However, as anyone reading this post already knows, the terrain in which DEA operates is very complex.

Linking Patient Records is currently being handled by different organizations in different ways. For example, in Massachusetts, MA-Share is using a federated architecture, with a shared record locator service (RLS). In California, the Mendocino HRE uses a brokered architecture with mirrored data at a central HRE. And in Indiana, IHIE is using a central data repository with standardized data.

The figure below serves to illustrate the three models. Furthermore, each of these groups employs different standards, software preferences, etc., which adds complexity when it comes to one of these groups interoperating with another.



Achieving an efficient interconnected electronic health information infrastructure requires the collaboration of individuals from many disciplines. My reference section below reflects this by citing the application of DEA to the optimization of computer networks in addition to the outputs of hospitals and physicians.

The nation-wide electronic health information system will be established by interconnecting a large number of preexisting regional systems, including many of the kind shown in the figure above, plus another large number of heretofore all-paper-record-keeping organizations. DEA methods may be applied to any and all of these, as long as you avoid comparing apples with oranges .

As a standalone system evolves, you should compare its sole performance before and after changes are made. Similarly, as individual systems join a national (and, perhaps, eventually, an international) grid, you need to compare their performances before and after they do so. The obvious question is which metrics should be tracked and included in the analyses. This is a very big question and far beyond the scope of this post. To illustrate how DEA works, however, I'll proceed with a simple example.

Consider a group of three hospitals. To simplify matters, assume that each hospital "converts" two inputs into three different outputs. The two inputs used by each hospital are

Input 1 = capital (measured by the number of hospital beds)
Input 2 = labor (measured in thousands of labor hours used during a month)

The outputs produced by each hospital are

Output 1 = hundreds of patient-days during months for patients under age 14
Output 2 = hundreds of patient-days during months for patients between 14 and 65
Output 3 = hundreds of patient-days during months for patients over 65

Illustrative inputs and outputs for these three hospitals are given in the table below.



The efficiency of hospital x = value of hospital x’s outputs / cost of hospital x’s inputs

From here, the math and theory get rather complicated.

Software solutions

Fortunately, there are computer programs -- some free, others not -- that can help you with all of this. Most of the general-purpose mathematical optimization software can be adapted to solve Data Envelopment Analysis problems. In addition, there are several DEA-specific programs that provide a variety of interesting facilities.

For a technical introduction to DEA, the following two videos

Data Envelopment Analysis 1
http://www.youtube.com/watch?v=xjECr9dveKk

Note: this video starts off referring to "the efficient frontier." For a review of this concept, you might take a look at my article “Capital Budgeting: Managing Efficient IT Project Portfolios,” which I cite in the bibliography at the bottom of this blog.

and

Data Envelopment Analysis 2
http://www.youtube.com/watch?v=eBx9iZPc4i8&feature=related

plus

a brief white paper that explains how DEA can help you get a good overall picture of your organization's performance and where potential improvements might be made http://www.banxia.com/frontier/pdf/FA_InUse.pdf

may be helpful.

Breakups or mergers

The Options For Clinical Data figure above shows three of the many different ways in which individual silos of clinical data can be distributed; i.e., broken up or merged. The decision on which architecture to adopt is based on considerations of security, privacy and many other factors.

With DEA, you can evaluate the performance of a silo and one or a combination of a few other silos. Such a comparison can indicate whether or not a breakup or merger of units needs to be considered. The scope of this kind of analysis may be limited by many factors, not the least of which is the degree of cooperation forthcoming from those who control the individual components of the overall system. Thus, sometimes breakups and mergers are not an option, even though DEA indicates there might be benefits from a breakup or merger.

If the output bundles produced individually by two hospitals (or other units) can be produced more efficiently together by a single hospital (or other unit), there is an efficiency argument in favor of merging these two units. Similarly, in some cases, breaking up an existing hospital (or other unit) into a number of smaller units would improve efficiency.

Attaining technical efficiency ensures that a unit produces the maximum output possible from a given input bundle or uses a minimum input quantity to produce a specified output level. Full economic efficiency lies in selecting the cost-minimizing input bundle when the output is exogenously determined (e.g., the number of patients treated in a unit) and in selecting the profit-maximizing input and output bundles when both are choice variables, as in the case of a business firm.

A composite hospital - dual prices

DEA analysis reports typically have a column entitled "Dual prices" that can give you great insight into Hospital 2's (or any organization's found inefficient by DEA) inefficiency.

After running a DEA analysis on the data in the table shown above, you would be inclined to create a composite hospital derived from the model that's made out of, say, 26.1 percent of the input level used by hospital number 1 and 66.1 percent of the input used by hospital number 3. If it were then found that the composite hospital uses "a" capital and "b" labor (with a or b lower than the corresponding amount required by Hospital number 2 and the other of these two variables no larger than its corresponding amount) to achieve the same level of outputs achieved by hospital 2, you could use these numbers as performance targets for hospital 2.

Bottom line: Dual prices can sometimes find a composite hospital that is superior to an inefficient hospital and where the origen of this inefficiency arises. Breakups and mergers aren't the only options.

In the news: The Johns Hopkins Health System Corporation has recently aquired Suburban Hospital in nearby Bethesda, MD, converting it into a Hopkins subsidiary. With its new association with an information technology-, medical- and management-savy institution like Johns Hopkins University, Suburban Hospital is now well positioned to participate in the development of the coming nation-wide interconnected electronic health information infrastructure.

References

This is a big topic and can only be handled by teams of experts who understand the complexities of the medical, financial, management and political issues involved! However, for anyone interested in more information on DEA (in addition to the two videos and the white paper cited above), consider these additional, albeit much more sophisticated, sources:

For examples using DEA in hospital and physician evaluation, see Chilingerian, J.A. (1994). Exploring why some physicians' hospital practices are more efficient: Taking DEA inside the hospital, in Charnes, A., Cooper, W.W., Lewin, A.Y., and Seiford, L.M. (Eds.), "Data Analysis: Theory, Methodology, and Applications, Boston: Kluwer Academic Publishers; Sherman, H.D. (1988). "Survive Organization Productivity." The Society of Management Accountants of Canada: Hamilton, Ontario.

and

Since Information Technology has a good deal to do with determining the overall efficiency of an interconnected, electronic health information infrastructure, some readers might also be interested in Medhi, D. and Ramasamy, K. (2007). "Network routing: algorithms, protocols, and architectures." San Fransisco: Morgan Kaufmann. In section 7.7, DEA is applied to the problem of finding the best topology for a computer network.

and

Comparing Performance with Data Envelopment Analysis
http://www.lindo.com/downloads/LINGO_text/Chapter14.pdf

and

A DEA tutorial
http://www.deazone.com/tutorial/index.htm

Electronic Health Records (EHR)

The health information technology provisions in the Obama administration's stimulus bill are a step toward the goal of nearly universal electronic health record adoption in the U.S. over the next 10 years (compared with 17 percent today). The central feature of the plan is incentive payments for using electronic records for improvements in health quality, efficiency, prevention and safety.

This kind of spending, if done wrong, can have the negative market consequence of interfering with rapid innovation by locking in today’s processes and technologies, which although well-intended, came about without a systemic view. And we risk locking out the very innovations we need for meaningful health information sharing to support better decisions.

The goal for health IT should not be primarily the creation of standards or the certification of software. Rather, standards and certification should support measurable health improvements. Health improvements are not achieved by the mere installation of software; they are achieved through the effective use of information for better decision-making.

At the same time, individual states -- for example Massachusetts, which has a newly passed law that requires hospitals and community health centers in the state to implement an electronic health record systems by Oct. 1, 2015 -- have also been moving to improve the delivery of better health care through the use of EHR.

To implement these programs, the healthcare industry seems poised to increase the adoption of EHR and electronic transmission standards to promote accuracy, transparency, and processing speed across disparate information systems. Today, they have a smorgasbord of health information technologies available to help them build a far better health system.

There are, in fact, too many standards and too many organizations writing them. There are the standards that support the systems we have in place today as well as the XML/Web-based standards that support newer web-centric systems and healthcare information exchanges.

While creating EHR data is an important first goal, an EHR is much more valuable if it can be summarized, moved, and shared. This and future posts will address EHR in that broader context. These discussions will address many technical, clinical, economic, political and managerial issues of electronic health record systems.

Perhaps the most difficult challenge is to bind the standards to structured vocabularies to ensure that there is the transfer of unambiguous knowledge of the meaning of the data among cooperating systems.

Before I get specific (sometimes talking about tools to facilitate implementation of these systems), I want to point out that building technology on U.S. standards alone would leave us with essentially a non-standard EHR platform. Remember that other countries, for example Canada and England, have single-payer health systems, while the U.S. does not.

A look back to before the PC, the Internet and all that

And, before looking ahead to how Electronic Health Records of the future will likely work, I thought I'd also display a pulmonary function test report produced at Yale New Haven Hospital (YNHH) several decades ago. While its underlying data (held in a PDP-8 computer) could be transmitted electronically -- analog modem to analog modem -- over a 128 bits/sec telephone line connection, these paper reports were generally sent from the laboratory where they were generated to the office of a YNHH physician via inter-office mail or to an outside physician via the U.S. Postal Service.



Computer-generated paper report: This photograph was produced by a Poloroid instant camera that was suspended over the front of a cathode ray tube (CRT). The CRT, along with a teletypewriter, provided human readable output for the PDP-8.

In subsequent posts, I'll include talk about safe wireless practices, Web services and other seemingly off-topic subjects, because they too are important parts of the EHR story.

Interoperability will be among these topics. The linking of vital information as patients receive care from a fragmented healthcare system is a problem that has consistently plagued interoperability efforts in healthcare. The privacy, technical, and policy issues involved need be addressed in order to effectively share information across multiple organizations. Making the information available will help to prevent drug interactions and adverse events, avoid medical errors, and help inform decision making for the patient and clinician. It will also enable the support of public health efforts, improvements in research, better physician and organizational performance and benchmarking, and greater empowerment of patients and families as active participants in their own healthcare, among other benefits.

In discussing these issues, I will sometimes cite my earlier writing on financial, legal and organizational issues that appears in articles available through links provided in my bibliography at the bottom of this page.

Finally, this might be a good time to introduce a few technical terms: HL7, HL7 mapping, HIPPA etc. They're central to the discussion of IT health records management.

Health Level 7 (HL7)

HL7 refers to both a standards organization and the set of healthcare messaging standards that it creates. Founded in 1987 to create a set of standards for hospital information systems (HIS), HL7 has expanded its reach to the creation of international standards that transgress hospitals to address clinical and administrative data in healthcare domains such as pharmaceutical, medical device, and insurance transactions. There are already a large number of countries that have mandated the use of HL7 for the transmission of healthcare data and there is an expectation that HL7 will become a part of the United State's Health Insurance Portability and Accountability Act (HIPAA) in the future.

For the large number of international healthcare organizations that are embracing the electronic transmission of healthcare data, there remain some formidable challenges. Though some compliance regulations specify the newer, XML-based HL7 v3.x, there are many jurisdictions that still need to update their legacy systems to handle this format, and many that even have multiple disparate data formats in the same system.

In the US, for example, many legacy HISs employ HL7 EDI messages alongside HIPAA X12N messages. Though these formats have quite a lot in common, syntactically speaking, they are by no means interoperable and must be mapped on-the-fly to create a dynamic workflow for managing healthcare transactions. Of course, the introduction of the XML-based HL7 v3.x adds EDI/XML mapping to the complication of mapping data from EDI to EDI.

HL7 mapping

There are off-the-shelf, any-to-any graphical data mapping tool (e.g., Altova MapForce) that supports mapping HL7 data, in its legacy EDI or newer XML-based format, to and from XML, databases, flat files, other EDI formats, and Web services. Mappings are implemented by simply importing the necessary data structures (MapForce ships with configuration files for the latest EDI standards and offers the full set of past and present HL7 standards as a free download on its Web site) and dragging lines to connect nodes. A built-in function library lets you add advanced data filters and functions to further manipulate the output data. MapForce can also facilitate the automation of your HL7 transaction workflow through code generation in Java, C#, or C++ and an accessible command line interface. Additional support for mapping HL7 data to and from Web services gives healthcare organizations the ability to meet new technology challenges and changing enterprise infrastructures as they unfold within internal and external provider domains.

Website and Supermarket Optimization {there are similarities and differences}

The relatively new field of Website optimization uses specialties such as statistics, user experience testing, and cognitive psychology to get visitors to convert (i.e., do what you want them to do, once they've landed on your site). I talk about these topics in my recent article Statistical and Financial Considerations in Website Optimization. There's a link to it at in my selected bibliography at the bottom of this blog, for anyone who's interested.

The optimization of Websites and supermarkets are both data driven tasks and both have the same goal: to capture visitor/customer activities in your Website/store and transform data about these behaviors into actionable management information.

Before getting too caught up in the art and science of Website optimization, it might be useful to pause for a moment and review some of the widely-used practices for in-store marketing and layout. As you do so, try to see the rather conspicuous parallels between what we do (as outlined in my article) and what they do (as outlined below).

But, remember that unique differences exist between how internet and brick and mortar channels can and do make money. For example, search engines like Google, Yahoo and Sphere help Website publishers to find stories by aggregating links to newspaper websites and blogs. In so doing, they wrest ad dollars from them that they think should be theirs. Not to mention the fact that, in so doing, these Websites are taking copyrighted material. Option like this are clearly not available to brick and mortar stores.

Note: The Associated Press and its member newspapers will take legal action against Web sites that use newspaper articles without legal permission, the group said recently, in a clear shot at aggregators like Google.

From a consumers point of view, a supermarket is quite simple; Put what you want into your cart and go through the check-out. Behind the scenes though, psychology is used a lot to define what products and brands you buy in supermarkets. Stands are designed to catch your eye and the store layout is structured to maximize profit.

Eye level marketing

Generally speaking, the most expensive items with high profit margins are placed on shelves that are at shoppers' eye level. Statistics show that you are more likely to see them than the less profitable brands at the very top or near your feet.

Aisle order

Some customers, particularly men, tend to simply shop for what they want, walking down an aisle grabbing what they want, turning back and walking the way they came, this is called the 'Boomerang Effect'. In order to maximize shopper and produce contact time, markets therefore place major items and brands in the middle of aisles ensuring that from any direction the customer doesn't have to walk the farthest to reach them.

Product grouping

Items that complement each other are often found close together to entice you to buy more. You'll often find pasta sauces on the same display as a featured brand of pasta.

Food smells make you feel hungry

Another tactic supermarkets use is the smell of freshly baked bread coming from the in-store bakery. The smell of warm bread makes people feel hungry. When you feel hungry while shopping you are more likely to buy additional items. Most Supermarkets bake their bread early in the morning; however, to entice more customers, some have resorted to pumping out the smell of fresh baking bread to add to the illusion that it is constantly baked through the day.

Essentials at the back

Supermarkets hit upon the idea of placing the essentials, such as bread and milk, at the back of the store. This is in order to make people have to walk past the rest of the produce, and heighten the possibility of impulse buys, in order to get their necessities. (Changing rooms in clothes stores are almost always situated at the rear of the store.)

Attracting children

One American supermarket chain came up with the idea of drawing a hopscotch in the aisle next to the children's cereal in order to make the children play and thus pin Mom & Pop to a point where the children could hassle them for treats.

Irrational Pricing

Irrational pricing is putting the price of items at say 4.99 instead of 5. The reason offered for not instead rounding $4.99 to $5.00 is based on memory processing time. Rounding upward involves an additional decision compared with storing the first digits. Furthermore, due to the vast quantity of information available for consumers to process, the information on price must be stored in a very short interval. The cheapest way to do so, in memory and attention terms, is by storing the first digits. Therefore customers perceive to be getting a better deal than they in fact are.

Point Of Sale

While you are waiting to pay, retailers often install Point Of Sale displays, this is especially prevalent in Supermarkets who install racks of chocolate to tempt bored children waiting with their parents.

Shuffle and Time

Many stores have a policy of regularly rotating the stock. This happens especially in supermarkets where people regularly shop for the same items. The idea obviously is to confront customers with a variety of items aside from their regulars and encourage them to explore areas of the store they may not usually visit.

The longer customers spend in a store the more money they are likely to spend there. Therefore stores work to make sure customers have to spend the maximum amount of time in their stores, placing obstacles constantly in the way of efficient shopping.

Newer areas where a lot more research is needed

In the UK, the British buy almost two-thirds of their wine from supermarkets; and more than a third of all wines sold in America are purchased at grocery stores, even though only 33 of the 50 US states allow supermarkets to sell wine. Wine is now the largest supermarket category in New Zealand and supermarket sales represent around 60 per cent of total wine sales.

As with any product promotion, there is no 'one solution' for supermarket wine departments. Unlike staples such as milk or eggs, wine is a luxury item (although many will beg to differ). This is the first hurdle for retailers. The difficulty of overcoming this hurdle varies from region to region. Additionally, retailers must decide how much effort and expense to invest in this part of their store. Decisions on these matters must also take into account the local competition, be it a wholesaler or a new chic wine boutique. First and foremost, know your customers.

At the end of the day, basic marketing principles will sell more wine than the most experienced supermarket wine steward. The longer customers stay in the wine department, the more likely they are to make a purchase. That can be encouraged by a tasting, music, or warm lighting. One Piggly Wiggly supermarket in Wisconsin features an expansive wine and spirits department that replicates a speciality wine cellar complete with wood shelving with a library ladder and an extensive walk-in cooler. Music can help in more specialist sales settings. Research at Leicester University showed that French music played in a supermarket's wine aisle boosted sales of French wines. The following day, German folk music led to German wines flying off the shelves.

Here, as in Website optimization, there's no substitute for subject matter expertise!