Evaluating EHRs and EMRs in Canada’s e-Health System

Sandra Milicic
McMaster University, Hamilton, Ontario, Canada

A Commentary

This issue (Volume 2 | Issue 3) of Health Reform Observer -Observatoire des Réformes de Santé is devoted to the policy attempt(s) at developing and implementing e-health inCanada. The articles by Daniels (2014) and Zelmer and Hagens (2014) document two key moments in the history of e-health policy formation: first, the establishment of Canada Health Infoway (CHI) as an arms-length body by the federal government in 2001 in order to provide partial funding to provinces for the set-up costs of developing interoperable e-health systems (Daniels 2014). A total of $2.1 billion have been disbursed on e-health projects over the course of 13 years. Second, and more recently, the federal government increased CHI’s funding by $500 million through its 2009 budget, to increase the implementation of electronic medical records (EMRs) in physicians’ practices across the country (Zelmer and Hagens 2014; Auditor General of Canada 2009; Government of Canada 2010).

E-health is comprised of many different tools (including telemedicine for instance) but the main two components of e-health systems in Canada are the electronic health record (EHR) and the EMR. Both EHR and EMR are records under the custodianship of the health care provider(s) holding health information about an individual over their lifetime, but three main differences to note (Hodge 2011; Auditor General of Canada 2009) are:

1. an EHR is a complete health record while an EMR is a partial health record;
2. an EHR is described as a ‘person-centric’ health record while an EMR is described as a ‘provider-centric’ or ‘health organization-centric’ health record;
3. an EHR can be accessed online from many separate, compatible systems within a network (i.e., used by many approved health care providers or health care organizations) while an EMR can be accessed from a single system in a doctor’s office and it may, or may not, be shared with other health care professionals.

EHRs are intended to offer solutions to some persistent problems in Canada’s health system such as sharing patient information among health care professionals—avoiding unnecessary or duplicate diagnostic tests, multiple prescriptions, and the risk of adverse drug reactions (Auditor General of Canada 2009). Ultimately, it is expected that the use of EHRs can reduce patient wait times, reduce costs, and save lives (Auditor General of Canada 2009). EMRs are a much more modest tool and, in and of themselves, cannot be expected to yield all the benefits of EHRs listed above (even though there is some evidence in the literature that EMR use in a primary care setting may improve quality of care, for example by helping patients manage their chronic conditions since they receive reminders for appointments) (CIHI 2014; Crosson et al. 2012; Loo et al. 2011; Holbrook et al. 2009). But EMRs can be seen as a necessary building block in the implementation of interoperable EHRs: it is only when primary care doctors can document the health history and situation of their patients that person-centric systems can be developed, the EMR being the central piece of the EHR. It is necessary that the percentage of primary care doctors using EMRs be significantly high in order for the potential benefits offered by EHRs to be fully realized (CMA 2010).

In line with the mandate of the journal, the articles (Daniels 2014; Zelmer and Hagens 2014) document how and why the reforms were decided and implemented and discuss the evidence available on the impact of these reforms. This Commentary focuses on the evaluation aspect, not only of these two reforms per se, but, more broadly of the implementation and use of EHRs and EMRs in Canada. Evaluating EMR/EHR programs is challenging due to a lack of available data and context-specific literature. First, most studies have relied on comparisons with the United States, even though the drastic difference in the structure of the health care systems between the two countries makes generalizability questionable. Second, much is made of raw rates of adoption of EMRs in primary care practice (e.g., comparisons of the 64% of primary care doctors in Canada using EMR to enter and retrieve clinical patient notes in the care of their patients with the more than 90% of doctors using EHRs in countries such as Austria, the United Kingdom, New Zealand and the Netherlands (Webster 2010)), but these comparisons may be meaningless, for at least two reasons. First, we must keep in mind that the comparison is not based on one survey following the same methodology and guaranteeing the same response rate in all countries. The rate for Canada is based on the 2013 National Physician Survey that had a response rate of 17.5% only. Moreover, it is based on self-reports and a vague question about any kind of use of EMRs. If surveys conducted in other countries included questions with more probes or questions inviting more positive answers, Canada’s perceived backwardness may be nothing more than a methodological artifact. Second, the context in which Canadian doctors operate is very specific. This might explain, more than a lack of infostructure or projects on the ground, why EMR adoption is slow in Canada.

In the Canadian context, primary care physicians are the gatekeepers to the health care system—that is, most often they are the first point of service in health care. Adopting EMRs in the clinical setting means that health professionals must make changes to the way they do their work—and this is not a small and uncomplicated feat. The traditional remuneration mechanism for physicians in Canada has been fee-for-service (FFS) which is retrospective, rewarding physicians exclusively for volume of care and providing no incentives for cost management. The incentives of FFS and of EMRs are not aligned. Although alternative payment options are appealing since they may provide the right incentives for physicians to consider the benefits of treatments and associated costs, 71% of total clinical payments in 2011-2012 were for FFS with alternative payments making up the 29% (CIHI 2013). Thus, in order for more doctors to adopt EMRs, it may be necessary to provide the right incentives (that can offset the significant cost of EMR adoption), or to change the methods for remuneration for physicians. The Canadian Medical Association (CMA) makes the suggestion to physicians to contact their respective provincial medical association in order to obtain the latest information about primary care reform and incentives that may be part of offered alternative payment plans (CMA 2011). The CMA makes a clear and direct statement that a comprehensive EMR system that will capture all of the required data would be beneficial for the physician and it would help to ensure that physicians receive all of the bonuses for which they may qualify (CMA 2011). This observation that the cost of investing in an EMR system is not easy to offset in a FFS remuneration scheme may be what prompted CHI, as part of the 2010 investment, to offer a program called ‘EMR & Integration’, which aims to promote the effective use of EMR software by physicians in order to achieve clinical value and to increase the number of clinicians using EMR systems (Zucker 2011).

Not only is it complex to measure and interpret the rate of use of EMRs, but also it is far from being the only measure of interest in evaluating EHRs. Using the aim of the 2010 investment (“to enhance the safety, quality and efficiency of the health care system, and create thousands of sustainable, knowledge-based jobs throughout Canada” (Government of Canada 2010)), any evaluation of EMR/EHR programs should study the effect of EMR and EHR implementation on the outcomes of the health care system (and compare their value to the costs of implementing an EMR system). How can we make progress in this direction?

The CMA has suggested several ways to model physician-EMR functional performance and physician EMR-related quality outcome indicators (CMA 2010). These can be used to create a consistent system of measurement across the provinces for evaluation of EMRs.

Physician-EMR functional performance measures. In this model, two categories of functional indicators are used to measure physician EMR performance: 1) physician use of EMRs in their practices on a routine basis for ‘core tasks’ (e.g., electronic ordering of laboratory tests and electronic prescribing of medication) and; 2) computerized capacity to generate patient information (e.g., list of patients by diagnosis, list of patients by lab result). Thus, each jurisdiction should undertake an annual survey based on a model of physician-EMR performance indicators to measure how primary care practices are progressing in EMR performance and use according to each of the indicators.

Physician EMR-related quality outcome indicators. Since it is expected that targeted investment in frontline Health Information Systems (HIS)¹ solutions will generate improvements in health outcomes, a set of indicators needs to be defined that measure improvements and track progress over time. These indicators would permit comparison of Canadian results with other jurisdictions and/or countries. Canadian Institute for Health Information (CIHI) has developed a set of primary health care indicators and a subset of these was identified for physician-EMR systems. Organized into three categories, these indicators are: 1) primary and secondary prevention (such as health risk screening, screening for modifiable risk factors in adults with hypertension); 2) patient safety (such as antidepressant monitoring); and 3) outcomes (such as glycemic control for diabetes, treatment of depression). Using these indicators as the basis, a set of finalized indicators should be established which can be used as a set of physician-EMR quality outcome indicators in order to assess the impact of e-health investments.

Lau et al. (2010) undertake a meta-level synthesis² in order to consolidate published systematic reviews on the effects of HIS on the quality of care. They use the CHI Benefits Evaluation (BE) framework (which explains how information, system, and service quality can affect the use of an HIS and user satisfaction) as an organizing scheme to categorize benefit measures in meaningful ways. In this framework, net benefits are measured under the dimensions of health care quality, provider productivity, and access to care. Lau et al. (2010) grouped measures that did not fit into the existing BE dimensions under new categories. From the 50 HIS reviews that were included in their critical assessment, Lau et al. suggest—based on the state of knowledge on the effects of HIS in medication management, health conditions, preventive care, data quality, and care process/outcome—that it would be fruitful to consider for refinement additional factors not covered by the traditional BE. These include important contextual factors such as: perception and attitude, implementation, improvement, incentives, and interoperability—which overlap with the CMA suggestions noted above.

Although there has been persistent suggestion that Canada’s health care ‘system’ must transform from one in which a large number of participants working in silos focus primarily on managing illness, to one in which collaborative and coordinated care aims to deliver an integrated array of services to Canadians (Romanow 2002; Kirby et al. 2002; Lalonde 1974), the move from a health system that is focused on acute, episodic care to one that is patient-centered and integrated across providers and organizations has been slow. Improving access to health data for those providing care by building Canada’s health information technology infrastructure (e-health system) is a key factor in such a transformation. But in order for EHRs to do their job and to realize the benefits of an e-health system, EMRs need to be adopted by physicians. The evaluation of EMRs and EHRs for Canada is lacking, and it would be fruitful to undertake economic analyses of the associated benefits and costs of use and implementation, taking into consideration the suggestions by the CMA and Lau et al. (2010). Moreover, linking administrative health care use data to survey data may provide some direction and insight.

In summary:

1. Implementing EHRs is not only a matter of technology; it requires changes in the way the health system is organized (payment schemes and roles and responsibilities of doctors).
2. Evaluating EHRs should focus on their effect on outcomes of the health system. Both the CMA and a recent review by Lau et al. (2010) provide interesting insights on how to conduct such economic analyses in the Canadian context.
3. The following criteria should be used to evaluate an EHR/EMR policy in the Canadian context (as suggested by Lau et al. and CMA):

• Consideration of organizational context, setting and practice (e.g., FFS physician or team-based organization);
• Performance measures: physician use of EMRs in their practices and, for example, provider time efficiency, computerized capacity to generate patient information, technical performance, information availability, user readiness and user competency;
• Quality Outcome Indicators: primary and secondary prevention, patient safety, health outcomes, care access/availability.

¹ HIS is comprised of a variety of systems such as EHRs, EMRs, personal health records, electronic prescribing, computerized physician order entry systems and clinical decision support systems.

² Meta-synthesis is a rigorous qualitative method that uses a specific methodology (outlined by the researcher) to interpret and transform data across a series of qualitative studies on similar phenomena. It provides direction for the effect of a variable or intervention on a clinical problem when the variables of interest are more subjective experiences and not easily quantified (e.g., grief or stigma). See Rice (2008) for further detail.