Evidence-Based Informatics

The evolution of EBP underscores its potential impact on quality of care and health outcomes. To Err Is Human, a well-known Institute of Medicine report, estimated that approximately 100,000 patients were harmed annually by the healthcare system (Institute of Medicine, 2000). The report emphasized the significance of preventable harm and death. Experts in 2014 considered 100,000 deaths as an underestimation suggesting that more recent data indicate the mortality rate of medical errors ranges from 210,000 to 400,000 annually (McCann, 2014; James, 2013; Makary & Daniel, 2016; Anderson & Abrahamson, 2017). There is a debate that the actual number of deaths attributed to medical errors has been overestimated (Jarry, 2021). Nonetheless, healthcare organizations have not irradicated avoidable medical errors.

Crossing the Quality Chasm (Chasm), a 2001 Institute of Medicine report following To Err Is Human, reported that national leaders identified the gap between what is known about best care and what is practiced. Moreover, the Chasm report cited more than 100 surveys of quality of care with scores on the “report card” indicating healthcare performance as poor. Box 11.1 presents findings from these surveys, comparing current care to what was deemed the best care or standards of care. These results indicated that improvement was needed if every patient is to receive consistent, high-quality care. The report identified EBP as a solution to achieve the goal of quality healthcare.

Another set of reports demonstrating this challenge is the National Healthcare Quality Report (NHQR) and the National Healthcare Disparities Report (NHDR) produced by the Agency for Healthcare Research and Quality (AHRQ). These reports, published annually since 2003, have provided an annual snapshot of the quality of care across the country (Agency for Healthcare Research and Quality [AHRQ], 2015). Beginning in 2014, findings on healthcare quality and healthcare disparities were integrated into a single document titled the National Healthcare Quality and Disparities Report (QDR). This report highlights the importance of examining quality and disparities together to gain a more comprehensive picture of healthcare. “The report demonstrates that the nation has made clear progress in improving the health care delivery system to achieve the three aims of better care, smarter spending, and healthier people, but there is still more work to do, specifically to address disparities in care.” (AHRQ, 2015). For example, patient safety improved, as demonstrated by a 17% reduction in rates of hospital-acquired conditions (HACs) between 2010 and 2013. However, across a broad range of measures, recommended care is delivered only 70% of the time. Similarly, the downward trend for the HACs continued for the years between 2014 and 2017, as evidenced by a 13% reduction (AHRQ, 2020). Failure of care delivery, including clinician-related inefficiencies such as variability in care, has not been resolved, resulting in continued waste and cost (Shrank, Rogstad, & Parekh, 2019).

Solutions to the healthcare quality gap are offered in the Chasm report (Institute of Medicine [IOM], 2001). The IOM expert panel issued recommendations for urgent action to redesign healthcare so that it is safe, timely, effective, efficient, equitable, and patient-centered, often referred to as the STEEEP principles (IOM, 2001). Each of the STEEEP redesign principles is described further in Table 11.1.

The Chasm report continues to be a major influence, directing national efforts targeted at transforming healthcare. For example, the STEEEP recommendations are now reflected in health profession education programs. The American Association of Colleges of Nursing (AACN) educational competencies include requirements for programs to prepare nurses who contribute to quality improvement (American Association of Colleges of Nursing [AACN], 2021). AACN Essentials, updated in 2021, specify that professional nursing practice be grounded in translation of current evidence into practice and further point to the need for knowledge and skills in information management as being critical in the delivery of quality patient care (AACN, 2021). Likewise, the Accreditation Council for Graduate Medical Education (ACGME) requires medical education in quality improvement (Accreditation Council for Graduate, 2020). The STEEEP principles are also reflected in clinical practice resources, such as the AHRQ Health Care Innovations Exchange, where the STEEEP elements are the selection criteria for inclusion in this unique clearinghouse (AHRQ, 2021).

Quality of care and EBP are conceptually linked, and form the hub of healthcare improvement. The descriptions and definitions reflect the overlap of these concepts and offer reference points against which to expand their understanding. In particular, the focal point of both is the use of knowledge in practice.

POC testing can occur within the LHS framework. Through informatics, a health system can implement an evidence-based approach to learn what is and what is not working in healthcare quality and safety. An LHS is broadly defined as “any entity that routinely and continuously seeks to generate and learn from data, for purposes of improving individual and population health.” (Guise et al., 2018). It is important to note that an LHS is not designed to replace more a time-intensive rigorous research design but seeks to compliment such efforts. There are tradeoffs to relying solely on a less controlled approach in determining optimal health care delivery practices.

The LHS recognizes the vital role the patient plays in the continual learning process. Providing a patient-centered approach to health outcomes is achieved by using patient data to formulate, evaluate, and update processes. With the advancements in informatics and IT, organizations now seek to develop “rapid” LHSs by creating well-orchestrated pipelines of near-real time data that flows back to the clinician and allows for iterative and continual refinement of the knowledge base (Abernathy et al., 2010). It must be noted that while beneficial, the LHS is not without implementation difficulties (Smoyer et al., 2016). For example, an LHS requires substantial IT investment, including a well-constructed backend data architecture, and advanced analytic capabilities, all of which can be resource-intensive. Both data storage and analytic capabilities are equally necessary. An LHS can begin to provide the environment to initiate precision medicine. For example, if genetic test results are simply stored as scanned image files without further analytic processing, then the data interpretation is still left for busy clinicians to complete. Data capture coupled with advanced analytics is required in rendering these data suitable for clinical use, such as being able to quickly determine a patient’s genetic attributes and appropriate model for treatment (Williams et al., 2018). New infrastructures are recommended for the future, including computable representations, augmented with traditional human-readable representations (e.g., text, graphs, and figures) due to the quantity of evidence and the time to change practice. Computable representations include other types of evidence (e.g., rules and predictive models). Notably, the novel representations can be digitized as knowledge objects (DKOs). The DKOs would be available through digital libraries.

The primary impetus for EBP in healthcare is the selection of the option that improves the patient’s health. Evaluation and use of research evidence support this option. In concert with the use of research evidence is the use of an institutional LHS using patient data (evidence) and analytics to assess a personalized, evidence-based approach, to the current medical issue. Clients present multiple actual or potential health problems requiring management or resolution (e.g., living with asthma, learning disabilities, and obesity management). Clinical actions may not be based on the best available scientific knowledge and, therefore, offer less effective client care (IOM, 2001). The essential role of a healthcare provider is to select and provide interventions providing a positive patient outcome and the most effective strategies for changing the microsystem or system of care. Clinicians choose from and interpret various clinical data and information while facing pressure to decrease the uncertainty, risks to patients, and costs. Knowledge underlies these decisions and plays a primary role in the care provided.

Evidence-based clinical decision making can be described as a prescriptive approach to making choices in diagnostic and intervention care, based on the idea that research-based care improves outcomes most effectively. Research-based care provides evidence about which option is most likely to produce the desired outcome. EBP is seen as a key solution in closing the gap between what is known and practiced.

However, essential questions lie between accepting this as true and the clinician’s and system’s ability to enact it: How do clinicians know which interventions will most likely diminish or resolve the health problem and help the client reach their health goal? What resources are available to apply EBP principles directly in clinical decision making? Answering these questions begins with analyzing the different models of EBP.

Prominent EBP models can be grouped into one of three categories of models for designing and implementing systematic approaches to strengthen evidence-based clinical decision making (Mitchell et al., 2010). Table 11.2 describes the critical attributes and provides examples in three categories.

The first category includes models that focus on EBP, research use, and knowledge transformation principles. These models emphasize a systematic approach to synthesizing knowledge. The models specify a series of processes designed to:

Identify a question, topic, or problem in healthcare.

Some models in this category emphasize the process where research findings can be transformed into a more useful form, such as a clinical practice guideline (CPG) or standard of care, that can be used to guide clinical decision making in practice. Other models reflect a PBE approach by addressing the outcomes' evaluation to determine whether the EBP change has produced the expected clinical outcomes or to compare actual practice and ideal practice (thereby identifying unacceptable practice variation).

The second category includes models that offer an understanding of the mechanisms by which individual, small group, and organizational contexts affect the diffusion, uptake, and adoption of new knowledge, as well as innovation, which is essential to the design of EBP initiatives. These models propose that specific interventions accelerate the adoption of practices based on best evidence. Examples of such interventions include:

Thus, within these models, feedback regarding both patient and practitioner outcomes is seen as a change strategy.

A third category of models and frameworks postulates that formalized, bidirectional, and ongoing interactions among practitioners, researchers, policymakers, and consumers accelerate the application of new discoveries in clinical care. This ongoing interaction increases the likelihood that researchers will focus on problems of importance to clinicians. Such models simultaneously address the generation of new knowledge (discovery) and uptake. This collaboration supports the exchange of expertise and knowledge to strengthen decision making and action for all involved parties (Mitchell et al., 2010).

Each EBP model described previously has perspectives that may prove valuable in designing and advancing informatics approaches to EBP. The following section examines the application of one of these models, the Stevens ACE Star Model of Knowledge Transformation, (Stevens, 2015) which will be used to demonstrate how an EBP model can:

The Star Model provides a framework for converting research knowledge into a form that has utility in the clinical decision-making process. The model articulates a necessary process for reducing the volume and complexity of research knowledge, evolving one form of knowledge to the next, and incorporating a broad range of sources of knowledge throughout the EBP process.

The model addresses two major hurdles in employing EBP:

In both instances, informatics-based solutions have been created. The Star Model explains the key concept of knowledge transformation. Knowledge transformation is defined as the conversion of research findings from the discovery of primary research results, through a series of stages and forms, to increase the relevance, accessibility, and utility of evidence at the POC to improve healthcare and health outcomes by way of evidence-based care (Stevens, 2015).

When considering the volume of knowledge, experts point out, “no unaided human being can read, recall, and act effectively on the volume of clinically relevant scientific literature.”^11,p.25 It is estimated that more than 10,000 new research articles are published annually in medicine. Collectively, approximately 12.8 million medical and health studies were published between 1980 and 2012 (Bornmann & Mutz, 2015).

Even the most enthusiastic clinician or researcher would find it challenging to stay abreast of this volume of literature. When considering the form of knowledge as a barrier, it is clear that most research reports are not directly useful in a clinical setting but must be converted to a form applicable at the POC. Research results, often presented as statistical results, exist in a larger body of knowledge. The complexity and lack of congruence across all studies on a topic create a barrier when using this as a basis for clinical decision making. The Stevens Star Model addresses the transformation necessary for converting research results from single-study findings to guidelines that can be applied and measured for effect. Discussion of this model is expanded in the next section, with specific electronic resources for each form of knowledge identified and described.

In the Star Model, individual studies move through four cycles, ending in practice outcomes and patient outcomes. The knowledge transformation process occurs at five points, conceptualized as a five-point star, as shown in Fig. 11.1. These five points are discovery research, evidence summary, translation to guidelines, practice integration, and evaluation of process and outcome (Stevens, 2015). A description of each point is provided below, along with identification and descriptions of computerized resources and examples.

Point 3: Translation to Guidelines

In the third stage of EBP, translation, experts are called on to consider the evidence summary, fill in gaps with consensus expert opinion, and merge research knowledge with expertise to produce clinical practice guidelines (CPGs). This process translates the research evidence into clinical recommendations. The IOM defines clinical guidelines as “systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances” (IOM, 2011)

CPGs have evolved during the past 20 years from recommendations based mainly on expert judgment to recommendations grounded primarily in evidence. Expert consensus is used in guideline development when research-based evidence is lacking (Clancy & Cronin, 2005). CPGs are commonly produced and sponsored by a clinical specialty organization. Such guidelines are present throughout all organized healthcare as clinical pathways, nursing care standards, and unit policies. An exemplar of the development of CPGs in nursing is Putting Evidence into Practice in Oncology (Oncology Nursing Society [ONS], 2015). This program engages scientists and expert clinicians in examining evidence, conducting evidence summaries, generating practice recommendations, and developing tools to implement the guidelines. These online guidelines are accessible to clinicians and are also published in nursing literature (Mitchell et al., 2009).

Criteria essential for well-developed CPGs are (1) the evidence is explicitly identified and (2) the evidence and recommendation are rated. Several taxonomies have been developed to assist in rating evidence. One such taxonomy was developed by the Center for Evidence Based Medicine in the United Kingdom. This rating system identifies SRs as the uppermost strength of evidence (Center for Evidence-Based Medicine [CEBM, 2021).

The consensus of expert opinion, rated the weakest strength of all levels of evidence, is also included and counted as evidence. However, if no other evidence exists, this may be used to support clinical decision making. Well-developed CPGs and care standards share several characteristics:

• • a specified process is followed during guideline development
• • the guideline identifies the evidence upon which each recommendation is made, whether it is research or expert opinion
• • and the evidence is rated using a strength-of-evidence rating scale.

Fig. 11.2 illustrates a strength-of-evidence rating hierarchy (Stevens et al., 2009). The higher the evidence is placed on the pyramid, the more confident the clinician can be that the intervention will cause the targeted health effect.

Some EBP approaches emphasize the usefulness of CPGs in bridging the gap between primary research findings and clinical decision making (IOM, 2008). CPGs are systematically developed statements to assist practitioners and patients in decision-making about appropriate healthcare for specific clinical circumstances (IOM, 2011). CPGs are seen as tools to help move scientific evidence to the bedside. To increase the likelihood that the recommended action will positively impact the clinical outcome, it is imperative that guidelines are based on best available evidence, systematically located, appraised, and synthesized (i.e., evidence based).

Guideline resources and examples

Today, the AHRQ provides the National Guideline Clearinghouse, a searchable database of more than 2500 CPGs entered by numerous sources. Although this knowledge management database can be easily used to locate a wide variety of CPGs, the user must examine the information presented with the CPG to determine that the CPG is current, was developed systematically, and is based on best evidence.

Other guidelines can be located on the AHRQ’s website, U.S. Preventive Services Task Force (USPSTF) segment (US Preventive Services Task Force [USPSTF], 2021). These recommendations focus on screening tests, counseling, immunizations, and chemoprophylaxis and are based on evidence summary work performed by the AHRQ.

Critical appraisal of the various forms of knowledge is essential. However, for clinical decision making, appraisal of guidelines is crucial for effective clinical care. Several standards for critical appraisal of CPGs are available and can be used to examine CPGs as clinical agencies consider adoption into practice. An excellent example of a systematic approach is the instrument used to assess practice guidelines developed by the Appraisal of Guidelines for Research and Evaluation (AGREE) enterprise (Appraisal of Guidelines, 2010). The 23-item AGREE II instrument developed by this international collaboration is reliable and valid. It outlines primary facets of the CPG to be appraised: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, application, and editorial independence. Even though the instrument is not easily used by the individual clinician, it is helpful to groups and organizations charged with the adoption of specific CPGs. Box 11.4 identifies key elements to be examined when appraising a CPG.

Box 11.4

Critically Appraising a Clinical Practice Guideline

1. 1. Why was this guideline developed?
2. 2. What was the composition (expertise and disciplinary perspective) of the panel that developed the guideline?
3. 3. What entity provided financial sponsorship?
4. 4. Were decision-making processes used in developing the guideline?
   1. a. What clinical question was the guideline developed to address?
   2. b. How was the evidence used in the guideline gathered and evaluated?
   3. c. Were gaps in the evidence explicitly identified?
   4. d. How explicitly is the available evidence linked to the recommendations in the guideline?
   5. e. If lower levels of evidence were incorporated (e.g., expert opinion) in the guideline, are these instances labeled explicitly, and are the reasons for the inclusion of expert opinion, the line of reasoning, and the strength of extrapolation from other data clearly identified?
   6. f. How are patient preferences incorporated into the guideline?
   7. g. Is cost-effectiveness considered?
   8. h. What is the mechanism and interval for updating the guideline?

Recommendations flowing from evidence are rated in terms of strength. The USPSTF uses a schema for rating its evidence-based recommendations (USPSTF, 2021). It grades the strength of recommendations according to one of five classifications (A, B, C, D, and I). The recommendation grade reflects the strength of evidence and magnitude of net benefit (benefits minus harms). Table 11.4 defines each grade and indicates the suggestion for practice.

Table 11.4

U.S. Preventive Services Task Force Grade, Definition, and Suggestion for Practice

Grade	Definition	Suggestion for Practice
A	The USPSTF recommends the service. There is high certainty that the net benefit is substantial.	Offer/provide this service.
B	The USPSTF recommends the service. There is high certainty that the net benefit is moderate, or there is moderate certainty that the net benefit is moderate to substantial.	Offer/provide this service.
C	The USPSTF recommends selectively offering or providing this service to individual patients based on professional judgment and patient preferences. There is at least moderate certainty that the net benefit is small.	Offer or provide this service for selected patients depending on individual circumstances.
D	The USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits.	Discourage the use of this service.
I	Evidence is lacking, of poor quality, or conflicting, and the balance of benefits and harms cannot be determined.	If the service is offered, patients should understand the uncertainty about the balance of benefits and harms.

USPSTF, U.S. Preventive Services Task Force.

US Preventive Services Task Force. Grade definitions. http://www.uspreventiveservicestaskforce.org/Page/Name/grade-definitions; October 2014. Reprinted with permission of the Agency for Healthcare Research and Quality.

Professional groups within the agency can move forward with confidence that all research evidence has been systematically gathered and amassed into a powerful conclusion of what will work. The evidence summary is vetted through clinical experts, and interpretations are made for direct clinical application. In instances where there is a gap in the evidence summary, the experts consider the evidence of lower rating, and finally add their expertise into the fully developed CPG. Box 11.5 presents an example of Point 3 in the form of a CPG for preventing falls in the elderly. Note that both the evidence and the recommendation are rated.

Box 11.5

Example of Stevens Star Point 3: Clinical Practice Guidelines for Preventing Falls in the Elderly

Multifactorial Interventions

Strong—All older people with recurrent falls or assessed as being at increased risk of falling should be considered for an individualized multifactorial intervention. (Evidence level I)

Strong—In successful multifactorial intervention programs, the following specific components are common (Evidence level I):

• • Strength and balance training
• • Home hazard assessment and intervention
• • Vision assessment and referral
• • Medication review with modification/withdrawal

From National Collaborating Centre for Nursing and Supportive Care. (2004). CPG for the assessment and prevention of falls in older people. London, UK: National Institute for Clinical Excellence. https://www.nice.org.uk/guidance/cg161; Updated 2013, Reviewed January 2016.

Many available rating scales convey the strength of the recommendation. Coupled with the strength of evidence, the clinician can use “Level 1” evidence and “Grade A” recommendations to support clinical decisions. The USPSTF adopted a system linking strength of evidence with the strength of recommendation as follows:

• • “A” (strongly recommends)
• • “B” (recommends)
• • “C” (no recommendation for or against)
• • “D” (recommends against)
• • “I” (insufficient evidence to recommend for or against) (Harris et al., 2001).

Box 11.6 presents three examples of USPSTF recommendations, along with the grade of each recommendation.

Box 11.6

Examples of Star Point 3 USPSTF Clinical Recommendations and Grades

Ocular Prophylaxis for Gonococcal Ophthalmia Neonatorum

Release Date: July 2011

Summary of Recommendation

• • The USPSTF recommends prophylactic ocular topical medication for all newborns for the prevention of gonococcal ophthalmia neonatorum.

Grade: A Recommendation

Prevention of Falls in Community-Dwelling Older Adults

Current Recommendations

Release Date: May 2012

• • The USPSTF recommends exercise or physical therapy and vitamin D supplementation to prevent falls in community-dwelling adults aged 65 years or older who are at increased risk for falls.
  Grade: B Recommendation
• • The USPSTF does not recommend automatically performing an in-depth multifactorial risk assessment in conjunction with comprehensive management of identified risks to prevent falls in community-dwelling adults aged 65 years or older because the likelihood of benefit is small. In determining whether this service is appropriate in individual cases, patients and clinicians should consider the balance of benefits and harms based on the circumstances of prior falls, comorbid medical conditions, and patient values.

Grade: C Recommendation

Screening for Prostate Cancer

Current Recommendation

Release Date: May 2012

• • The USPSTF recommends against prostate-specific antigen-based screening for prostate cancer.

Grade: D Recommendation

This recommendation applies to men in the general U.S. population, regardless of age. This recommendation does not include the use of the prostate-specific antigen test for surveillance after diagnosis or treatment of prostate cancer; the use of the prostate-specific antigen test for this indication is outside the scope of the USPSTF.

From US Preventive Services Task Force (USPSTF). (2011). Ocular prophylaxis for gonococcal ophthalmia neonatorum. USPSTF. http://www.uspreventiveservicestaskforce.org/uspstf/uspsgononew.htm. From US Preventive Services Task Force (USPSTF). (2012). Prevention of falls in community-dwelling older adults. USPSTF. http://www.uspreventiveservicestaskforce.org/uspstf/uspsfalls.htmFrom US Preventive Services Task Force (USPSTF). (2012). Screening for prostate cancer. USPSTF. http://www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/prostate-cancer-screening. From National Collaborating Centre for Nursing and Supportive Care. (2004). CPG for the assessment and prevention of falls in older people. London, UK: National Institute for Clinical Excellence. http://www.nice.org.uk/..

Translating guidelines into practice is a labor-intensive process involving a significant cognitive load. In the busy world of healthcare, clinicians cannot routinely take time to search out CPGs and then translate their application to individual patients. However, using CPGs to design clinical decision support (CDS) systems in EHRs can make it possible for busy clinicians to access evidence-based guidelines that have been individualized to the patient’s needs and status at the POC.

Point 5: Evaluation

The fifth stage in knowledge transformation is evaluation. Practice changes are followed by evaluation of the impact on a wide variety of outcomes, including

• • Safety
• • The effectiveness of the care in producing the desired redesign of care, patient outcomes, population outcomes, efficiency, and cost factors in the care (short term and long term)
• • Satisfaction of both healthcare providers and patients.

Evaluation of specific outcomes is at a high level of public interest (AHRQ, 2015; IOM, 2001). As a result, quality indicators are being established for healthcare improvement and public reporting. Additional information on how EHRs and computerization can be used to provide evaluation data is discussed under PBE later in this chapter. Chapter 12 provides information about integrating EBP into CDS applications.

Evaluation resources and examples

AHRQ, through its National Healthcare QDR, established significant quality indicators (AHRQ, 2015). AHRQ reports provide a comprehensive overview of the quality of healthcare received by the U.S. population, as well as disparities in care experienced by different racial, ethnic, and socioeconomic groups. The reports are based on more than 250 measures of quality and disparities covering a broad array of healthcare services and settings. Key selection criteria include measures that are the most important and scientifically supported. The QDR, using these measures, present in summary statements and charts, form a snapshot of how our healthcare system is performing and the extent to which healthcare quality and disparities have improved or worsened over time.

Fig. 11.3 is a summary chart showing trends across the NQS priorities from the QDR.

Another influential entity establishing quality measures is the National Quality Forum (NQF). This nonprofit organization brings together a variety of healthcare stakeholders, including consumer organizations, public and private purchasers, physicians, nurses, informaticians, hospitals, accrediting and certifying bodies, supporting industries, and healthcare research and quality improvement organizations. The NQF’s mission includes consensus building on priorities for performance improvement, endorsing national consensus standards for measuring and reporting on performance, and education and outreach (National Quality Forum [NQF], 2016). An example of an NQF-endorsed measure for patient safety is presented in Box 11.8.

Box 11.8

Example of Star Point 5 From the National Quality Forum

Patient Safety Measure

#0674: Percent of Residents Experiencing One or More Falls with Major Injury (Long Stay) (Centers for Medicare & Medicaid Services): Description: This measure reports the percentage of residents who have experienced one or more falls with a major injury during their episode of nursing home care ending in the target quarter (3-month period). Major injury is defined as bone fractures, joint dislocations, closed head injuries with altered consciousness, or subdural hematoma. Long-stay residents are identified as residents who have had at least 101 cumulative days of nursing facility care.

Measure Type: Outcome

Level of Analysis: Facility

Setting of Care: Post-Acute/Long-Term Care Facility: Nursing Home/Skilled Nursing Facility

Data Source: Electronic Clinical Data

Adapted from National Quality Forum (NQF). (2016). Patient Safety 2015 Final Report. February 2016. http://www.qualityforum.org/Publications/2016/02/Patient_Safety_2015_Final_Report.aspx.

An important collection of quality measures is assembled in the National Quality Measures Clearinghouse (NQMC), an initiative of the AHRQ (AHRQ, 2021). The NQMC is a database and website for information on evidence-based healthcare quality measures and measure sets. Its purpose is to promote widespread access to quality measures by the healthcare community and other interested individuals. The key targets are practitioners, healthcare providers, health plans, integrated delivery systems, purchasers, and others. The aim is to provide an accessible mechanism for obtaining detailed information on quality measures and to further their dissemination, implementation, and use to inform healthcare decisions. Box 11.9 presents a measure from the NQMC. Computerized tools for analyzing the big data in these national databases and teasing out the factors influencing healthcare outcomes will provide a basis for evidence in the search for other levels of EBP.

Box 11.9

Example of Star Point 5 From the National Quality Measures Clearinghouse

Fall Risk Management

This measure reports the percentage of Medicare members 65 years of age and older who had a fall or had problems with balance or walking in the past 12 months who were seen by a practitioner in the past 12 months and who received fall risk intervention from their current practitioner.

From National Quality Measures Clearinghouse (NQMC). (2015). Measure summary: Fall risk management. https://www.qualitymeasures.ahrq.gov/summaries/summary/43758.

Successful models for integrating EBP into practice are emerging. For example, the national team performance training, Team Strategies and Tools to Enhance Performance and Patient Safety (Team STEPPS), has been promoted across military healthcare since 1995, with a “civilian” rollout initiated in 2006 (Team STEPPS, 2015). The program is built on a solid evidence base from human factors engineering and solves urgent problems arising from team-based care. The program demonstrated improvements in communication and teamwork skills among healthcare professionals leading to improvements in patient care.

The role of health IT in quality measurement was described in an AHRQ report (Anderson et al., 2012). Past quality processes were conducted via manual chart entry, manual chart abstraction, and analysis of administrative claims data. Advances are seen in locations with existing health IT. The ability to pull meaningful data from the POC is increasingly computerized as information systems are created and expanded. A potential for health IT is to evolve existing measures into electronic measures and computerized data collection (Anderson et al., 2012).

The long-term goal for improving health IT–enabled quality measurement is to achieve a robust information infrastructure that supports national quality measurement and reporting strategies. Key goals include interoperability that ensures that EHRs can share information for care coordination, patient-centered care, and cost savings. Such interoperability relies on harmonizing standards. Information exchange, integrating interoperability standards into vendor products, and data linkage are critical for advancement (Anderson et al. 2012).

Many challenges must be overcome to achieve the next generation of health IT–enabled quality measurement. First, consensus among quality stakeholders is required to move forward on the purpose of measurement, achieving patient-centricity in a fragmented health delivery system, alignment of incentives; ownership and funding; increased information exchange; and ensuring privacy, security, and confidentiality. Second, measurement challenges to be overcome include measures valuable to consumers, measures to assess value, measures for specialty uses, and accounting for variations in risk in measurement. Third, technology challenges to be overcome include expansion of eMeasures; advancement in measure capture technologies; advancement in patient-focused technologies, health information exchange, interoperability, and standards; internet connectivity; and aggregation and analysis. Notably, the report concludes with a call for stakeholder input to inform pathways to achieving the next generation of quality improvement, an important opportunity in which healthcare providers can engage (Anderson et al. 2012).

The research agenda for health informatics research focuses on issues related to identifying challenges, priorities, and those actions related to increasing the use of the EHR for research, (Zayas-Cabán et al., 2020) population health (Kharrazi et al., 2017) and supporting patient and family empowerment in care after hospitalization (Collins et al., 2018). The new research results in the need for research that tests patient centered and population and infrastructure foci in real-world settings. When the conditions for standardization and interoperability of EHRs are satisfied, patient-specific data can be connected across healthcare settings. Once this occurs, knowledge discovery about coordination and care processes can be expanded using data generated routinely from care settings using PBE principles. By organizing the informatics approach of DIKW into the EBP knowledge transformation and application in real-world settings, the field of quality improvement and EBP translation into practice can be advanced.

As recent as 2020 the ONC (Caban et al., 2020) released two goals and nine priorities as health IT research priorities. The goals included

The 2020 federal health IT principles reflect the ONC continued commitment to translating evidence into practice (Office of the National Coordinator, 2020). Accordingly, EBP and guidelines are necessary for the value-based care model.

Increased adoption of EHRs and other health information systems has resulted in vast amounts of structured and textual data. Data may be a partial or a complete copy of all data collected in the course of care provided. Moreover, data can be stored on servers in a data warehouse (a large data repository integrating data across clinical, administrative, and other systems). The data can include billing information, physician and nursing notes, laboratory results, radiology images, and other diverse data types. In some settings, data describing individual patients and their characteristics, health issues, treatments, and outcomes have accumulated for years, forming longitudinal records. The clinical record can also be linked to repositories of genetic or familial data (Duvall et al., 2012; Slattery & Kerber, 1993; Hu et al., 2011). These data constitute an underused resource for scientific research in biomedicine and nursing.

The potential of using these data stores to advance scientific knowledge and patient care is widely acknowledged. However, clinical concepts are typically represented in the EHR to support healthcare delivery but not necessarily research. For example, pain might be qualitatively described in a patient’s note and EHR as “mild” or “better.” The description may meet the immediate need for documentation and care. However, the description does not allow the researcher to measure differences in pain over time and across patients, as would measurement using a pain scale. Clinical concepts may not be adequately measured or represented in a way that enables scientific analysis. Data quality affects the feasibility of secondary analysis.

PBE studies are observational studies that attempt to mitigate the weaknesses traditionally associated with observational designs. The mitigation is accomplished by exhaustive attention to determining patient characteristics that may confound conclusions about the effectiveness of an intervention. For example, observational studies might indicate that aerobic exercise is superior to nonaerobic exercise in preventing falls. Nevertheless, if the prescribers tend to order nonaerobic exercise for those who are more debilitated, the severity of illness is a confounder and should be controlled in the analysis.

This method requires training and quality-control checks for the reliability of the measures of the actual process of care. Statistical analysis involves correlations among patient characteristics, intervention process steps, and outcomes. PBE can uncover best practices and combinations of treatments for specific types of patients while achieving many of the presumed advantages of randomized controlled trials (RCTs), especially the presumed advantage that RCTs control for patient differences through randomization. Frontline clinicians treating the study patients lead the study design and analyses of the data prospectively based on clinical experience. The characteristics of PBE are summarized in Table 11.5.

PBE is an innovative prospective research design that uses data gathered from current practice to identify what care processes work in the real world. EBP is about using evidence to guide practice. PBE is about obtaining evidence from practice.

PBE studies mitigate the weaknesses usually associated with traditional observational designs in four main ways:

PBE studies require comprehensive data acquisition. By using bivariate and multivariate associations among patient characteristics, process steps, and outcomes can be identified. Concurrently, PBE study designs are structured to minimize the potential for false associations between treatments and outcomes. These studies focus on reducing the effects of potential alternative factors or explanations when estimating the complex associations between treatments and outcomes within a specific care context (Horn et al., 2005). However, the identified associations between treatment and outcome are not considered causal links. The associations still inform causal judgments to the extent that the research design can measure and statistically control for these confounders or alternative explanations.

The PBE approach does not infer causality directly like RCTs, but several sources indicate the strength of the evidence that a causal link exists. First, alternative hypotheses regarding possible causes are tested using a large number of available variables to identify additional potential variables that may influence outcomes. Results can be used to drill down to discover potential alternative causes and to generate additional specific hypotheses. Analyses continue until the project team is satisfied that they cannot think of any other variables to explain the outcomes. Second, one can test the predictive validity of significant PBE findings by introducing findings into clinical practice and assessing whether outcomes change when treatments change, as predicted by PBE models. Third, studies can be repeated in different healthcare settings and assessed to determine if the findings remain the same.

Underlying the common criticism of observational studies (that they demonstrate association but not causation) is an unchallenged assumption that the evidence for causation is dichotomous; that is, something either is or is not the cause. Instead, the evidence for causation should be viewed as a continuum that extends from mere association to undeniable causation. While observational studies cannot prove causation in some absolute sense, by chipping away at potential confounders and by testing for predictive validity in follow-up studies, we move upward on the continuum from mere association to causation. PBE studies offer a methodology for moving up this continuum.

Research design involves a balance of internal validity (the validity of the causal inference that the treatment is the “true” cause of the outcome) and external validity (the validity that the causal inference can be generalized to other subjects, forms of the treatment, measures of the outcome, practitioners, and settings). Essentially, PBE designs trade away the internal validity of RCTs for external validity (Mitchell & Jolley, 2001). PBE designs have high external validity (generalizability) because they include virtually all patients with or at risk for the condition under study, as well as potential confounders that could alter treatment responses. PBE designs attempt to minimize threats to internal validity by collecting information on all patient variables—demographic, medical, nursing, functional, and socioeconomic—that might account for differences in outcome. By doing so, PBE designs minimize the need for compensating statistical techniques such as instrumental variables and propensity scoring to mitigate selection bias effects, unknown sources of variance, and threats to internal validity.

PBE study designs attempt to capture the complexity of the healthcare process presented by patient and treatment differences in routine care; PBE studies do not alter or standardize treatment regimens to evaluate the efficacy of a specific intervention or combination of interventions, as one usually does in an RCT or other types of experimental designs (Horn & Gassaway, 2007, 2010). PBE studies measure multiple concurrent interventions, patient characteristics, and outcomes. This comprehensive framework provides for consequential analyses of significant associations between treatment combinations and outcomes, controlling for patient differences.

Table 11.6 outlines the steps involved in conducting a PBE study and gives a brief description of what each step involves. Once a clinical issue is identified, PBE methods by forming a multidisciplinary team, often with representatives of multiple sites. Participation of informaticians on the team is critical to ensuring that the electronic documentation facilitates data capture for the research and clinical practice without undue documentation burden.

PBE methods work best in situations where one wishes to study existing clinical practice. However, there are no limitations related to conditions or settings for the use of PBE study methods. The technique can be time-consuming in terms of conducting the initial PBE steps, as well as data extraction. Although the relevant variables may change, PBE study designs have been found to work in various practice settings, including acute and ambulatory care, inpatient and outpatient rehabilitation, hospice, and long-term care, and for adult and pediatric patients.

Data elements needed for PBE, and especially for CSI, can be captured in structured, exportable formats while also being used for clinical documentation of care EHR use in patient care as research expands. This concept is implemented already in health systems in Israel and various PBE studies in the United States (Deutscher et al., 2008). However, transitioning to EHRs presents its own challenges, especially for data-intensive PBE studies. EHRs can facilitate data acquisition, but they are not always research-friendly because many desired data elements are in text, such as clinical notes, and cannot be exported easily. If EHR data cannot be exported directly, they must be abstracted manually or alternatively, processed using Natural Language Processing (NLP) methods (Spyns, 1996). Whereas manual abstraction from EHR can be more labor-intensive than abstracting paper charts are, when relying on NLP methods, a bulk of the effort is spent on training the computer algorithms to extract the needed data elements. In addition, EHR modifications for optimizing point-of-care data documentation and abstraction are costly and time-consuming, potentially slowing down the planning and implementation of PBE studies based on routine electronic data capture. With HITECH, EHRs have become pervasive in clinical practice (Mennemeyer et al., 2016). Over time, new EHR exporting and reporting software are emerging, making EHR data abstraction less labor-intensive.

Examination of CSI elements themselves may show a reduced set that differentiates severity, as well as the full set. It is possible that two valid data elements would each contribute clinically unique information but be fully redundant with respect to their ability to differentiate severity in a population. For example, unresponsive neurological status and fever ≥ 104 degrees are clinically unique indicators of severity in pneumonia. If these indicators differentiate only the same most gravely ill pneumonia patients from the rest of the population, they provide redundant information with respect to the severity of pneumonia. In this case, it may be possible to use only one of the two indicators in CSI scoring, reducing the information burden to compute a CSI score and increasing efficiency.

PBE requires a multidisciplinary team approach for comparative effectiveness research and ensures inclusion of a wide spectrum of variables so that differences in patient characteristics and treatments are measured and controlled statistically (Horn et al., 2012).

The next step for comparative effectiveness research is to conduct more rigorous, prospective large-scale observational cohort studies. National efforts such as the Patient-Centered Outcomes Research Institute’s (PCORI) Clinical Data Research Networks (CDRN) can enable comparative effectiveness research and PBE by providing informatics solutions for sharing data across multiple institutions (Amin et al., 2014). From a PBE perspective, rigor entails controlled measurement of outcomes related to multiple intervention combinations and a variety of patient characteristics in diverse clinical settings (Horn et al., 2012). PBE studies address questions in the real world where multiple variables and factors can affect the outcomes; they can fit seamlessly into everyday clinical documentation and, therefore, have the potential to influence and improve the evidence in EBP in the real-world clinical environment of patient care.