Quality Management

Fig. 7.1B illustrates that if a systematic bias (error) occurs in the measurements, the mean value shifts to another value. Note that the imprecision is the same as before the bias occurred because it is unlikely, although not impossible, that a change in imprecision would occur at the same time as a bias shift. The primary purpose of measuring QC samples is to statistically evaluate the measurement procedure to verify that it continues to perform within the specifications consistent with its acceptable expected stable condition or to identify that a change in performance occurred that needs to be corrected. QC result acceptance criteria are based on the probability for an individual QC result to be different from the variability in results expected when the measurement procedure is performing in a stable condition within its specifications.

Fig. 7.2 shows a Levey-Jennings chart that is the most common presentation for evaluating QC results. This format shows each QC result sequentially over time and allows a quick visual assessment of performance. Assuming the measurement procedure is performing in a stable condition, the mean value represents the target (or expected) value for the QC result, and the SD lines represent the expected imprecision. Assuming a Gaussian (normal) distribution of imprecision, the results should be distributed uniformly around the mean with results observed more frequently closer to the mean than near the extremes of the distribution. The data show fluctuations in performance at different intervals and illustrate the importance of calculating the SD over a long enough interval to appropriately include all sources of variability in the measurement procedure. Note that a small number of results in Fig. 7.2 are greater than 2 SDs, and three results slightly exceed 3 SDs, which is expected for an approximately Gaussian distribution of imprecision. The number of results expected within the standard deviation intervals (SDIs) is:

Model 2 bases the TE_a on a fraction of the within and between individual biological variations of the measurand. (For additional information on biological variation refer to Chapter 4.) This model minimizes the ratio of the “analytical noise” to the “biologic signal,” with an assumption that a small ratio will identify measurement procedure performance that relates to the medical requirements. Tables of optimal, desirable, and minimal TE_a based on biological variation are available.

Model 3 bases the performance specifications on the “state of the art,” which is the performance capability of a measurement procedure usually derived from QC data. The laboratory should consult with clinical care providers to agree on an appropriate TE_a for the patient population served.

Because sigma assumes a Gaussian or normal distribution for repeated measurements, the probability of a defect (i.e., an erroneous laboratory result) can be predicted. The term six-sigma refers to a condition when the variability in the measurement process is sufficiently smaller than the medical requirement that erroneous results are very uncommon. Fig. 7.3A shows a six-sigma test that has the TE_a limits 6 SDs away from the center point of the distribution of variability in measurements. A small amount of bias or increased imprecision will have little influence on the number of erroneous results produced, and the risk of producing an erroneous result even with some loss of performance is very low. Consequently, less stringent QC is suitable.

Fig. 7.3B shows a three-sigma measurement procedure that has the TE_a limits 3 SDs away from the center point of the expected distribution of variability in measurements. In the three-sigma situation, a small amount of bias or increased imprecision will cause the number of erroneous results to increase substantially. More frequent QC and more stringent acceptance criteria will allow the laboratory to identify when small changes in performance occur so they can be corrected to minimize the risk of harm to a patient from erroneous results being acted on to make clinical care decisions.

A second limitation of QC materials is gradual deterioration of the analyte during storage and after reconstitution, thawing, or vial opening.

The conventional way to express QC interpretive rules is by using an abbreviation nomenclature popularized among clinical laboratories by Westgard and summarized in Table 7.1. Note that fractional SDIs can be used as in the 2_2.5S and 8_1.5S examples and that combinations of numbers of controls and limits can be used as appropriate for QC interpretive rules. Trend detection procedures such as cumulative sum (CUSUM) or exponentially weighted moving average (EWMA) are recommended if supported by an available computer system because they are more powerful for detecting trends than approaches based on counting the number of sequential observations exceeding a specified SDI. A trend rule can be set to give an alert as a warning that may not require discontinuing testing but indicate that a problem is developing that should be investigated.

In practice, empirical judgment is frequently used to establish acceptance criteria (rules) to evaluate QC results based on data acquired over a long enough period of time to adequately estimate the expected variability when a measurement procedure is working correctly. An empirical approach can be used by obtaining a set of QC data that represents stable measurement procedure performance over a time interval expected to include most sources of variability. Using those data, the false-alert rate for a rule can be determined, and bias errors of different magnitudes can be added to estimate the ability of a rule, or a combination of rules, to identify that error. Table 7.2 gives an example of an empirically developed multirule for a 6-sigma measurement procedure. Such control rules should allow the laboratory to detect errors before they are of a magnitude that will affect clinical decisions. A 10_x rule was not used because it would have increased the false-alert rate by 10.6%. A 10_x rule or other rule that counts the number of sequential QC results on one side of the target value is not recommended because this condition typically does not indicate a problem with clinical interpretation of patient results when the magnitude of the difference from the target value is small. Counting the number of sequential results that exceed a larger SD from the target value, such as 8_1.5S in this example, is more likely to represent a measurement condition that might need investigation.

For measurement procedures with small sigma values, small deviations from the expected performance need to be identified. Consequently, more stringent QC practices need to be used such as selecting rules such as 1_2S that will give an alert at smaller error conditions, using additional rules in a multirule set, measuring QC more frequently, using more than two QC samples, and not releasing patient results until QC assessment is complete for the time interval during which patient samples were measured. More stringent QC rules will have more false alerts, but this is an unavoidable cost when lower sigma measurement procedures are used.

It is necessary to use patient samples to verify the consistency of results between old and new lots of reagents because of the unpredictability of a matrix-related noncommutability bias being present for QC materials. Fig. 7.8 presents a protocol to verify or adjust QC material target values after a reagent lot change. A group of patient samples and the QC samples are measured using both the current (old) and new reagent lots. The first step is to verify that results for a group of patient samples measured with the new reagent lot are consistent with results from the current (old) lot. The patient sample results, not the QC results, provide the basis for verifying that the new reagent lot is acceptable for use. If a problem is identified, the calibration of the new reagent lot must be investigated and corrected, or the new reagent lot may be defective and should not be used. When evaluating the patient results, keep in mind that the calibration of the old reagent lot may have drifted and should be verified before concluding that the new reagent lot is not giving acceptable results for the patient samples.

When the results for patients are acceptable, the second step in Fig. 7.8 evaluates results for each QC material to determine if its target value is correct for use with the new lot of reagent(s). If the target value has changed, it must be adjusted to correct for the change in matrix-related noncommutability bias between old and new lots of reagent(s). This adjustment keeps the expected variability centered around the QC target value so that QC interpretive rules will remain valid.

Failure to make a target value adjustment will cause subsequent QC results to be evaluated incorrectly, as illustrated in Fig. 7.9. The shift in target value would cause some of the results shown by blue squares to exceed the old upper QC rule limit, when in reality there is no defect in patient results because the increase in QC results is caused by the matrix-related noncommutability bias with the new reagent lot. Similarly, the increased magnitude of the gap to the old lower QC rule limit will permit a low bias condition, as shown by the blue square points at sequence number 29 and 30, to be undetected. In most cases the SD for a QC material will be the same with any lot of reagent(s).

Note that a reagent lot induced matrix-related noncommutability bias change in the numeric values for the QC results will cause an incorrect increase in the cumulative SD if all results are used for the calculation. For this reason, it is recommended to use the cumulative SD from a single reagent lot or the pooled SD from more than one reagent lot when determining the SD to use for interpreting QC rules.

Experience in clinical laboratories has shown that there are changes, other than reagent lot changes, in measurement procedures that can also affect the QC values but not the results for patient samples. Such changes could be caused by instrument component replacement or other causes. In theory, there should be an assignable cause for such effects, but such a cause is not always identifiable. In practice, any condition that affects QC results but does not affect patient results is treated in the same manner as described for reagent lot changes. The important QC principle is that if the results for patient samples are consistent between the two conditions, then the target value for the QC sample should be adjusted, if necessary, to reflect its value under the new condition. Failure to adjust the QC target value will cause inappropriate acceptability criteria to be used for evaluating the QC results.

Fig. 7.10 is an example of a typical evaluation report sent to a participating laboratory. Each reported result is compared with the mean result for the peer group using the same measurement procedure. The report also includes the SD for the distribution of results in the peer group, the number of laboratories in the peer group, and the SDI (also called a z-score), which expresses the reported result as the number of SDs it is from the mean value (SDI = [result − mean]/SD). The limits of acceptability are shown. Acceptability criteria may be a number of SDs from the mean value, a fixed percent from the mean value, or a fixed concentration from the mean value. For example, in Fig. 7.10, calcium acceptability criteria are ±1 mg/dL (0.25 mmol/L) from the mean value, and iron criteria are ±20% from the mean value.

In Fig. 7.10 the calcium results are in close agreement with the peer group mean (SDI ranges from −0.2 to −1.4), and the laboratory can conclude that its results are consistent with those of others in the peer group using the same measurement procedure and that it is using the measurement procedure according to the manufacturer’s specifications. However, the iron results show greater variability, with one result +3.5 SDI. Although all iron results are within the acceptability criteria, it is recommended to investigate the measurement procedure because a +3.5 SDI is more likely to be different from other participants in the peer group than to be in agreement with them.

Fig. 7.11 shows another type of evaluation report sent to a primary care office for hemoglobin A_1c (HbA_1c) for one of two EQA/PT samples. In this situation the EQA/PT provider is communicating directly with the clinician or the coworker in the general practice office, and the feedback must be easy to understand for nonlaboratory professionals. The EQA/PT result is evaluated as “good,” “acceptable,” or “poor.” The lot numbers of the reagent are registered so that the participant, in case of an aberrant result, can get information if the result was due to the measurement procedure used, the reagent lot used, or the performance of the user.

Fig. 7.12 shows a similar report from the same HbA_1c survey provided to hospital laboratories. In addition to the figures about the distribution of results, information is provided on how different measurement procedures performed, as well as a historical overview of performance on consecutive EQA/PT samples and performance related to the concentration of the sample. The EQA/PT material used for the HbA_1c is pooled fresh patient blood (commutable), and the target value is set by a reference measurement procedure and is therefore the same for all measurement procedures. Each sample was measured in duplicate (as requested by the EQA/PT provider), and the mean of the duplicate was used to estimate bias versus the reference measurement procedure. In the present example the performance was within the acceptability limits but with a generally high bias during the whole period. Because this observation was true for all the instruments using this measurement procedure, the EQA/PT organizer discussed the results with the manufacturer to solve the problem. Until the problem was solved (the manufacturer had to make a new calibrator), the participants were advised by the EQA/PT provider to use a correction factor when reporting their results for patient samples.

If an unacceptable EQA/PT result is identified, the measurement procedure must be investigated for possible causes and the necessary corrective action taken. Even when an EQA/PT result is within acceptability criteria, it is a good laboratory practice to investigate results that are more than approximately 2.5 SDI from the peer group mean. When the SDI is 2.5, there is only a 0.6% probability that the result will be within the expected distribution for the peer group; consequently, the probability is reasonable that a measurement procedure problem may need to be corrected.

Common causes for EQA/PT failure are listed in Box 7.1. Incorrect handling and reporting are unique to EQA/PT events and may not reflect the process used in the laboratory for patient samples. Because the influence of reagent lots on noncommutability related bias is well documented, a reagent lot-specific bias is a possible explanation when no other root cause can be identified.