A quality systems approach for identifying and controlling sources of error with point of care testing devices

Historically, due to the size and nature of the instrumentation, highly skilled laboratory professionals performed clinical testing in centralized laboratories. Today’s clinicians demand realtime test data at the point of care. This has led to a new generation of compact, portable instruments permitting ”laboratory” testing to be performed at or near the patient’s bedside by nonlaboratory workers who are unfamiliar with testing practices. Poorly controlled testing processes leading to poor quality test results are an insidious problem facing point of care testing today. Manufacturers are addressing this issue through instrument design. Providers of clinical test results, regardless of location, working with manufacturers and regulators must create and manage complete test systems that eliminate or minimize sources of error. The National Committee for Clinical Laboratory Standards (NCCLS) in its EP18 guideline, ”Quality management for unit-use testing,” has developed a quality management system approach specifically for test devices used for point of care testing (POCT). Simply stated, EP18 utilizes a ”sources of error” matrix to identify and address potential errors that can impact the test result. The key is the quality systems approach where all stakeholders – professionals, manufacturers and regulators – collaboratively seek ways to manage errors and ensure quality. We illustrate the use of one quality systems approach, EP18, as a means to advance the quality of test results at point of care. Received: 26 June, 2002 Accepted: 17 July 2002 Presented at the European Conference on Quality in the Spotlight in Medical Laboratories, 7–9 October 2001, Antwerp, Belgium Abbreviations NCCLS National Committee for Clinical Laboratory Standards (formerly) · POCT point of care testing · QC quality control · HACCP hazard analysis critical control points · CLIA clinical laboratory improvement amendments (of 1988) Correspondence to S. S. Ehrmeyer     

Is CLIA 2003 the quality alliance envisioned between manufacturers,laboratory professionals,and regulatory agencies?

The Clinical Laboratory Improvement Amendments of 1988 (CLIA88) had a groundbreaking effect on laboratory professionals, instrument and reagent manufacturers, and regulators in the United States, and by association, worldwide. CLIA88 defined new levels of responsibility for all three of these stakeholders in modern laboratory testing. As a result, we envisioned a CLIA-driven approach to regulations, which in effect, created a three-member quality alliance. In 2003, the long awaited CLIA updates to the quality control and quality assurance requirements were published. The revisions in CLIA 2003 will herald an era in which manufactures take responsibility for meeting new standards of quality (improved accuracy and precision or reduced total allowable error) in the clinical laboratory and, especially, at point of care where testing is performed by non-laboratorians. The European (Bureau International des Poids ed Mesures) efforts at traceability, the National Committee of Clinical Laboratory Standards efforts at estimating total allowable error, and the new CLIA 2003 quality systems approach for quality requirements return to the fundamental concepts of accuracy to assess the efficacy of clinical laboratory testing.Presented at the 8th Conference on Quality in the Spotlight, 17–18 March 2003, Antwerp, Belgium     

From GUM to alternative methods for measurement uncertainty evaluation

Since the advent of the Guide to the expression of Uncertainty in Measurement (GUM) in 1995 laying the principles of uncertainty evaluation numerous projects have been carried out to develop alternative practical methods that are easier to implement namely when it is impossible to model the measurement process for technical or economical aspects. In this paper, the author presents the recent evolution of measurement uncertainty evaluation methods. The evaluation of measurement uncertainty can be presented according to two axes based on intralaboratory and interlaboratory approaches. The intralaboratory approach includes “the modelling approach” (application of the procedure described in section 8 of the GUM, known as GUM uncertainty framework) and “the single laboratory validation approach”. The interlaboratory approaches are based on collaborative studies and they are respectively named “interlaboratory validation approach” and “proficiency testing approach”.     

Visualisation of proficiency test exercise results in Kiri plots

A novel graphical method (‘Kiri plots’) for the presentation of proficiency test exercise results is presented. The Kiri plot visualises the evaluation of the proficiency test results based on three statistical tests (the z score, the zeta score and the relative uncertainty outlier test) by defining six zones including a central “in agreement” zone.     

Urine dry reagent strip “error” rates using different reading methods

 The need for “quality” in near patient testing (NPT) has been acknowledged since the mid 1980s. The commonest biochemical NPT device is the dry reagent strip or “dipstick” for urinalysis. Dipsticks may be read in three ways, against the color chart printed along the side of the bottle, using a benchreader (the color chart printed on a flat card) or using an electronic reader. This report uses the results of a urinalysis quality assurance (QA) program, over 1998, to evaluate the “error” rates which occur using the three different reading methods. The QA samples are buffered aqueous solutions which are “spiked” to give concentrations midway between two color blocks for each analyte. Results are scored as ±1 if a color block adjacent to the target value, ±2 for results two color blocks (defined as “error”) and ±3 for results three color blocks (defined as “gross error”) from the target value. Analysis of the results show that the error rates are similar reading visually by either method, but greatly reduced when read electronically. Some persisting errors when using the electronic reader are explained by observation studies. The study highlights the value of a urinalysis QA program for NPT urinalysis in understanding the error rates of this simple but ubiquitous test. Received: 10 July 2000 / Accepted: 10 July 2000     

The role of proficiency testing in the detection and resolution of calibration bias in the LeadCare® blood lead analyzer; limitations of peer-group assessment

Following implementation of the CLIA ‘88 laboratory regulations, the primary role of proficiency testing (external quality assessment, PT) in the U.S. has been widely viewed as one of assuring regulatory compliance. PT can also be an effective tool for detecting widespread analytical problems, subject to limitations based on the method of PT assigned value determination. A recent case study describes the role of two PT programs in detecting and resolving a calibration bias in the LeadCare blood lead analyzer, and illustrates the limitations of peer-group target determination in fulfilling that PT role.     

Warning: false sense of certainty from the illusion of accuracy and precision in measurements

The illusion of accuracy and precision in testing is often associated with overconfident safety checks before catastrophes: National Aeronautics and Space Administration (NASA) “no safety-of-flight issues,” Deepwater Horizon site in Gulf Mexico reported as “industry model for safety” a year before its explosion, and Fukushima nuclear plant safety test found “satisfactory” at seismic magnitude viewed highest possible. In health care, testing overuse and misuse play significant roles in overconfidence and mistakes. Gaps between systems are most susceptible to errors, such as in miscommunications between the laboratory and the clinic. A safety culture is characterized by ability to communicate, anticipation of misunderstanding and resilient learning from failures. The discussion will show how messages from the laboratory may reduce the false sense of certainty from a test result; how results of tests are meaningless without pre-test estimates; how multiplicity of routine tests increases chances of false positivity; and how reporting measurement limitations can educate all users, including laboratory technicians, clinicians and patients, about the importance of quality assurance and about the pervasive nature of uncertainty beyond analytical performance. In conclusion, an illusion of accuracy and precision in testing evokes among clinicians a false sense of certainty leading to overuse and misuse of tests. Messages from the laboratory may reduce overconfidence from test results and lead to safer use of testing.     

Metrology and protometrology: the ordinal question

Laboratory medicine provides results for quantities as well as for properties having no magnitude. The terminology of the latter is less well established and sources are contradictory. Two recent papers on “protometrology” published in this journal offer an opportunity to discuss the necessary concept systems. The delineations of “metrology” versus “protometrology”, “observation” versus “measurement”, and the generic division of “property” are examined with emphasis on avoiding conflict with the International Vocabulary of Metrology. It is suggested that having “examination” as a top generic concept coupled with systematic modifiers for division, especially ‘nominal’ and ‘ordinal’, is a preferable terminological solution.

Metrological traceability is not always a straight line

The two most important concepts in metrology are certainly “traceability to standards” and “measurement uncertainty evaluation”. So far the questions related to these concepts have been reasonably solved in the metrology of “classical quantities”, but for the introduction of metrological concepts in new fields, such as chemistry and biology, a lot of problems remain and must be solved in order to support international arrangements. In this presentation, the authors want to develop the strategy implemented at Laboratoire national de métrologie et d’essais (LNE) in metrology in chemistry and biology. The strategy is based on: (1) pure solutions for calibration of analytical instruments, (2) use of certified reference materials (matrix reference materials), and (3) participation to proficiency testing schemes. Examples will be presented in organic and inorganic chemistry. For laboratory medicine, proficiency testing providers play an important role in the organization of External Quality Assessment Schemes. For the time being, the reference value or the assigned value of the comparison is calculated with the results obtained by the participants. This assigned value is not often traceable to SI units. One of the methods suggested by LNE is to ensure the metrological traceability to SI units of the assigned value for the more critical quantities carried on analytes by implementing the Joint committee for traceability in laboratory medicine reference methods.     

Predictive values instead of normal ranges: less data, more information

Diagnostic strategies can have various goals at two levels: to facilitate the diagnostic process on the cognitive level, and to serve considerations on the level of the doctor–patient relationship. Requests for laboratory tests could be intended to exclude a disease or to affirm the presence of disease. Thirdly, tactical motives to smoothen the negotiations between doctor and patient probably seem to be important as well. These three intentions differ in prior probability, should lead to different sets of tests, and to different interpretations. Even the cut-off points should differ. This leads to three different decision strategies, both at requesting, as at interpreting the results. Following this line of thought, post-test probabilities are more suitable than normal ranges. Excluding strategy: this is the most prevalent. However, the disadvantage of an excluding strategy (prior 1–5%) is a false-positive result. A positive test result should lead to follow-up by wait and see or by repeated testing. More extensive testing usually is not a very sensible strategy. In practice, physicians simply ignore slightly abnormal values. Mentally they put the cut-off points for normality more broader. The number of tests is small. Confirmative strategy: the disadvantage of a confirmative intention (prior 10–30%) is a false-negative result. Follow-up without testing, repeated testing, or even accepting marginal normal results as abnormal is a proper strategy. The number of tests is moderate to high. Tactical strategy: the tactical intention strategy to reassure the patient – or avoid referrals – could lead to ignoring all slightly positive test results by choosing a higher cut-off point. Actually, considering the usual insignificant diagnostic gain when testing for tactical reasons, all test results are clinically insignificant, unsuspected outliers excluded. Here, a very limited set of tests should be chosen. The laboratory test is the currency in mutual trading medical expectations and relationship considerations between doctor and patient. The number of tests is minimal. If the physician chooses a strategy, a limited range of prior probability is chosen. Then a possibly computerized algorithm produces a “Value (posterior probability)” as test result, replacing “Value (normal ranges)”. Thus one number less on the lab form, yielding more significant information.Presented at the 10th Conference Quality in the Spotlight, March 2005, Antwerp, Belgium     

Why do food analysts need quality assurance?

 Most sophisticated products require testing for compliance with specifications and safety regulations before release into many markets, and trade in many simpler commodities and products also requires supporting technical information. Test documentation has become an essential element in this trade. Food intended for human consumption certainly falls into the "sophisticated products" category. Lack of acceptance of laboratory test data across national borders may be a significant barrier to trade. In order to avoid such barriers and unnecessary duplication of laboratory tests, mutual recognition of laboratory results should be regarded as an important means of facilitating international trade in food products. It is difficult to envisage recognition of test data across borders without internationally agreed criteria for assessing the competence of testing. These criteria should, as a minimum, require that a laboratory involved in the analysis of foods operates a suitable quality system. The laboratory must create a quality system appropriate to the type, range and volume of work performed. It is necessary for the elements of this system to be documented in a quality manual which is available for use by the laboratory personnel. The quality manual must be kept up-to-date by a person or persons having responsibility for quality assurance within the laboratory. This paper describes and discusses the elements of a quality system in a food laboratory, including suitable quality assurance measures, the use of validated analytical methods and participation in proficiency testing schemes. Received: 24 February 1996 Accepted: 13 March 1996     

Fission product ratios as treaty monitoring discriminants

The International Monitoring System (IMS) of the Comprehensive Test Ban Treaty Organization (CTBTO) is currently under construction. The IMS is intended for monitoring of nuclear explosions. The radionuclide part of the IMS monitors the atmosphere for short-lived radioisotopes indicative of a nuclear weapon test, and includes field collection and measurement stations, as well as laboratories to provide reanalysis of the most important samples and a quality control function. The Pacific Northwest National Laboratory in Richland, Washington hosts the United States IMS laboratory, with the designation “RL16.” Since acute reactor containment failures and chronic reactor leakage may also produce similar isotopes, it is tempting to compute ratios of detected isotopes to determine the relevance of an event to the treaty or agreement in question. In this paper we will note several shortcomings of simple isotopic ratios: (1) fractionation of different chemical species, (2) difficulty in comparing isotopes within a single element, and (3) the effect of unknown decay times. While these shortcomings will be shown in the light of an aerosol sample, several of the problems extend to xenon isotopic ratios. Due to the difficulties listed above, considerable human expertise will be required to convert a simple mathematical isotope ratio into a criterion which will reliably categorize an event as ‘reactor’ or ‘weapon’.     

From phlogiston to caloric: chemical ontologies

The ‘triumph of the anti-phlogistians’ is a familiar story to the historians and philosophers of science who characterize the Chemical Revolution as a broad conceptual shift. The apparent “incommensurability” of the paradigms across the revolutionary divide has caused much anxiety. Chemists could identify phlogiston and oxygen, however, only with different sets of instrumental practices, theoretical schemes, and philosophical commitments. In addition, the substantive counterpart to phlogiston in the new chemistry was not oxygen, but caloric. By focusing on the changing visions of chemical body across the revolutionary divide with a more sensitive probe into the historical actors’ material manipulations and linguistic usage, we can historicize their laboratory realities and philosophical agenda. An archeology of chemical bodies that configures the fragile stability of the material worlds chemists created in succession promises a philosophical horizon that would recognize our hybrid (natural–artificial) environment as an evolving investigative object of science.     

Quality (r)evolution – guidelines and practical implementation of quality management: the U.S. perspective

 All clinical laboratories in the United States were required for the first time in 1992 to comply with universal minimum regulations: the Clinical Laboratory Improvement Amendments of 1988 (CLIA'88). The CLIA'88 regulations precipitated revolutionary and evolutionary changes based in both total quality management and continuous quality improvement principles. The regulations specify minimum requirements for personnel, quality control, and proficiency testing. Under quality assurance, the requirements are more individualized and allow laboratories to meet the needs of their customers while being in compliance with the regulations. Biannual inspection is integral to CLIA'88. However, laboratories can choose other federally approved, "deemed" professional organizations, with regulations equivalent to CLIA'88. Selection of one of these organizations allows for some flexibility in the application of the requirements, and eliminates having federal CLIA'88 inspectors perform the inspection. Since CLIA's inception in 1992, inspectors have noted a decline in the number of deficiencies cited. An improved level of proficiency testing performance also indicates that better quality test results are being provided to clinicians. The quality evolution and revolution will continue in the United States, and a major impetus will be health care reform's mandate to reduce cost. Received: 13 December 1995 Accepted: January 1996     

Calibration of activity readings from radiopharmaceuticals with impurities

Accurately determining the amount of a radionuclide actually administered to a patient is an important aspect of quality control in nuclear medicine. In this study, in order to accurately assess the “apparent” radioactivities of radionuclides with impurities by a dose calibrator, two methods, (1) curve fitting, and (2) relative sensitivities calculation, have been employed to determine the parameter, M (relative response). From the comparisons between the measurement and re-calculated radioactivity using the M values, each method was found to have its own advantage and disadvantage, depending upon the fraction ratio of the impurity as well as the energy level of the impurity’s photons. This study might be helpful for improving the assessment of radiopharmaceuticals’ “true” activity and its “apparent” activity measured by a dose calibrator.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Validation and qualification: the fitness for purpose of mass spectrometry-based analytical methods and analytical systems

The context of validation for mass spectrometry (MS)-based methods is critically analysed. The focus is on the fitness for purpose depending on the task of the method. Information is given on commonly accepted procedures for the implementation and acceptance of analytical methods as ‘confirmatory methods’ according to EU criteria, and strategies for measurement. Attention is paid to the problem of matrix effects in the case of liquid chromatography-mass spectrometry-based procedures, since according to recent guidelines for bioanalytical method validations, there is a need to evaluate matrix effects during development and validation of LC-MS methods “to ensure that precision, selectivity and sensitivity will not be compromised”. Beneficial aspects of the qualification process to ensure the suitability of the MS analytical system are also evaluated and discussed.     

Comparability of measurement results for pesticide residues in foodstuffs: an open issue?

The dispersion of results from proficiency tests for the analysis of pesticide residues in foodstuffs suggests that improvements in the compatibility of measurement results are needed. Currently observed divergences can make the evaluation conclusion on foodstuffs compliance with certain legislation dependent on the consulted laboratory. This work discusses the origin and impact of this lack of compatibility, following the metrological concepts presented at the latest version of the “International Vocabulary of Metrology” (VIM3), thus allowing for a clear diagnostic of the problem. The reporting of results from different measurement methods uncorrected for the observed analyte recovery makes them traceable to different “operationally defined measurement procedures” (VIM3) and, therefore, not comparable. When results from different measurement methods are reported corrected for analyte recovery, R, and R is different for spiked and incurred residues, measurement results may be not compatible if this effect is not considered on the uncertainty budget. This discussion is illustrated with metrological models for any possible combination of “measurement performance” and “maximum residue level”. These models are complemented with experimental data of the analysis of pesticide residues in a sample of ginseng powder from a proficiency test. The adopted experimental design allowed the identification of additional threats to metrological compatibility in this field. Solutions to the faced problem are discussed for practicability and impact on regulatory issues. The use of a universal “reference measurement procedure” proves to be the most feasible way of ensuring comparability of measurements in this field.     

Clinical quality vs analytical performance: what are the right targets and target values?

For optimal performance of laboratory tests, testing processes should be designed to provide clinically useful quality and QC procedures should be selected to assure that the necessary clinical quality is achieved in routine production. One important issue is how to define clinical quality. Today’s practice guidelines, quality regulations, and quality standards provide some targets for analytical quality, but they fail to adequately address clinical quality. Target values for precision and accuracy are not the same as clinical quality, though analytical performance certainly contributes to the clinical quality of test results. How can we proceed in our quest to improve quality when there are inconsistencies and inadequacies in the specifications found in practice guidelines, regulations, and standards? Today target values are often being set for the wrong targets. A better approach is possible if we focus on test interpretation guidelines to define clinical quality, then derive specifications for the accuracy and precision that are appropriate for the method, as well the QC rules and numbers of control measurements that are necessary to guarantee the desired quality will be achieved in routine operation of the testing process.     

Electrochemical criteria for evaluating conservative treatments applied to contemporary metallic sculpture. A case study

A combination of different electrochemical techniques, namely, voltammetry of microparticles, open circuit potential, potentiodynamic polarization, electrochemical impedance spectroscopy, and electrochemical noise was used for studying a contemporary kinetic sculpture: “I Hábitat en órbita baja de la Tierra” (1 Habitat in low-orbit around the Earth, Elvira Alfageme, 1981). It introduced the concept of ‘electrochemical equivalence’ in order to evaluate alternative conservative treatments consisting of doré and golden patination. Electrochemical information is relevant with regard to aesthetic and ethic problems associated to the conservation/restoration of works of art.