The measurement assurance concept in calibration and traceability at NBS/NIST

During the last quarter of the twentieth century, The United States National Bureau of Standards (NBS), later the National Institute of Standards and Technology (NIST), introduced a measurement quality control concept called ”measurement assurance,” and developed measurement assurance programs, or MAPs, for high-level calibration processes. The measurement assurance approach has, over time, become increasingly popular in the metrology community, and in recent years has become well accepted both inside and, to some extent, outside the United States as a rigorous way to ensure the quality of calibrations. The concept has also found application in defining traceability to national standards. This paper traces the history of the measurement assurance concept. Received: 23 October 2000 Accepted: 2 November 2000     

An estimation of the measurement uncertainty for an optical emission spectrometric method

 A validation procedure based on the ISO/IEC 17025 standard was used to demonstrate the long-term stability of a calibration process and to assess the measurement uncertainty of a standard test method for optical emission vacuum spectrometric analysis of carbon and low-alloy steel (ASTM E 415–99a). The validation was used to provide documented evidence that the selected method fulfils the requirements and that the method is ”fit for purpose”. A test for drift was applied to determine statistically whether the analytical results vary systematically with time. The accuracy and traceability of the optimised method were tested by an analysis of closely matched matrix certified reference materials (CRMs). The measurement uncertainty estimations took account of the precision study, the bias and its uncertainty, and the qualification of uncertainties not considered in the overall performance studies. Received: 2 November 2002 Accepted: 2 January 2003 Acknowledgement The author expresses gratitude to Dr. Aleš Fajgelj for helpful discussions during the 3rd Central European Conference on Reference Materials and Measurements. Presented at CERMM-3, Central European Reference Materials and Measurements Conference: The function of reference materials in the measurement process, May 30–June 1, 2002, Rogaška Slatina, Slovenia Correspondence to T. Drglin     

A validation concept for purity determination and assay in the pharmaceutical industry using measurement uncertainty and statistical process control

 The analytical chemists in process development in the pharmaceutical industry have to solve the difficult problem of producing high quality methods for purity determination and assay within a short time without a clear definition of the substance to be analyzed. Therefore the quality management is very difficult. The ideal situation would be that every method is validated before use. This is not possible because this would delay the development process. A process-type quality development approach with an estimation type fast validation (measurement uncertainty) is therefore suggested. The quality management process consists of the estimation of measurement uncertainty for early project status. Statistical process control (SPC) is started directly after measurement uncertainty estimation and a classical validation for the end of the project. By this approach a process is defined that allows a fast and cost-efficient way of supporting the development process with the appropriate quality at the end of the process and provides the transparency needed in the development process. The procedure presented tries to solve the problem of the parallelism between the two development processes (chemical and analytical development) by speeding up the analytical development process initially. Received: 25 March 1997 · Accepted: 17 May 1997     

From GUM to alternative methods for measurement uncertainty evaluation

Since the advent of the Guide to the expression of Uncertainty in Measurement (GUM) in 1995 laying the principles of uncertainty evaluation numerous projects have been carried out to develop alternative practical methods that are easier to implement namely when it is impossible to model the measurement process for technical or economical aspects. In this paper, the author presents the recent evolution of measurement uncertainty evaluation methods. The evaluation of measurement uncertainty can be presented according to two axes based on intralaboratory and interlaboratory approaches. The intralaboratory approach includes “the modelling approach” (application of the procedure described in section 8 of the GUM, known as GUM uncertainty framework) and “the single laboratory validation approach”. The interlaboratory approaches are based on collaborative studies and they are respectively named “interlaboratory validation approach” and “proficiency testing approach”.     

Development of a reference measurement procedure for the determination of methylmercury in fish products

The development of an analytical procedure for speciation analysis of methylmercury in fish products is presented. The method is based on high-performance liquid chromatography hyphenated to inductively coupled plasma-mass spectrometry. The metrological approach is stressed out in this paper, in order to provide reliable and comparable results. A complete uncertainty budget has been evaluated and the method has been validated by the use of a certified reference material. Moreover, the detection could rely on the isotope dilution mass spectrometry, a powerful strategy capable of highly accurate results traceable to the “Système International d’Unités” and recognised by the “Comité Consultatif pour la Quantité de Matière” as a primary method of measurement. Presented at MEFNM 2008, September 2008, Budapest, Hungary.     

Quality assurance in analytical measurement

 The peculiarities of analytical measurement require to check characteristics of the error (its components) of the obtained analysis results to assure the quality of the measurements. This article deals with the various quality assurance procedures and algorithms which are used to check the quality indices, i.e. the accuracy, reproducibility, certainty and repeatability of analytical measurements: These procedures include: laboratory rapid control; Intra-laboratory statistical control (statistical selection control by alternative attribute, statistical selection control by quantity method of periodic check of the analysis procedure for conformity to the specified requirements) and external control (inter-laboratory control checks, inter-laboratory comparison tests, and intra-laboratory control algorithms carried out by the appropriate supervisory body.) in the separately taken laboratory. The respective algorithms, control plans and control requirements, specified according to the different control aims and assurance tasks, enable the quality and certianty of analytical information obtained in laboratories in Russia to be assured. Received: 9 November 1998 / Accepted: 24 November 1998     

Metrology in laboratory medicine – A necessity

 In a recent Letter to the Editor, Dr. B. Neidhart questioned the need to make clinical chemical measurement results traceable to international standards and to incorporate the principles of analytical quality assurance into clinical chemistry. An analysis of the arguments presented shows that modern laboratory medicine has to deliver accurate results, which are comparable over space and time, in order to improve the accuracy of diagnostic tools and minimize cost. The means to achieve accuracy and comparability are metrological traceability and quality assurance as supported by many national and international initiatives.     

Comparability of measurement results for pesticide residues in foodstuffs: an open issue?

The dispersion of results from proficiency tests for the analysis of pesticide residues in foodstuffs suggests that improvements in the compatibility of measurement results are needed. Currently observed divergences can make the evaluation conclusion on foodstuffs compliance with certain legislation dependent on the consulted laboratory. This work discusses the origin and impact of this lack of compatibility, following the metrological concepts presented at the latest version of the “International Vocabulary of Metrology” (VIM3), thus allowing for a clear diagnostic of the problem. The reporting of results from different measurement methods uncorrected for the observed analyte recovery makes them traceable to different “operationally defined measurement procedures” (VIM3) and, therefore, not comparable. When results from different measurement methods are reported corrected for analyte recovery, R, and R is different for spiked and incurred residues, measurement results may be not compatible if this effect is not considered on the uncertainty budget. This discussion is illustrated with metrological models for any possible combination of “measurement performance” and “maximum residue level”. These models are complemented with experimental data of the analysis of pesticide residues in a sample of ginseng powder from a proficiency test. The adopted experimental design allowed the identification of additional threats to metrological compatibility in this field. Solutions to the faced problem are discussed for practicability and impact on regulatory issues. The use of a universal “reference measurement procedure” proves to be the most feasible way of ensuring comparability of measurements in this field.     

Evaluation of measurement uncertainty for analytical procedures using a linear calibration function

In the EURACHEM/CITAC draft ”Quantifying uncertainty in analytical measurement” estimations of measurement uncertainty in analytical results for linear calibration are given. In this work these estimations are compared, i.e. the uncertainty deduced from repeated observations of the sample vs. the uncertainty deduced from the standard residual deviation of the regression. As a result of this study it is shown that an uncertainty estimation based on repeated observations can give more realistic values if the condition of variance homogeneity is not correctly fulfilled in the calibration range. The complete calculation of measurement uncertainty including assessment of trueness is represented by an example concerning the determination of zinc in sediment samples using ICP-atomic emission spectrometry. Received: 9 February 2002 Accepted: 17 April 2002     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Warning: false sense of certainty from the illusion of accuracy and precision in measurements

The illusion of accuracy and precision in testing is often associated with overconfident safety checks before catastrophes: National Aeronautics and Space Administration (NASA) “no safety-of-flight issues,” Deepwater Horizon site in Gulf Mexico reported as “industry model for safety” a year before its explosion, and Fukushima nuclear plant safety test found “satisfactory” at seismic magnitude viewed highest possible. In health care, testing overuse and misuse play significant roles in overconfidence and mistakes. Gaps between systems are most susceptible to errors, such as in miscommunications between the laboratory and the clinic. A safety culture is characterized by ability to communicate, anticipation of misunderstanding and resilient learning from failures. The discussion will show how messages from the laboratory may reduce the false sense of certainty from a test result; how results of tests are meaningless without pre-test estimates; how multiplicity of routine tests increases chances of false positivity; and how reporting measurement limitations can educate all users, including laboratory technicians, clinicians and patients, about the importance of quality assurance and about the pervasive nature of uncertainty beyond analytical performance. In conclusion, an illusion of accuracy and precision in testing evokes among clinicians a false sense of certainty leading to overuse and misuse of tests. Messages from the laboratory may reduce overconfidence from test results and lead to safer use of testing.     

The application of the measurement uncertainty concept to in-process control in pharmaceutical development

 Any analytical data is used to provide information about a sample. The "possible error" of the measurement can be of extreme importance in order to have complete information. The measurement uncertainty concept is a way to achieve quantitative information about this "possible error" using an estimation procedure. On the basis of the analytical result, the chemist makes a decision on the next step of the development process. If the uncertainty is unknown, the information is not complete; therefore this decision might be impossible. The major problem for the in-process control (IPC) procedure is that not only the repeatability but also the intermediate precision (which expresses the variations within laboratories related to different days, different analysts, different equipment, etc.) has to be good enough to make a decision. Unfortunately, the statistical information achieved from one single analytical run only gives information about the repeatability. This paper shows that the estimation of the measurement uncertainty for IPC is a way to solve the problem and gives the necessary information about the quality of the procedure. An example demonstrates that an estimate of uncertainty based on the standard deviations of an analytical method gives a value similar to one based on the standard deviations obtained from a control chart. Therefore, the estimation is both a very useful and also a very cost-effective tool. Though measurement uncertainty cannot replace validation in general, it is a viable alternative to validation for all methods that will never be used routinely. Received: 24 May 1996 Accepted: 10 August 1996     

A quality systems approach for identifying and controlling sources of error with point of care testing devices

Historically, due to the size and nature of the instrumentation, highly skilled laboratory professionals performed clinical testing in centralized laboratories. Today’s clinicians demand realtime test data at the point of care. This has led to a new generation of compact, portable instruments permitting ”laboratory” testing to be performed at or near the patient’s bedside by nonlaboratory workers who are unfamiliar with testing practices. Poorly controlled testing processes leading to poor quality test results are an insidious problem facing point of care testing today. Manufacturers are addressing this issue through instrument design. Providers of clinical test results, regardless of location, working with manufacturers and regulators must create and manage complete test systems that eliminate or minimize sources of error. The National Committee for Clinical Laboratory Standards (NCCLS) in its EP18 guideline, ”Quality management for unit-use testing,” has developed a quality management system approach specifically for test devices used for point of care testing (POCT). Simply stated, EP18 utilizes a ”sources of error” matrix to identify and address potential errors that can impact the test result. The key is the quality systems approach where all stakeholders – professionals, manufacturers and regulators – collaboratively seek ways to manage errors and ensure quality. We illustrate the use of one quality systems approach, EP18, as a means to advance the quality of test results at point of care. Received: 26 June, 2002 Accepted: 17 July 2002 Presented at the European Conference on Quality in the Spotlight in Medical Laboratories, 7–9 October 2001, Antwerp, Belgium Abbreviations NCCLS National Committee for Clinical Laboratory Standards (formerly) · POCT point of care testing · QC quality control · HACCP hazard analysis critical control points · CLIA clinical laboratory improvement amendments (of 1988) Correspondence to S. S. Ehrmeyer     

The establishment of a quality evaluation system of reference materials

Reference materials have been applied widely to ensure the traceability, comparability and reliability of measurement results. To achieve this purpose, the quality of reference materials (RMs) themselves is surely an important aspect to be pay attention to. A quality evaluation system of RMs has been established through the project “The National Sharing Platform of Reference Materials” in China to give a reliable assessment on the quality of RMs from various sources including the accuracy and comparability of their property values, which is very useful to promote the appropriate selecting and using of RMs in China. Through the application of National Metrology Institute calibration and measurement capabilities on the basis of the international mutual recognition arrangement, it can also provide a powerful supplement to the current activities such as the accreditation of RM producers in the construction of a global harmonized quality control and assurance system of RMs.     

The role of different soil sample digestion methods on trace elements analysis: a comparison of ICP-MS and INAA measurement results

The measurement of trace-element concentration in soil, sediment and waste, is generally a combination of a digestion procedure for dissolution of elements and a subsequent measurement of the dissolved elements. “Partial” and “total” digestion methods can be used in environmental monitoring activities. To compare measurement results obtained by different methods, it is crucial to determine and to maintain control of the bias of the results obtained by these methods. In this paper, ICP-MS results obtained after matrix digestion with modified aqua regia (HCl+HNO₃+H₂O₂) method and two “total” digestion methods (microwave aqua regia+HF and HNO₃+HF) are compared with those obtained by instrumental neutron activation analysis, a non-destructive analytical method for the determination of the total mass concentrations of inorganic components in environmental matrices. The comparison was carried out on eight agricultural soil samples collected in one test area and measured by k₀-INAA and ICP-MS to determine As, Co, Cr, Sb and Zn mass concentration. The bias of results for As, Cd, Co, Cr, Cu, Ni, Pb, Sb and Zn of the three digestion methods were assessed using selected measurement standards. This paper highlights that the digestion procedure is an integral part of the measurement and can affect the measurement result in environmental analysis.     

On the accuracy of micro Winkler titration procedures: a case study

Accuracy data (expressed as precision and trueness) presented by the authors of three different micro modifications of the Winkler titration procedure for dissolved oxygen concentration determination are critically evaluated. Tentative uncertainty estimates are extracted from the data based on the single-laboratory validation approach (originally published in the Nordtest Technical Report 537) and they lead to expanded uncertainty (k = 2) estimates in the range from 0.13 to 0.27 mg l⁻¹ for the three procedures. It is demonstrated that, in all cases, the authors have presented the accuracy and/or precision estimates of the procedures in a way that can lead to too optimistic conclusions about the uncertainty of their procedures. This case study demonstrates the usefulness and flexibility of the single-laboratory validation approach to uncertainty estimation, even in the case of insufficient data, and can be of interest to laboratory workers dealing with measurement procedures from the literature. It is also expected to be of interest to university instructors of analytical chemistry and metrology in chemistry as a real-life example of the critical evaluation of the literature data. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

A comparison of location estimators for interlaboratory data contaminated with value and uncertainty outliers

While estimation of measurement uncertainty (MU) is increasingly acknowledged as an essential component of the chemical measurement process, there is little agreement on how best to use even nominally well-estimated MU. There are philosophical and practical issues involved in defining what is “best” for a given data set; however, there is remarkably little guidance on how well different MU-using estimators perform with imperfect data. This report characterizes the bias, efficiency, and robustness properties for several commonly used or recently proposed estimators of true location, μ, using “Monte Carlo” (MC) evaluation of “measurement” data sets drawn from well-defined distributions. These synthetic models address a number of issues pertinent to interlaboratory comparisons studies. While the MC results do not provide specific guidance on “which estimator is best” for any given set of real data, they do provide broad insight into the expected relative performance within broadly defined scenarios. Perhaps the broadest and most emphatic guidance from the present study is that (1) well-estimated measurement uncertainties can be used to improve the reliability of location determination and (2) some approaches to using measurement uncertainties are better than others. The traditional inverse squared uncertainty-weighted estimators perform well only in the absence of unrepresentative values (value outliers) or underestimated uncertainties (uncertainty outliers); even modest contamination by such outliers may result in relatively inaccurate estimates. In contrast, some inverse total variance-weighted-estimators and probability density function area-based estimators perform well for all scenarios evaluated, including underestimated uncertainties, extreme value outliers, and asymmetric contamination.     

Multi-capillary column-ion mobility spectrometry: a potential screening system to differentiate virgin olive oils

The potential of a headspace device coupled to multi-capillary column-ion mobility spectrometry has been studied as a screening system to differentiate virgin olive oils (“lampante,” “virgin,” and “extra virgin” olive oil). The last two types are virgin olive oil samples of very similar characteristics, which were very difficult to distinguish with the existing analytical method. The procedure involves the direct introduction of the virgin olive oil sample into a vial, headspace generation, and automatic injection of the volatiles into a gas chromatograph-ion mobility spectrometer. The data obtained after the analysis by duplicate of 98 samples of three different categories of virgin olive oils, were preprocessed and submitted to a detailed chemometric treatment to classify the virgin olive oil samples according to their sensory quality. The same virgin olive oil samples were also analyzed by an expert’s panel to establish their category and use these data as reference values to check the potential of this new screening system. This comparison confirms the potential of the results presented here. The model was able to classify 97% of virgin olive oil samples in their corresponding group. Finally, the chemometric method was validated obtaining a percentage of prediction of 87%. These results provide promising perspectives for the use of ion mobility spectrometry to differentiate virgin olive oil samples according to their quality instead of using the classical analytical procedure.     

Collaborative sampling trial in the context of quality assurance in the German marine monitoring programme for the North Sea and the Baltic Sea

This paper presents the assessment of a collaborative trial in sampling in the Baltic Sea in the framework of quality assurance in the German marine monitoring programme for the North Sea and the Baltic Sea. The objective of investigations was to determine the influence of sampling on analytical results for selected monitoring parameters and to harmonize the procedure for sampling of sea water to a large extent. In these studies the staff of three vessels took replicate sea water samples, 1 m below the surface and below the halocline, at two monitoring stations. Mass concentration mean values for different nutrient parameters were obtained from each sample, all in one laboratory. Data produced from the hierarchical design were treated with robust analysis of variance (ANOVA) to generate uncertainty estimates, as standard uncertainties (“u” expressed as standard deviation), for geochemical variation (s _geochem), primary sampling (s _sampling), and chemical analysis (s _analysis). Geochemical variation dominated the total variance in all cases. Sampling and analytical uncertainties contributed together up to 15% of the total variance and had a relative measurement uncertainty (u%) of less than 2% for all the parameters investigated. Thus for this study the sampling protocol and the analytical method could be regarded as fit-for-purpose. M. Gluschke was formerly affiliated to the Federal Environmental Agency, P.O. Box 33 00 22, 14191 Berlin, Germany.     

The role of certified reference materials in ensuring the quality of spectrochemical measurements – the present status in Romania

 The objective of quality assurance programme for spectrochemical measurements is to reduce the measurement errors to accepted limits. Reference materials are being widely used as measurement standards in the fields of industrial production, environmental protection and clinical chemistry, and are playing an important role in ensuring the quality of measurement results. This paper presents some aspects, practices and examples of the activity of the Reference Materials Laboratory of the National Institute of Metrology, Bucharest, in the field of spectrochemical measurements. An attempt to describe the role and use of reliable certified reference materials to ensure the quality of spectrochemical measurements is presented. A short review of the locally available certified reference materials used in spectrochemical measurements is given. The use of reference materials data in estimating the measurement uncertainty is discussed. An interlaboratory comparison, recently organized in Romania, is also presented as a useful response to the need for quality assurance of spectrochemical results. Received: 20 March 1999 / Accepted: 25 February 2000