Principles of the EURACHEM/CITAC guide “Use of uncertainty information in compliance assessment”

Many, possibly most, analytical measurements are carried out to assess compliance with a specification or a regulation, for example in the control contaminants in food or the detection of performance enhancing substances in sport. When making an assessment of compliance the presence of unavoidable measurement uncertainty introduces the risk of making incorrect decisions, that is of accepting a batch of material which is outside the specification or rejecting one that is within. This often leads to controversy over whether or not the compliance decision is correct. How to make reliable assessment decisions is described in the EURACHEM/CITAC Guide “Use of uncertainty information in compliance assessment”. The key is the use of decision rules that lead to an unambiguous interpretation of the measurement result and its uncertainty. These decision rules need to be designed to ensure that requirements of the specification or regulation are met and that the risk of making an incorrect decision is acceptable. Ideally they should form part of the specification or regulation. Presented at the Measurement Uncertainty Symposium, April 2008, Berlin, Germany.     

Measurement uncertainty in the pH measurement procedure

A measured value without even an approximate knowledge of the uncertainty is worthless. The uncertainty is part of every measured value and specification of the uncertainty is part of every analytical procedure. The uncertainty makes the value independent of its origin. The basis for estimation of the uncertainty is the "Guide to the Expression of Uncertainty in Measurement ". For some procedures, however, for example pH measurement, several problems arise in practice. This article describes a practical and inexpensive way of calculating the uncertainty of pH values.     

Estimating the uncertainty of stability for matrix CRMs.

The new version of ISO Guide 34 requires producers of certified reference materials (CRMs) to include contributions of possible instability to the overall CRM uncertainty, to obtain a value for the uncertainty in compliance with the Guide to the Expression of the Uncertainty in Measurement (GUM). A pragmatic approach to estimating the uncertainty of stability is presented. It relies on regression analysis of stability data with subsequent testing of the slope of the regression line for significance. If the slope is found to be statistically insignificant, a shelf life is chosen and the uncertainty connected with this time is estimated via the standard deviation of the slope. This uncertainty is included in the overall uncertainty of the CRM. This approach is explained with examples showing its applicability to matrix CRMs.     

Implementing the GUM in the certification of reference materials: the revision of ISO Guide 35

The availability of certified reference materials, certified in accordance to the GUM is an important tool for the proper estimation of measurement uncertainty in routine analysis. Many CRMs may suffer from incomplete or wrongly estimated uncertainties, mainly due to lack of guidance on how to implement the GUM in the production of CRMs. In particular the inclusion of the impact of inhomogeneity and instability in the uncertainty budget is often missing. The ongoing revision of ISO Guide 35 aims to fill this gap in providing guidance how (batch) inhomogeneity and instability can be translated into measurement uncertainty. The structure of the current ISO Guide 35 has been maintained as far as possible, but major parts underwent revision to become better aligned with GUM and ISO Guide 34 (2000). Received: 9 April 2001 Accepted: 22 October 2001     

Estimating the uncertainty of stability for matrix CRMs

The new version of ISO Guide 34 requires producers of certified reference materials (CRMs) to include contributions of possible instability to the overall CRM uncertainty, to obtain a value for the uncertainty in compliance with the Guide to the Expression of the Uncertainty in Measurement (GUM). A pragmatic approach to estimating the uncertainty of stability is presented. It relies on regression analysis of stability data with subsequent testing of the slope of the regression line for significance. If the slope is found to be statistically insignificant, a shelf life is chosen and the uncertainty connected with this time is estimated via the standard deviation of the slope. This uncertainty is included in the overall uncertainty of the CRM. This approach is explained with examples showing its applicability to matrix CRMs. Received: 12 October 2000 / Revised: 2 January 2001 / Accepted: 3 January 2001     

Comments to EURACHEM/CITAC guide “Measurement uncertainty arising from sampling”

The EURACHEM/CITAC Guide “Measurement uncertainty arising from sampling” deals with the design and analysis of experiments for the evaluation of the sampling and analytical standard deviation when a defined sampling and analytical method is used for the determination of the concentration, expressed as mass fraction (mg/kg), of an analyte in a specified material. The Guide recommends reporting the relative expanded uncertainty and using it directly, i.e. it implicitly assumes that the standard deviation is proportional to the mass fraction even in case the experimental data do not support this assumption. Example A1 (and some of the other examples of the Guide) demonstrates that this can result in extreme levels of underestimation or overestimation of the uncertainty of measurement results. Hence, such recommendations should be avoided!     

化学计量中不确定度评定研究进展

测量不确定度是表征合理地赋予被测量之值的分散性的参数。本文针对化学计量不确定度评定基础模型仅适用于线性模型、概率分布为正态分布或缩放位移t分布等局限,介绍了近年来不确定度评定的研究热点:蒙特卡罗方法(Monte Carlo Method,MCM),不确定度评定的来源、评定概念、评估方法及其发展过程,扩大了测量不确定度评定与表示的适用范围。     

Evaluating uncertainty in analytical measurements: the pursuit of correctness

 Simple in principle, the evaluation of uncertainty, especially in chemical analysis, is not a routine task and needs great care to be correct. This can be seen, particularly, from an examination of the EURACHEM Guide, Quantifying Uncertainty in Analytical Measurement (1995), which is the most important document on the subject. The examination reveals, in the author's opinion, a shortage of correctness in some principal details of the uncertainty estimation process as presented in worked examples in the Guide, and the author has therefore formulated some "in pursuit of correctness" rules for estimating uncertainty. The rules and respective comments are concerned with the following items: (1) choosing an appropriate distribution function in type B evaluation of uncertainty, (2) the necessity for consideration of separate contributions to the combined uncertainty, and (3) taking account of actual influence factors in the uncertainty estimation process. Furthermore, the problem of estimation of conditional versus overall uncertainty is touched upon in connection with comparative trials where only internal consistency of results is required. Received: 29 January 1998 · Accepted: 10 February 1998     

Reply to comments on EURACHEM/CITAC guide “Measurement uncertainty arising from sampling”

The Eurachem/CITAC Guide on ‘Measurement uncertainty arising from sampling’ describes a number of methods and approaches that can be used for the estimation of this uncertainty. A recent comment upon this Guide by Wilrich questions the expression of the measurement uncertainty in a form that is relative to the concentration, rather than just as an absolute number (i.e. as relative expanded uncertainty rather than expanded uncertainty), in one of the worked examples. This reply argues that the measurement results from the ‘duplicate’ method cannot reliably distinguish between constant standard deviation and constant relative standard deviation over the range observed in the example and that the most appropriate model must accordingly rely on prior knowledge. Since extensive prior knowledge indicates that the precision of sampling and of chemical analysis both tend to increase as a function of concentration, the body of the Guide recommends expression as a relative standard deviation. It is acknowledged that this assumption should be restated with the results of the worked examples as well as in its current position in the main body of the text, in a future edition of the Guide.     

Introduction to measurement uncertainty in chemical analysis

 This paper reviews the experience of the use of the Eurachem Guide and gives a brief overview of the principles of evaluating uncertainty. This is followed by discussion of the comments received on the Guide and highlights some of the issues that need to be considered in the next version. Accepted: 21 October 1997     

Uranium assay determination using Davies and Gray titration: an overview and implementation of GUM for uncertainty evaluation

The International Organization for Standardization (ISO) Guide to the expression of Uncertainty in Measurement (GUM) was developed to meet the demand for a standardized way of evaluating and expressing uncertainties. The Davies and Gray (D&G) titrimetry method is routinely used in nuclear safeguards for uranium accountability measurement and a statement of the uncertainty that can reasonably be attributed to the measured assay value is therefore of importance. A mathematical model for an uncertainty evaluation of D&G measurements in compliance with ISO GUM is presented. This is illustrated by a numerical example and the utilization of the uncertainty budget is explored.     

A new terminology for the approaches to the quantification of the measurement uncertainty

A new terminology for the approaches to the quantification of the measurement uncertainty is presented, with a view to a better understanding of the available methodologies for the estimation of the measurement quality and differences among them. The knowledge of the merits, disadvantages and differences in the estimation process, of the available approaches, is essential for the production of metrologically correct and fit-to-purpose uncertainty estimations. The presented terminology is based on the level of the analytical information used to estimate the measurement uncertainty (e.g., supralaboratory or intralaboratory information), instead of the direction of information flow (“bottom-up” or “top-down”) towards the level of information where the test is performed, avoiding the use of the same designation for significantly different approaches. The proposed terminology is applied to the approaches considered on 19 examples of the quantification of the measurement uncertainty presented at the Eurachem/CITAC CG4 Guide, Eurolab Technical Report 1/2002 and Nordtest Technical Report 537. Additionally, differences of magnitude in the measurement uncertainty estimated by various approaches are discussed.     

The determination of the uncertainty of reference materials certified by laboratory intercomparison

A pragmatic method is proposed for the implementation of the Guide to the expression of uncertainty in measurement in the certification of reference materials by laboratory intercomparison. It is based on the establishment of a full uncertainty budget for each laboratory result and the estimation of the impact of various laboratory standard uncertainties and of between-units variability on the certified reference material (CRM) uncertainty.     

Comparison of ISO-GUM and Monte Carlo methods for the evaluation of measurement uncertainty: application to direct cadmium measurement in water by GFAAS

The propagation stage of uncertainty evaluation, known as the propagation of distributions, is in most cases approached by the GUM (Guide to the Expression of Uncertainty in Measurement) uncertainty framework which is based on the law of propagation of uncertainty assigned to various input quantities and the characterization of the measurand (output quantity) by a Gaussian or a t-distribution. Recently, a Supplement to the ISO-GUM was prepared by the JCGM (Joint Committee for Guides in Metrology). This Guide gives guidance on propagating probability distributions assigned to various input quantities through a numerical simulation (Monte Carlo Method) and determining a probability distribution for the measurand.In the present work the two approaches were used to estimate the uncertainty of the direct determination of cadmium in water by graphite furnace atomic absorption spectrometry (GFAAS). The expanded uncertainty results (at 95% confidence levels) obtained with the GUM Uncertainty Framework and the Monte Carlo Method at the concentration level of 3.01 μg/L were ±0.20 μg/L and ±0.18 μg/L, respectively. Thus, the GUM Uncertainty Framework slightly overestimates the overall uncertainty by 10%. Even after taking into account additional sources of uncertainty that the GUM Uncertainty Framework considers as negligible, the Monte Carlo gives again the same uncertainty result (±0.18 μg/L). The main source of this difference is the approximation used by the GUM Uncertainty Framework in estimating the standard uncertainty of the calibration curve produced by least squares regression. Although the GUM Uncertainty Framework proves to be adequate in this particular case, generally the Monte Carlo Method has features that avoid the assumptions and the limitations of the GUM Uncertainty Framework.     

Reproducibility,complementary measure of predictability for robustness improvement of multivariate calibration models via variable selections

In multivariate calibration with the spectral dataset, variable selection is often applied to identify relevant subset of variables, leading to improved prediction accuracy and easy interpretation of the selected fingerprint regions. Until now, numerous variable selection methods have been proposed, but a proper choice among them is not trivial. Furthermore, in many cases, a set of variables found by those methods might not be robust due to the irreproducibility and uncertainty issues, posing a great challenge in improving the reliability of the variable selection. In this study, the reproducibility of the 5 variable selection methods was investigated quantitatively for evaluating their performance. The reproducibility of variable selection was quantified by using Monte-Carlo sub-sampling (MCS) techniques together with the quantitative similarity measure designed for the highly collinear spectral dataset. The investigation of reproducibility and prediction accuracy of the several variable selection algorithms with two different near-infrared (NIR) datasets illustrated that the different variable selection methods exhibited wide variability in their performance, especially in their capabilities to identify the consistent subset of variables from the spectral datasets. Thus the thorough assessment of the reproducibility together with the predictive accuracy of the identified variables improved the statistical validity and confidence of the selection outcome, which cannot be addressed by the conventional evaluation schemes.     

Uncertainty in chemical analysis for the example of determination of caffeine in coffee

The comprehensive quantifying of uncertainty for the determination of caffeine in coffee is given according to the EURACHEM/CITAC Guide. The cause–effect diagram is very useful for the identification of the sources of uncertainty and the relationship between them. In this case the uncertainty caused by the pipette, the stock solution and the calibration curve are the three main sources. Concerning the concentration of the stock solution the purity of the caffeine was most important. The combined uncertainty showed that none of the three parts (stock solution, pipette, calibration curve) can be neglected. The expanded uncertainty (coverage factor k=2) is about 5.4%.     

Estimating measurement uncertainty in an afternoon. A case study in the practical application of measurement uncertainty

The ISO/IEC 17025:1999 standard requires chemical testing laboratories to have an estimate of the uncertainty of their measurements. This is a new requirement for many laboratories and there is confusion as to how to estimate uncertainty. Concerns have been raised about the time and effort required to obtain uncertainty estimates.Uncertainty budgets were prepared for the measurement of benzene, toluene, ethyl benzene and xylenes (BTEX) in water using purge and trap coupled with GC/MS. A time limit of one working afternoon (2 pm–5.30 pm) was imposed for preparing the uncertainty estimate. Details of the uncertainty estimate for toluene are described.The method in question had been in routine use for several years and the laboratory held third party (NATA) accreditation for the test. Consequently a considerable amount of performance data was readily available. Relevant information was extracted from the documented test method, validation data, instrument calibration and from routine quality control. This data was combined according to the principles of the ISO Guide to the Expression of Uncertainty, as promulgated in the Eurachem document "Quantifying Uncertainty in Analytical Measurement."The uncertainty estimates were compared to estimates obtained from generalised empirical models (the Horwitz and Lowthian equations), and from interlaboratory studies of this analysis.A traceability chain from routine measurements to the SI units of metre, kilogram and mole is described.Realistic and useful uncertainty estimates were obtained with a few hours work using data extant in the laboratory.