A new terminology for the approaches to the quantification of the measurement uncertainty

A new terminology for the approaches to the quantification of the measurement uncertainty is presented, with a view to a better understanding of the available methodologies for the estimation of the measurement quality and differences among them. The knowledge of the merits, disadvantages and differences in the estimation process, of the available approaches, is essential for the production of metrologically correct and fit-to-purpose uncertainty estimations. The presented terminology is based on the level of the analytical information used to estimate the measurement uncertainty (e.g., supralaboratory or intralaboratory information), instead of the direction of information flow (“bottom-up” or “top-down”) towards the level of information where the test is performed, avoiding the use of the same designation for significantly different approaches. The proposed terminology is applied to the approaches considered on 19 examples of the quantification of the measurement uncertainty presented at the Eurachem/CITAC CG4 Guide, Eurolab Technical Report 1/2002 and Nordtest Technical Report 537. Additionally, differences of magnitude in the measurement uncertainty estimated by various approaches are discussed.     

A comparison of location estimators for interlaboratory data contaminated with value and uncertainty outliers

While estimation of measurement uncertainty (MU) is increasingly acknowledged as an essential component of the chemical measurement process, there is little agreement on how best to use even nominally well-estimated MU. There are philosophical and practical issues involved in defining what is “best” for a given data set; however, there is remarkably little guidance on how well different MU-using estimators perform with imperfect data. This report characterizes the bias, efficiency, and robustness properties for several commonly used or recently proposed estimators of true location, μ, using “Monte Carlo” (MC) evaluation of “measurement” data sets drawn from well-defined distributions. These synthetic models address a number of issues pertinent to interlaboratory comparisons studies. While the MC results do not provide specific guidance on “which estimator is best” for any given set of real data, they do provide broad insight into the expected relative performance within broadly defined scenarios. Perhaps the broadest and most emphatic guidance from the present study is that (1) well-estimated measurement uncertainties can be used to improve the reliability of location determination and (2) some approaches to using measurement uncertainties are better than others. The traditional inverse squared uncertainty-weighted estimators perform well only in the absence of unrepresentative values (value outliers) or underestimated uncertainties (uncertainty outliers); even modest contamination by such outliers may result in relatively inaccurate estimates. In contrast, some inverse total variance-weighted-estimators and probability density function area-based estimators perform well for all scenarios evaluated, including underestimated uncertainties, extreme value outliers, and asymmetric contamination.     

Measurement uncertainty as a tool for evaluating the ‘grey zone’ to reduce the false negatives in immunochemical screening of blood donors for infectious diseases

The risk of misclassifying infected individuals as healthy constitutes a crucial challenge when screening blood donors by means of immunoassays. This risk is especially challenging when the numerical results are close to the clinical decision level, i.e. in the ‘grey zone’. The concept of using measurement uncertainty for evaluating the ‘grey zone’ has previously not been systematically applied in this context. This article explains methods, models and empirical (top-down) approaches for the calculation of measurement uncertainty using results from a blood bank according to the internationally accepted GUM principles, focusing on uncertainty sources in the analytical phase. Of the different approaches available, the intralaboratory empirical approaches are emphasised since modelling (bottom-up) approaches are impracticable due to the lack of reliable model equations for immunoassays. Different methods are applied to estimate the measurement uncertainty for the Abbott Prism^® HCV immunoassay. The expanded uncertainty obtained at the clinical decision level from the intralaboratory empirical approach was 36 %. The estimated uncertainty was used to set acceptance and rejection zones following the procedure set in the Eurachem guideline, emphasising the need to minimise the occurrence of false negatives.     

Estimation of the uncertainty of mass measurements from in-house calibrated analytical balances

The uncertainty evaluation of mass measurements when using “in-house” calibrated analytical balances is revisited according to the Guide to the expression of Uncertainty Measurement (GUM). The calibration of analytical balances is discussed according to the guidelines of several bodies such as ASTM, UKAS and DKD/PTB. The remainder components of uncertainty can be estimated from the balance data sheet specifications.     

Towards a new edition of the “Guide to the expression of uncertainty in measurement”

The “Guide to the expression of uncertainty in measurement” (GUM) is an extremely important document. It unifies methods for calculating measurement uncertainty and enables the consistent interpretation and comparison of measurement results, regardless of who obtained these measurements and where they were obtained. Since the document was published in 1995, it has been realised that its recommendations do not properly address an important class of measurements, namely, non-linear indirect measurements. This drawback prompted the initiation of the revision of the GUM in the Working Group 1 of the Joint Committee for Guides in Metrology, which commenced in October 2006. The upcoming revision of the GUM provides the metrological community with an opportunity to improve this important document, in particular, to reflect developments in metrology that have occurred since the first GUM publication in 1995. Thus, a discussion of the directions for this revision is important and timely. By identifying several shortcomings of the GUM and proposing directions for its improvement, we hope this article will contribute to this discussion. Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher.     

Uncertainty sources in UV-Vis spectrophotometric measurement

An overview is given of the most important uncertainty sources that affect analytical UV-Vis spectrophotometric measurements. Altogether, eight uncertainty sources are discussed that are expected to have influence in chemical analysis. It is demonstrated that the well-known intrinsic (or “physical”) sources of uncertainty that originate from the instrument itself (repeatability of spectrophotometer reading, spectrophotometer drift, stray light, etc.) often have significantly lower contributions to the combined uncertainty of the result than the “chemical” sources of uncertainty that originate from the object under study (interference from the constituents of the matrix, decomposition of the photometric complex, etc.). Although selectivity of a photometric procedure is often considered more a validation topic than an uncertainty topic, it is very often important to include it also in the uncertainty budget.Usually the most difficult part of uncertainty estimation of a chemical measurement result is to evaluate the magnitude of the actual uncertainty components, especially the chemical ones. For most of the uncertainty sources discussed in this paper, approaches for their evaluation are given. A generic uncertainty budget for absorbance is presented. Electronic Supplementary Material Supplementary material is available for this article at     

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.     

Comparability of measurement results for pesticide residues in foodstuffs: an open issue?

The dispersion of results from proficiency tests for the analysis of pesticide residues in foodstuffs suggests that improvements in the compatibility of measurement results are needed. Currently observed divergences can make the evaluation conclusion on foodstuffs compliance with certain legislation dependent on the consulted laboratory. This work discusses the origin and impact of this lack of compatibility, following the metrological concepts presented at the latest version of the “International Vocabulary of Metrology” (VIM3), thus allowing for a clear diagnostic of the problem. The reporting of results from different measurement methods uncorrected for the observed analyte recovery makes them traceable to different “operationally defined measurement procedures” (VIM3) and, therefore, not comparable. When results from different measurement methods are reported corrected for analyte recovery, R, and R is different for spiked and incurred residues, measurement results may be not compatible if this effect is not considered on the uncertainty budget. This discussion is illustrated with metrological models for any possible combination of “measurement performance” and “maximum residue level”. These models are complemented with experimental data of the analysis of pesticide residues in a sample of ginseng powder from a proficiency test. The adopted experimental design allowed the identification of additional threats to metrological compatibility in this field. Solutions to the faced problem are discussed for practicability and impact on regulatory issues. The use of a universal “reference measurement procedure” proves to be the most feasible way of ensuring comparability of measurements in this field.     

The determination of an accepted reference value from proficiency data with stated uncertainties

Proficiency data with stated uncertainties represent a unique opportunity for testing that the reported uncertainties are consistent with the Guide to the expression of uncertainty in measurement (GUM). In most proficiency tests, however, this opportunity is forfeited, because proficiency data are processed without regard to their uncertainties. In this paper we present alternative approaches for determining a reference value as the weighted mean of all mutually consistent results and their stated uncertainties. Using an accepted reference value each reported uncertainty estimate can be expressed as an E _n number, but a value of confirms its validity only if the uncertainty of the reference value is negligible in comparison.Reference values calculated for results from an International Measurement Evaluation Programme (IMEP-9) by “bottom up” as well as “top down” methods were practically identical, although the first strategy yielded the lowest uncertainty. A plot of individual coefficients of variation (CV) versus E _n numbers helps interpretation of the proficiency data, which could be used to validate relative uncertainties down to <1%.     

Look at data to evaluate uncertainty: a comment on “Uranium assay determination …” by Mathew et al.

In a recent paper, Mathew et al. detailed, for a specific titration-based assay of uranium, a “step-by-step approach to calculate the GUM uncertainty of the measurand”, in which their uncertainty assessment was based solely on prior knowledge, ignoring the manifest variability in their replication data. A simple analysis of the variance from their data reveals that the uncertainty in the average of the replicated quantity (TEF) was at least 3.5 times their estimate. Since the observables that contribute most to the final uncertainty in their method were not replicated, it is unknown whether the estimates for the uncertainties of those quantities, and thus of the output quantity, were also underestimated. This comment demonstrates how a better uncertainty evaluation is possible by extracting as much knowledge as possible from the extant data.     

Multielement trace determination in SiC powders: assessment of interlaboratory comparisons aimed at the validation and standardization of analytical procedures with direct solid sampling based on ETV ICP OES and DC arc OES

The members of the committee NMP 264 “Chemical analysis of non-oxidic raw and basic materials” of the German Standards Institute (DIN) have organized two interlaboratory comparisons for multielement determination of trace elements in silicon carbide (SiC) powders via direct solid sampling methods. One of the interlaboratory comparisons was based on the application of inductively coupled plasma optical emission spectrometry with electrothermal vaporization (ETV ICP OES), and the other on the application of optical emission spectrometry with direct current arc (DC arc OES). The interlaboratory comparisons were organized and performed in the framework of the development of two standards related to “the determination of mass fractions of metallic impurities in powders and grain sizes of ceramic raw and basic materials” by both methods. SiC powders were used as typical examples of this category of material. The aim of the interlaboratory comparisons was to determine the repeatability and reproducibility of both analytical methods to be standardized. This was an important contribution to the practical applicability of both draft standards. Eight laboratories participated in the interlaboratory comparison with ETV ICP OES and nine in the interlaboratory comparison with DC arc OES. Ten analytes were investigated by ETV ICP OES and eleven by DC arc OES. Six different SiC powders were used for the calibration. The mass fractions of their relevant trace elements were determined after wet chemical digestion. All participants followed the analytical requirements described in the draft standards. In the calculation process, three of the calibration materials were used successively as analytical samples. This was managed in the following manner: the material that had just been used as the analytical sample was excluded from the calibration, so the five other materials were used to establish the calibration plot. The results from the interlaboratory comparisons were summarized and used to determine the repeatability and the reproducibility (expressed as standard deviations) of both methods. The calculation was carried out according to the related standard. The results are specified and discussed in this paper, as are the optimized analytical conditions determined and used by the authors of this paper. For both methods, the repeatability relative standard deviations were <25%, usually ~10%, and the reproducibility relative standard deviations were <35%, usually ~15%. These results were regarded as satifactory for both methods intended for rapid analysis of materials for which decomposition is difficult and time-consuming. Also described are some results from an interlaboratory comparison used to certify one of the materials that had been previously used for validation in both interlaboratory comparisons. Thirty laboratories (from eight countries) participated in this interlaboratory comparison for certification. As examples, accepted results are shown from laboratories that used ETV ICP OES or DC arc OES and had performed calibrations by using solutions or oxides, respectively. The certified mass fractions of the certified reference materials were also compared with the mass fractions determined in the interlaboratory comparisons performed within the framework of method standardization. Good agreement was found for most of the analytes.     

Uncertainty calculations in the certification of reference materials. 1. Principles of analysis of variance

 The preparation and certification of reference materials is a rapidly developing area. Many innovative reference materials have limited homogeneity and stability, and, additionally, the uncertainty estimation of the property values must be brought in agreement with the principles of the “Guide to the expression of uncertainty in measurement” (GUM). The results of the homogeneity and stability studies must be included to a certain extent in the uncertainty of the property values of the reference material, in order to comply with these requirements. The basic theory needed to accomplish this is essentially the theory of analysis of variance (ANOVA). As GUM also allows alternative evaluations other than Type A evaluations, a reinterpretation of the theory of ANOVA is necessary to establish a model for the certification of reference materials that is widely applicable. For this, analysis of variance can be used as a statistical technique to derive standard uncertainties from homogeneity, stability and characterisation data. Received: 10 May 2000 / Accepted: 29 July 2000     

Comparison of “bottom-up” and “top-down” strategies for the estimation of the uncertainty in organic elemental analysis

The estimation of uncertainty in organic elemental analysis for C, H, N and S is reported. Both “bottom up” and “top down” strategies are used for uncertainty calculations. The bottom up approach used the results of C, H, N, and S obtained from the homogeneity study of two pure chemicals (toluene-4-sulfonamide and 4(6)-methyl-2-thiouracil). Two calibration systems, K factor and calibration curve, were applied in this study and no significant differences were obtained. For the “top down” approach, we used the data obtained from a proficiency test on both pure chemicals from among 45 Spanish laboratories. Both approaches are compared and discussed below.     

Uncertainty estimation of anions and cations measured by ion chromatography in fine urban ambient particles (PM2.5)

The present work presents a measurement uncertainty evaluation according to Guide to the Expression of Uncertainty in Measurement (GUM) of the concentration of the cations K⁺ and Li⁺ and anions NO₃⁻² and SO₄⁻² in fine airborne particulate matter, refers to particles less than 2.5 μm in diameter (PM_2.5), as measured by ion chromatography (US-EPA 300 method). The GUM method is not typically used to report uncertainty. In general, the analytical results only report the measurement’s standard deviation under repetition as an uncertainty; thus, not all sources of uncertainty are considered. In this work, the major sources of uncertainty regarding the measurements were identified as contributions to linear least square regression lines, repeatability, precision, and trueness. The expanded uncertainty was approximately 20% for anions and cations. The largest contribution to uncertainty was found to be repeatability.     

Measurement uncertainty of food carotenoid determination

For consistent interpretation of an analytical method result it is necessary to evaluate the confidence that can be placed in it, in the form of a measurement uncertainty estimate. The Guide to the expression of Uncertainty in Measurement issued by ISO establishes rules for evaluating and expressing uncertainty. Carotenoid determination in food is a complex analytical process involving several mass transfer steps (extraction, evaporation, saponification, etc.), making difficult the application of these guidelines. The ISO guide was interpreted for analytical chemistry by EURACHEM, which includes the possibility of using intra- and interlaboratory information. Measurement uncertainty was estimated based on laboratory validation data, including precision and method performance studies, and also, based on laboratory participation in proficiency tests. These methods of uncertainty estimation were applied to analytical results of different food matrices of fruits and vegetables. Measurement uncertainty of food carotenoid determination was 10–30% of the composition value in the great majority of cases. Higher values were found for measurements near instrumental quantification limits (e.g. 75% for β-cryptoxanthin, and 99% for lutein, in pear) or when sample chromatograms presented interferences with the analyte peak (e.g. 44% for α-carotene in orange). Lower relative expanded measurement uncertainty values (3–13%) were obtained for food matrices/analytes not requiring the saponification step. Based on these results, the saponification step should be avoided if food carotenoids are not present in the ester form. Food carotenoid content should be expressed taking into account the measurement uncertainty; therefore the maximum number of significant figures of a result should be 2.     

Evaluation of laboratory performance in IMEP water interlaboratory comparisons

The aim of International Evaluation Programme (IMEP) is to present objectively the quality of chemical measurements. Participants in IMEP compare their reported measurement results with independent external certified reference values with demonstrated traceability and uncertainty, as evaluated according to international guidelines. Three major interlaboratory comparisons (ILCs), IMEP-6, IMEP-9 and IMEP-12, on trace elements in water were carried out from 1994 to 2000. Participants' results for Cd, Fe and Pb concentrations from these three different IMEP water ILCs were compared by means of suitable performance indicators. The performance evaluation criteria were set according to the requirements stated in the Council Directive 98/83/EC of 3 November 1998 on the quality of water intended for human consumption. Two different performance indicators were selected for evaluation of the individual participants' results: the commonly used z-score and the not so well-known E _n number. The group performance indicator is based on the variation of z-scores. To assess the individual measurement performance, not only the deviation of the reported measurement values from the certified reference value, but also criteria for maximum and minimum acceptable uncertainties were taken into account. The participants' performance is also reviewed by means of using a simple graphical tool, called “Naji-plots”.     

Metrological traceability is not always a straight line

The two most important concepts in metrology are certainly “traceability to standards” and “measurement uncertainty evaluation”. So far the questions related to these concepts have been reasonably solved in the metrology of “classical quantities”, but for the introduction of metrological concepts in new fields, such as chemistry and biology, a lot of problems remain and must be solved in order to support international arrangements. In this presentation, the authors want to develop the strategy implemented at Laboratoire national de métrologie et d’essais (LNE) in metrology in chemistry and biology. The strategy is based on: (1) pure solutions for calibration of analytical instruments, (2) use of certified reference materials (matrix reference materials), and (3) participation to proficiency testing schemes. Examples will be presented in organic and inorganic chemistry. For laboratory medicine, proficiency testing providers play an important role in the organization of External Quality Assessment Schemes. For the time being, the reference value or the assigned value of the comparison is calculated with the results obtained by the participants. This assigned value is not often traceable to SI units. One of the methods suggested by LNE is to ensure the metrological traceability to SI units of the assigned value for the more critical quantities carried on analytes by implementing the Joint committee for traceability in laboratory medicine reference methods.     

The distribution of interlaboratory comparison data: response to the contribution by K. Heydorn

The distribution of interlaboratory results can often be successfully modeled as a mixture of distributions: a “majority” population of results from laboratories of high and roughly equal competence and a (hopefully) minority “contaminant” population from laboratories of lesser competence. Approximating the majority population as a normal distribution and the contaminant as a uniform distribution can be and has been used to produce mixture populations that have realistically “heavy-tails” (i.e., leptokurtic). Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher.     

Validation and qualification: the fitness for purpose of mass spectrometry-based analytical methods and analytical systems

The context of validation for mass spectrometry (MS)-based methods is critically analysed. The focus is on the fitness for purpose depending on the task of the method. Information is given on commonly accepted procedures for the implementation and acceptance of analytical methods as ‘confirmatory methods’ according to EU criteria, and strategies for measurement. Attention is paid to the problem of matrix effects in the case of liquid chromatography-mass spectrometry-based procedures, since according to recent guidelines for bioanalytical method validations, there is a need to evaluate matrix effects during development and validation of LC-MS methods “to ensure that precision, selectivity and sensitivity will not be compromised”. Beneficial aspects of the qualification process to ensure the suitability of the MS analytical system are also evaluated and discussed.