Problems and risks occurred during uncertainty evaluation of a quantity calculated from correlated parameters: a case study of pH measurement

Parameters of a model describing a measurement process obtained during a calibration experiment allow one to calculate a measurement result, but a simple estimation of measurement uncertainties of the parameters is not sufficient to assess the uncertainty of the result. In this paper, an example of a pH measurement conducted using an ion-selective electrode is presented, in which the uncertainty is evaluated taking into consideration the existing correlation between the parameters of the electrode. The calculations apply either covariances or correlation coefficients that have to be computed additionally. The example presented in this paper illustrates that there are some problems with rounding of variables which, because of the existing very strong correlations, significantly changes the sought uncertainty. This approach is compared with other approaches, that is, usage of uncorrelated variables and Monte Carlo simulations that are described in an earlier work. It is concluded that the approach of uncertainty evaluation, in which covariances or correlation coefficients are explicitly calculated, is work-consuming and may cause significant discrepancies between correct and obtained assessments if some roundings or approximations are done, or if the correlation coefficient is obtained experimentally based on data including random errors.     

Treatment of uncorrected measurement bias in uncertainty estimation for chemical measurements

Consistent treatment of measurement bias, including the question of whether or not to correct for bias, is essential for the comparability of measurement results. The case for correcting for bias is discussed, and it is shown that instances in which bias is known or suspected, but in which a specific correction cannot be justified, are comparatively common. The ISO Guide to the Expression of Uncertainty in Measurement does not provide well for this situation. It is concluded that there is a need for guidance on handling cases of uncorrected bias. Several different published approaches to the treatment of uncorrected bias and its uncertainty are critically reviewed with regard to coverage probability and simplicity of execution. On the basis of current studies, and taking into account testing laboratory needs for a simple and consistent approach with a symmetric uncertainty interval, we conclude that for most cases with large degrees of freedom, linear addition of a bias term adjusted for exact coverage ("U(e)") as described by Synek is to be preferred. This approach does, however, become more complex if degrees of freedom are low. For modest bias and low degrees of freedom, summation of bias, bias uncertainty and observed value uncertainty in quadrature ("RSSu") provides a similar interval and is simpler to adapt to reduced degrees of freedom, at the cost of a more restricted range of application if accurate coverage is desired.     

A comparison in the evaluation of measurement uncertainty in analytical chemistry testing between the use of quality control data and a regression analysis

The evaluation of measurement uncertainties has been widely applied to the calibration of measurement instruments, whereas its application to tests, despite increasing requirements, is a more recent phenomenon. The generalization of the evaluation of measurement uncertainties to tests has been a gradual process, in line with changes in the requirements of the normative framework that regulates the accreditation of tests laboratories and also as the perceived good practices have evolved. The sole identification of the relevant sources of uncertainty was followed by the requirement to provide a simplified estimate of the measurement uncertainty, and it is now an accepted requirement to properly evaluate the expanded measurement uncertainty associated with any tests. In this study, the evaluation of measurement uncertainty associated with the determination of sulfate in water will be attempted using a procedure that includes linear regression, with the regression parameters provided with associated uncertainties, and a Monte Carlo method applied as a validation tool of the conventional mainstream evaluation method, concerning the approximations in terms of linearization of the model and the assumed shape of the output distribution introduced by this approach.     

Evaluation of combined measurement uncertainty of the determination of Mn content in sediment samples

Combined uncertainties of an analysis of elemental content of sediment samples were evaluated. A monitoring system has been designed and implemented for the characterization of the environmental conditions of Lake Balaton in Hungary. Sediments samples were collected and an acidic digestion method was used to determine the concentration of elements. For the calculation of the result of each measurement three different approaches were considered, namely a.) the calculation of the result using a calibration curve and estimating the confidence limit by the Student t-distribution, b.) calculation of the combined uncertainty and c.) estimation of the sampling errors using the transport and field blanks. The latter approach gave the most reliable result since it included all the parameters which had to be considered regarding sampling and sample handling, and measurement. Determination of acid soluble Mn content in sediment samples has been chosen as an example, and the combined uncertainty is calculated using blanks for sampling. Received: 17 March 2000 Accepted: 4 October 2000     

From GUM to alternative methods for measurement uncertainty evaluation

Since the advent of the Guide to the expression of Uncertainty in Measurement (GUM) in 1995 laying the principles of uncertainty evaluation numerous projects have been carried out to develop alternative practical methods that are easier to implement namely when it is impossible to model the measurement process for technical or economical aspects. In this paper, the author presents the recent evolution of measurement uncertainty evaluation methods. The evaluation of measurement uncertainty can be presented according to two axes based on intralaboratory and interlaboratory approaches. The intralaboratory approach includes “the modelling approach” (application of the procedure described in section 8 of the GUM, known as GUM uncertainty framework) and “the single laboratory validation approach”. The interlaboratory approaches are based on collaborative studies and they are respectively named “interlaboratory validation approach” and “proficiency testing approach”.     

Software support for the Nordtest method of measurement uncertainty evaluation

Software support for the Nordtest method of measurement uncertainty evaluation is described. According to the Nordtest approach, the combined measurement uncertainty is broken down into two main components??the within-laboratory reproducibility (intermediate precision) s _Rw and the uncertainty due to possible laboratory bias u(bias). Both of these can be conveniently estimated from validation and quality control data, thus significantly reducing the need for performing dedicated experiments for estimating detailed uncertainty contributions and thereby making uncertainty estimation easier for routine laboratories. An additional merit of this uncertainty estimation approach is that it reduces the danger of underestimating the uncertainty, which continues to be a problem at routine laboratories. The described software tool??MUkit (measurement uncertainty kit)??fully reflects the versatility of the Nordtest approach: it enables estimating the uncertainty components from different types of data, and the data can be imported using a variety of means such as different laboratory data systems and a dedicated web service as well as manual input. Prior to the development of the MUkit software, a laboratory survey was carried out to identify the needs of laboratories related to uncertainty estimation and other quality assurance procedures, as well as their needs for a practical tool for the calculation of measurement uncertainty.     

Draft radioanalytical water and soil performance testing: Uncertainty acceptance criteria derived from historical mixed-analyte performance evaluation program results

Radioanalytical performance testing programs provide radioassay laboratories, regulators and the public performance-based evidence that measurement capabilities are in control. Performance acceptance criteria that combine aspects of measurements difference from a known value and the associated combined uncertainty establishes a quantitative statement of statistical confidence. However, there is need for a reasonable upper limit of the reported measurement uncertainty. Evaluation of thousands of historical U.S. DOE Mixed-Analyte Performance Evaluation Program measurement results for 17 radionuclides in soil and water samples provides predictive expectations for future measurement results and a statistical basis for establishing reasonable upper limits for reported measurement uncertainties for performance evaluation programs.     

From total allowable error via metrological traceability to uncertainty of measurement of the unbiased result

The concept of "total allowable error", investigated by Westgard and co-workers over a quarter of a century for use in laboratory medicine, comprises bias as well as random elements. Yet, to minimize diagnostic misclassifications, it is necessary to have spatio-temporal comparability of results. This requires trueness obtained through metrological traceability based on a calibration hierarchy. Hereby, the result is associated with a final uncertainty of measurement purged of known biases of procedure and laboratory. The sources of bias are discussed and the importance of commutability of calibrators and analytical specificity of the measurement procedure is stressed. The practicability of traceability to various levels and the advantages of the GUM approach for estimating uncertainty are shown.     

Worst case uncertainty estimates for routine instrumental analysis

A methodology for the worst case measurement uncertainty estimation for analytical methods which include an instrumental quantification step, adequate for routine determinations, is presented. Although the methodology presented should be based on a careful evaluation of the analytical method, the resulting daily calculations are very simple. The methodology is based on the estimation of the maximum value for the different sources of uncertainty and requires the definition of limiting values for certain analytical parameters. The simplification of the instrumental quantification uncertainty estimation involves the use of the standard deviation obtained from control charts relating to the concentrations estimated from the calibration curves for control standards at the highest calibration level. Three levels of simplification are suggested, as alternatives to the detailed approach, which can be selected according to the proximity of the sample results to decision limits. These approaches were applied to the determination of pesticide residues in apples (CEN, EN 12393), for which the most simplified approach showed a relative expanded uncertainty of 37.2% for a confidence level of approximately 95%.     

Comparison of different approaches to estimate the uncertainty of a liquid chromatographic assay

A measurement result cannot be properly interpreted without knowledge about its uncertainty. Several concepts to estimate the uncertainty of a measurement result have been developed. Here, four different approaches for uncertainty estimation are compared on the example of the RP-high-performance liquid chromatography (HPLC) assay for tylosin for veterinary use: the guide to the expression of uncertainty in measurement (GUM) approach, which derives the uncertainty of a measurement result by combining the uncertainties related to the uncertainty sources of the measurement process; the top-down approach, which uses the reproducibility estimate from an inter-laboratory study as uncertainty estimate; an approach recently presented by Barwick and Ellison, which combines precision, trueness and robustness data to obtain an uncertainty estimate of the measurement result and finally a further approach, which directly estimates the measurement uncertainty from a robustness test. The comparison shows that the different approaches lead to comparable uncertainty estimates.     

A case study in the practical estimation of measurement uncertainty

This case study is written for analytical laboratories, in order to give support to the implementation of the concept of measurement uncertainty for routine measurements. The aim is to provide a practical, understandable and common way of performing measurement uncertainty calculations, based mainly on pre-existing quality control and validation data. Practical examples taken directly from environmental laboratory monitoring are presented and explained. However, the approach is very general and should be applicable to most testing laboratories in the chemical field. Following the protocol of evaluation illustrated in the case study, it is possible to ensure that most relevant uncertainty components associated with the method are covered. Contributions associated with sampling, homogenisation, sub-sampling, and so on, are, however, excluded.     

Speciation analysis of chromium in drinking water samples by ion-pair reversed-phase HPLC–ICP-MS: validation of the analytical method and evaluation of the uncertainty budget

The approach presented in this article refers to the modification of a method for the detection and quantitative determination of chromium species in water by high-performance liquid chromatography inductively coupled plasma mass spectrometry. The main aim of this work was to establish a detailed validation of the analytical procedure and an estimation of the budget of measurement uncertainty which was helpful in recognizing the critical points of the presented method. As a result of the method validation experiment, the obtained limit of quantification, repeatability and intermediate precision were satisfied for the quantification Cr(III) and Cr(VI) in water matrices. The trueness of the method was verified via an estimation of the recovery of the spiked real samples. The recovery rate of both determined analytes was found to be between 93 and 115 %. Considering that the validation of the method and the evaluation of measurement uncertainty are crucial for quantitative analysis, the above-mentioned assessment of the uncertainty budget was performed in two different ways: a modelling approach and a single-laboratory validation approach. The measurement uncertainties of the results were found to be 4.4 and 7.8 % for Cr(III), 4.2 and 7.9 % for Cr(VI) using the classical concept and method validation data, respectively. This paper is the first publication to presenting all the steps needed to evaluate the measurement uncertainty for the speciation analysis of chromium species. In summary, the obtained results demonstrate that the method can be applied effectively for its intended use.     

Systematic errors in analytical measurement results

Definitions of the concepts of bias and recovery are discussed and approaches to dealing with them described. The Guide To Uncertainty in Measurement (GUM) recommends correction for all significant systematic effects, but it is also possible to expand measurement uncertainty to take account of uncorrected bias. Run, laboratory and method bias can be defined as components of the bias of a particular measurement result, and can be useful as concepts used in method validation. Estimation of run bias allows a simplification of the estimation of measurement uncertainty. Multivariate calibration brings its own biases that must be quantified and minimised.     

Measurement uncertainty as a tool for evaluating the ‘grey zone’ to reduce the false negatives in immunochemical screening of blood donors for infectious diseases

The risk of misclassifying infected individuals as healthy constitutes a crucial challenge when screening blood donors by means of immunoassays. This risk is especially challenging when the numerical results are close to the clinical decision level, i.e. in the ‘grey zone’. The concept of using measurement uncertainty for evaluating the ‘grey zone’ has previously not been systematically applied in this context. This article explains methods, models and empirical (top-down) approaches for the calculation of measurement uncertainty using results from a blood bank according to the internationally accepted GUM principles, focusing on uncertainty sources in the analytical phase. Of the different approaches available, the intralaboratory empirical approaches are emphasised since modelling (bottom-up) approaches are impracticable due to the lack of reliable model equations for immunoassays. Different methods are applied to estimate the measurement uncertainty for the Abbott Prism^® HCV immunoassay. The expanded uncertainty obtained at the clinical decision level from the intralaboratory empirical approach was 36 %. The estimated uncertainty was used to set acceptance and rejection zones following the procedure set in the Eurachem guideline, emphasising the need to minimise the occurrence of false negatives.     

A Monte Carlo approach for estimating measurement uncertainty using standard spreadsheet software

Despite the importance of stating the measurement uncertainty in chemical analysis, concepts are still not widely applied by the broader scientific community. The Guide to the expression of uncertainty in measurement approves the use of both the partial derivative approach and the Monte Carlo approach. There are two limitations to the partial derivative approach. Firstly, it involves the computation of first-order derivatives of each component of the output quantity. This requires some mathematical skills and can be tedious if the mathematical model is complex. Secondly, it is not able to predict the probability distribution of the output quantity accurately if the input quantities are not normally distributed. Knowledge of the probability distribution is essential to determine the coverage interval. The Monte Carlo approach performs random sampling from probability distributions of the input quantities; hence, there is no need to compute first-order derivatives. In addition, it gives the probability density function of the output quantity as the end result, from which the coverage interval can be determined. Here we demonstrate how the Monte Carlo approach can be easily implemented to estimate measurement uncertainty using a standard spreadsheet software program such as Microsoft Excel. It is our aim to provide the analytical community with a tool to estimate measurement uncertainty using software that is already widely available and that is so simple to apply that it can even be used by students with basic computer skills and minimal mathematical knowledge.     

Matrix effects on accurate mass measurements of low-molecular weight compounds using liquid chromatography-electrospray-quadrupole time-of-flight mass spectrometry

Liquid chromatography (LC) with high-resolution mass spectrometry (HRMS) represents a powerful technique for the identification and/or confirmation of small molecules, i.e. drugs, metabolites or contaminants, in different matrices. However, reliability of analyte identification by HRMS is being challenged by the uncertainty that affects the exact mass measurement. This parameter, characterized by accuracy and precision, is influenced by sample matrix and interferent compounds so that questions about how to develop and validate reliable LC-HRMS-based methods are being raised. Experimental approaches for studying the effects of various key factors influencing mass accuracy on low-molecular weight compounds (MW < 150 Da) when using a quadrupole-time-of-flight (QTOF) mass analyzer were described. Biogenic amines in human plasma were considered for the purpose and the effects of peak shape, ion abundance, resolution and data processing on accurate mass measurements of the analytes were evaluated. In addition, the influence of the matrix on the uncertainty associated with their identification and quantitation is discussed. A critical evaluation on the calculation of the limits of detection was carried out, considering the uncertainty associated with exact mass measurement of HRMS-based methods. The minimum concentration level of the analytes that was able to provide a statistical error lower than 5 ppm in terms of precision was 10 times higher than those calculated with S/N = 3, thus suggesting the importance of considering both components of exact mass measurement uncertainty in the evaluation of the limit of detection.     

Monte Carlo simulation for the evaluation of measurement uncertainty of spent fuel analytical results

The present paper describes an approach based on Monte Carlo simulation for the evaluation of uncertainty of nuclear spent fuel analysis. The mathematical model of measurement was established by examining the dissolution process step by step. The results are consistent with those obtained by the classical propagation of variance approach. This paper shows the importance of taking the process into account in order to give a more reliable uncertainty assessment to the result of a concentration ratio of two isotopes in spent fuel. Indeed, for some radionuclides, the uncertainty associated with the upstream steps of the analysis (“process” uncertainty) can represent up to 95 % of the overall uncertainty.     

Comparison of three strategies to evaluate uncertainty from in-house validation data. A case study: mercury determination in sediments

In the present paper, three approaches are compared for the evaluation of the combined uncertainty in the determination of mercury in aquatic sediments by an aqua regia extraction procedure. For this, the data obtained in validation studies from five certified reference materials (CRMs), covering a range of concentrations from 0.8 to 130 mg kg⁻¹ of mercury and analysed by three atomic spectroscopic techniques (cold vapour generation atomic fluorescence spectrometry, CV-AFS, cold vapour generation atomic absorption spectroscopy, and inductively coupled plasma mass spectroscopy), were considered. The combined uncertainty was firstly assessed by considering separately the data obtained for each CRM analysed (approach A). Moreover, this assessment was also performed with two other calculation approaches (B and C) based on the pooled data obtained from the validation step. The comparison of the results obtained for the different techniques showed a clear bias effect when using CV-AFS with nitric acid as a diluent. In relation to the strategies tested for the combined uncertainty assessment, approach C proved to be the easiest and friendliest method for uncertainty assessment.      

Impact of the analytical blank in the uncertainty evaluation of the copper content in waters by flame atomic absorption spectrometry

Chemical analysts use analytical blanks in their analyses, but seldom is this source of uncertainty evaluated. Generally, there is great confusion. Although the numerical value of the blank, in some situations, can be negligible, its source of uncertainty cannot be. This article discusses the uncertainty contribution of the analytical blank using a numerical example of the copper content in waters by flame atomic absorption spectrometry. The results indicate that the uncertainties of the analytical blank can contribute up to 50% when the blank sample is considered in this analysis, confirming its high impact. This effect can be primarily observed where the analyte concentration approaches the lower range of the analytical curve. Even so, the blank is not always computed. Therefore, the relevance of the analytical blank can be confirmed by uncertainty evaluation.     

Measurement uncertainty distributions and uncertainty propagation by the simulation approach

A complete and accurate evaluation of measurement uncertainty requires the knowledge of the uncertainty distributions. The latter are rarely determined or verified experimentally, and hence up to now only crude estimates or assumptions based on intuition have been used. The simulation of experimental results is readily accessible and provides a more relia-ble solution to this problem. When using an appropriate model of measurement and after determina-tion of input value parameters by present state-of-the-art techniques, simulation data supply reliable information about the distribution of the output results of a complex measurement. The method permits simple variation of preposition and therefore ready analysis of various features influencing the measurement of uncertainty intervals. In the paper we described examples of such evaluations related to the preparation of certified reference materials, where there is excellent agreement between the traditional and simulation approaches. And evaluation of more complex measurements of diffusion coefficients by the open capillary method, where uncertainty of the simulated result is more realistic than the re-sult from the traditional error method due to non-linearity and probably Cauchy distribution in some steps. Received: 17 March 2000 Accepted: 25 July 2000