Quantifying the measurement uncertainty of results from environmental analytical methods

The Eurachem–CITAC Guide Quantifying Uncertainty in Analytical Measurement was put into practice in a public laboratory devoted to environmental analytical measurements. In doing so due regard was given to the provisions of ISO 17025 and an attempt was made to base the entire estimation of measurement uncertainty on available data from the literature or from previously performed validation studies. Most environmental analytical procedures laid down in national or international standards are the result of cooperative efforts and put into effect as part of a compromise between all parties involved, public and private, that also encompasses environmental standards and statutory limits. Central to many procedures is the focus on the measurement of environmental effects rather than on individual chemical species. In this situation it is particularly important to understand the measurement process well enough to produce a realistic uncertainty statement. Environmental analytical methods will be examined as far as necessary, but reference will also be made to analytical methods in general and to physical measurement methods where appropriate. This paper describes ways and means of quantifying uncertainty for frequently practised methods of environmental analysis. It will be shown that operationally defined measurands are no obstacle to the estimation process as described in the Eurachem/CITAC Guide if it is accepted that the dominating component of uncertainty comes from the actual practice of the method as a reproducibility standard deviation.     

Measurement uncertainty in the pH measurement procedure

A measured value without even an approximate knowledge of the uncertainty is worthless. The uncertainty is part of every measured value and specification of the uncertainty is part of every analytical procedure. The uncertainty makes the value independent of its origin. The basis for estimation of the uncertainty is the "Guide to the Expression of Uncertainty in Measurement ". For some procedures, however, for example pH measurement, several problems arise in practice. This article describes a practical and inexpensive way of calculating the uncertainty of pH values.     

Reference materials – a view from a small country

ISO/IEC 17025 has an increased emphasis on traceability and estimation of uncertainty of measurement compared with ISO Guide 25. Demonstration of traceability is a new concept in analytical chemistry and depends on access to relevant reference materials or use of reference methods. Until now most reference materials used in New Zealand have been imported, because they offered international comparability. New Zealand is currently starting to develop the required infrastructure so that it will be able to produce unique reference materials that will contribute to the total international effort in improving the reliability of analytical chemistry. Received: 12 October 2000 / Revised: 18 January 2001 / Accepted: 23 January 2001     

A practitioner's approach to the assessment of the quality of sampling, sample preparation, and subsampling

 The methodology of evaluating the performance of sampling, sample preparation, and subsampling is reviewed. The requirements to be set for a successful experiment are revisited. The central role of the reference method is explained, and so is the choice of the parameters and the measurement methods. Based on the principles of the "Guide to the expression of uncertainty in measurement" (GUM), a statistical model is developed that demonstrates the influence of the experimental design on the outcome of the assessment experiment. This relationship is often overlooked in practice, as it is hardly mentioned in written standards dealing with this kind of quality assessments. The statistical framework thus developed covers the statistical procedures commonly appearing in written standards. Finally, the issue of testing the significance of the bias obtained from the experiment is discussed. Received: 14 June 1997 · Accepted: 2 September 1997     

Empirical versus modelling approaches to the estimation of measurement uncertainty caused by primary sampling

Measurement uncertainty is a vital issue within analytical science. There are strong arguments that primary sampling should be considered the first and perhaps the most influential step in the measurement process. Increasingly, analytical laboratories are required to report measurement results to clients together with estimates of the uncertainty. Furthermore, these estimates can be used when pursuing regulation enforcement to decide whether a measured analyte concentration is above a threshold value. With its recognised importance in analytical measurement, the question arises of 'what is the most appropriate method to estimate the measurement uncertainty?'. Two broad methods for uncertainty estimation are identified, the modelling method and the empirical method. In modelling, the estimation of uncertainty involves the identification, quantification and summation (as variances) of each potential source of uncertainty. This approach has been applied to purely analytical systems, but becomes increasingly problematic in identifying all of such sources when it is applied to primary sampling. Applications of this methodology to sampling often utilise long-established theoretical models of sampling and adopt the assumption that a 'correct' sampling protocol will ensure a representative sample. The empirical approach to uncertainty estimation involves replicated measurements from either inter-organisational trials and/or internal method validation and quality control. A more simple method involves duplicating sampling and analysis, by one organisation, for a small proportion of the total number of samples. This has proven to be a suitable alternative to these often expensive and time-consuming trials, in routine surveillance and one-off surveys, especially where heterogeneity is the main source of uncertainty. A case study of aflatoxins in pistachio nuts is used to broadly demonstrate the strengths and weakness of the two methods of uncertainty estimation. The estimate of sampling uncertainty made using the modelling approach (136%, at 68% confidence) is six times larger than that found using the empirical approach (22.5%). The difficulty in establishing reliable estimates for the input variable for the modelling approach is thought to be the main cause of the discrepancy. The empirical approach to uncertainty estimation, with the automatic inclusion of sampling within the uncertainty statement, is recognised as generally the most practical procedure, providing the more reliable estimates. The modelling approach is also shown to have a useful role, especially in choosing strategies to change the sampling uncertainty, when required.     

Uncertainty in analytical results from solid materials with electrothermal atomic absorption spectrometry: a comparison of methods

The combined uncertainty in the analytical results of solid materials for two methods (ET-AAS, analysis after prior sample digestion and direct solid sampling) are derived by applying the Guide to the Expression of Uncertainty in Measurement from the International Standards Organization. For the analysis of solid materials, generally, three uncertainty components must be considered: (i) those in the calibration, (ii) those in the unknown sample measurement and (iii) those in the analytical quality control (AQC) process. The expanded uncertainty limits for the content of cadmium and lead from analytical data of biological samples are calculated with the derived statistical estimates. For both methods the expanded uncertainty intervals are generally of similar width, if all sources of uncertainty are included. The relative uncertainty limits for the determination of cadmium range from 6% to 10%, and for the determination of lead they range from 8% to 16%. However, the different uncertainty components contribute to different degrees. Though with the calibration based on reference solutions (digestion method) the respective contribution may be negligible (precision < 3%), the uncertainty from a calibration based directly on a certified reference material (CRM) (solid sampling) may contribute significantly (precision about 10%). In contrast to that, the required AQC measurement (if the calibration is based on reference solutions) contributes an additional uncertainty component, though for the CRM calibration the AQC is “built-in”. For both methods, the uncertainty in the certified content of the CRM, which is used for AQC, must be considered. The estimation of the uncertainty components is shown to be a suitable tool for the experimental design in order to obtain a small uncertainty in the analytical result.     

Reply to comments on EURACHEM/CITAC guide “Measurement uncertainty arising from sampling”

The Eurachem/CITAC Guide on ‘Measurement uncertainty arising from sampling’ describes a number of methods and approaches that can be used for the estimation of this uncertainty. A recent comment upon this Guide by Wilrich questions the expression of the measurement uncertainty in a form that is relative to the concentration, rather than just as an absolute number (i.e. as relative expanded uncertainty rather than expanded uncertainty), in one of the worked examples. This reply argues that the measurement results from the ‘duplicate’ method cannot reliably distinguish between constant standard deviation and constant relative standard deviation over the range observed in the example and that the most appropriate model must accordingly rely on prior knowledge. Since extensive prior knowledge indicates that the precision of sampling and of chemical analysis both tend to increase as a function of concentration, the body of the Guide recommends expression as a relative standard deviation. It is acknowledged that this assumption should be restated with the results of the worked examples as well as in its current position in the main body of the text, in a future edition of the Guide.     

Plutonium accountability measurements by calorimetry and gamma spectrometry: evaluation of measurement uncertainties and method performance

The calorimetry exchange (CALEX) program is administered by New Brunswick Laboratory (NBL). The main objective of the program is to provide an independent verification of the internal quality control practices in nuclear material safeguards facilities making plutonium accountability measurements by non-destructive calorimetry/gamma spectrometry techniques. Facilities measure the calorimetric power, and plutonium and ²⁴¹Am isotope abundances of CALEX program standards using routine accountability procedures. The measurement results as well as two other quantities (effective specific power and plutonium mass) calculated from these results are evaluated for accuracy (or bias) and precision. In this paper, a limited number of measurement results of a CALEX program standard (identified as Calex I) are evaluated with specific goals to identify a suitable method for uncertainty estimation and to identify the major contributors to the uncertainties. In order to achieve the goals, the Calex I measurement results were evaluated using two different methods: the first method confined to uncertainty estimation from random variations of the measurement results alone, and the second method providing a more comprehensive evaluation of uncertainties from both the measurements and the characterized values of the measured standard according to the Guide to the Expression of Uncertainty in Measurement (GUM). The results of this study, and a subsequent study extended to a larger number of results in the CALEX program database, are expected to provide relevant input for developing the International Target Values for plutonium measurements by the calorimetry/gamma spectrometry method.     

The application of the measurement uncertainty concept to in-process control in pharmaceutical development

 Any analytical data is used to provide information about a sample. The "possible error" of the measurement can be of extreme importance in order to have complete information. The measurement uncertainty concept is a way to achieve quantitative information about this "possible error" using an estimation procedure. On the basis of the analytical result, the chemist makes a decision on the next step of the development process. If the uncertainty is unknown, the information is not complete; therefore this decision might be impossible. The major problem for the in-process control (IPC) procedure is that not only the repeatability but also the intermediate precision (which expresses the variations within laboratories related to different days, different analysts, different equipment, etc.) has to be good enough to make a decision. Unfortunately, the statistical information achieved from one single analytical run only gives information about the repeatability. This paper shows that the estimation of the measurement uncertainty for IPC is a way to solve the problem and gives the necessary information about the quality of the procedure. An example demonstrates that an estimate of uncertainty based on the standard deviations of an analytical method gives a value similar to one based on the standard deviations obtained from a control chart. Therefore, the estimation is both a very useful and also a very cost-effective tool. Though measurement uncertainty cannot replace validation in general, it is a viable alternative to validation for all methods that will never be used routinely. Received: 24 May 1996 Accepted: 10 August 1996     

Worst case uncertainty estimates for routine instrumental analysis

A methodology for the worst case measurement uncertainty estimation for analytical methods which include an instrumental quantification step, adequate for routine determinations, is presented. Although the methodology presented should be based on a careful evaluation of the analytical method, the resulting daily calculations are very simple. The methodology is based on the estimation of the maximum value for the different sources of uncertainty and requires the definition of limiting values for certain analytical parameters. The simplification of the instrumental quantification uncertainty estimation involves the use of the standard deviation obtained from control charts relating to the concentrations estimated from the calibration curves for control standards at the highest calibration level. Three levels of simplification are suggested, as alternatives to the detailed approach, which can be selected according to the proximity of the sample results to decision limits. These approaches were applied to the determination of pesticide residues in apples (CEN, EN 12393), for which the most simplified approach showed a relative expanded uncertainty of 37.2% for a confidence level of approximately 95%.     

Do we need to consider metrological meanings of different measurement uncertainty estimations?

Along the years, several approaches for measurement uncertainty estimation have been suggested. Emphasis has been put on the general metrological interpretation of measurement uncertainty, but not on its different meanings when it is associated to given conditions of measurement where analytical work is performed and errors are originated. Three different definitions for uncertainty are proposed for reproducibility and intermediate precision conditions of measurement. These definitions inherit features from the VIM 3 definition of measurement uncertainty. It is argued that if a high performance laboratory keeps errors under control with proper validation and quality assurance programs, measurement uncertainty from intermediate precision condition of measurement is justified as a suitable estimation of its capability to attribute values to a measurand. Alternatively, a laboratory that does not keep errors under control should use uncertainty from reproducibility condition of measurement as the cost of its imperfections. Selection of information sources for measurement uncertainty estimation should be in harmony with its metrological meaning.     

The Japanese traceability system and the role of uncertainty evaluation in measurement

 The new traceability system of measurement standards based on the Japanese Measurement Law has been established since November 1993. Some reference materials such as metal standard solutions, pH standard solutions and standard gas mixtures are included in the system together with relevant physical quantities. In this system, primary measurement standard instruments or primary reference materials are designated by the regulation for each quantity. For the practical dissemination of each quantity, accreditation of calibration bodies is recognized by the steering committee under the supervision of the government. In the course of assessment of a candidate calibration body, the concepts of ISO/IEC Guide 25 and ISO/IEC Guide 58 are effectively introduced. For the estimation of reliability, the concept of how to introduce the statistical approach is effectively considered. The method of uncertainty evaluation described in the ISO document entitled "Guide to the expression of uncertainty in measurement" is adopted.     

Effect of the sample matrix on measurement uncertainty in X-ray fluorescence analysis

The estimation of measurement uncertainty, with reference to univariate calibration functions, is discussed in detail in the Eurachem Guide “Quantifying Uncertainty in Analytical Measurement”. The adoption of these recommendations to quantitative X-ray fluorescence analysis (XRF) involves basic problems which are above all due to the strong influence of the sample matrix on the analytical response. In XRF-analysis, the proposed recommendations are consequently applicable only to the matrix corrected response. The application is also restricted with regard to both the matrices and analyte concentrations.In this context the present studies are aimed at the problems to predict measurement uncertainty also with reference to more variable sample compositions. The corresponding investigations are focused on the use of the intensity of the Compton scattered tube line as an internal standard to assess the effect of the individual sample matrix on the analytical response relatively to a reference matrix. Based on this concept the estimation of the measurement uncertainty of an analyte presented in an unknown specimen can be predicted in consideration of the data obtained under defined matrix conditions.     

Measurement uncertainty of food carotenoid determination

For consistent interpretation of an analytical method result it is necessary to evaluate the confidence that can be placed in it, in the form of a measurement uncertainty estimate. The Guide to the expression of Uncertainty in Measurement issued by ISO establishes rules for evaluating and expressing uncertainty. Carotenoid determination in food is a complex analytical process involving several mass transfer steps (extraction, evaporation, saponification, etc.), making difficult the application of these guidelines. The ISO guide was interpreted for analytical chemistry by EURACHEM, which includes the possibility of using intra- and interlaboratory information. Measurement uncertainty was estimated based on laboratory validation data, including precision and method performance studies, and also, based on laboratory participation in proficiency tests. These methods of uncertainty estimation were applied to analytical results of different food matrices of fruits and vegetables. Measurement uncertainty of food carotenoid determination was 10–30% of the composition value in the great majority of cases. Higher values were found for measurements near instrumental quantification limits (e.g. 75% for β-cryptoxanthin, and 99% for lutein, in pear) or when sample chromatograms presented interferences with the analyte peak (e.g. 44% for α-carotene in orange). Lower relative expanded measurement uncertainty values (3–13%) were obtained for food matrices/analytes not requiring the saponification step. Based on these results, the saponification step should be avoided if food carotenoids are not present in the ester form. Food carotenoid content should be expressed taking into account the measurement uncertainty; therefore the maximum number of significant figures of a result should be 2.     

A new terminology for the approaches to the quantification of the measurement uncertainty

A new terminology for the approaches to the quantification of the measurement uncertainty is presented, with a view to a better understanding of the available methodologies for the estimation of the measurement quality and differences among them. The knowledge of the merits, disadvantages and differences in the estimation process, of the available approaches, is essential for the production of metrologically correct and fit-to-purpose uncertainty estimations. The presented terminology is based on the level of the analytical information used to estimate the measurement uncertainty (e.g., supralaboratory or intralaboratory information), instead of the direction of information flow (“bottom-up” or “top-down”) towards the level of information where the test is performed, avoiding the use of the same designation for significantly different approaches. The proposed terminology is applied to the approaches considered on 19 examples of the quantification of the measurement uncertainty presented at the Eurachem/CITAC CG4 Guide, Eurolab Technical Report 1/2002 and Nordtest Technical Report 537. Additionally, differences of magnitude in the measurement uncertainty estimated by various approaches are discussed.     

A validation concept for purity determination and assay in the pharmaceutical industry using measurement uncertainty and statistical process control

 The analytical chemists in process development in the pharmaceutical industry have to solve the difficult problem of producing high quality methods for purity determination and assay within a short time without a clear definition of the substance to be analyzed. Therefore the quality management is very difficult. The ideal situation would be that every method is validated before use. This is not possible because this would delay the development process. A process-type quality development approach with an estimation type fast validation (measurement uncertainty) is therefore suggested. The quality management process consists of the estimation of measurement uncertainty for early project status. Statistical process control (SPC) is started directly after measurement uncertainty estimation and a classical validation for the end of the project. By this approach a process is defined that allows a fast and cost-efficient way of supporting the development process with the appropriate quality at the end of the process and provides the transparency needed in the development process. The procedure presented tries to solve the problem of the parallelism between the two development processes (chemical and analytical development) by speeding up the analytical development process initially. Received: 25 March 1997 · Accepted: 17 May 1997     

Estimating uncertainty in analytical procedures based on chromatographic techniques

Chromatographic techniques are very frequently used in analytical procedures for the separation, determination and identification of a wide spectrum of analytes present in samples with complex and sometimes variable matrices. However, the estimation of uncertainty of the final results does not include the uncertainties associated with the actual chromatographic process. In effect, such results cannot always be treated as a reliable source of analytical information. In this paper we present the basic terms, sources of uncertainty, and methods of calculating the combined uncertainty that any presentation of final determinations should include.     

Basic equations and uncertainties in isotope-dilution mass spectrometry for traceability to SI of values obtained by this primary method

A current interest in chemistry concerns traceability of analytical measurements to the International System of Units (SI) and the estimation of their uncertainties in accordance with principles of metrology, that is, measurement science. “Primary methods of measurement” achieve traceability to SI directly without intermediate reference standards or materials and without significant empirical correction factors. Isotope-dilution mass spectrometry should be regarded as such a method. It has the potential of smallest presently achievable uncertainties for analytical measurements directly or for the certification of reference materials including those with abnormal isotopic composition. A simple explanation of the method including its basic equations is given. Full uncertainty estimation is emphasized in terms of these equations. The wider use of concepts of metrology in chemistry is discussed.     

Implementing the GUM in the certification of reference materials: the revision of ISO Guide 35

The availability of certified reference materials, certified in accordance to the GUM is an important tool for the proper estimation of measurement uncertainty in routine analysis. Many CRMs may suffer from incomplete or wrongly estimated uncertainties, mainly due to lack of guidance on how to implement the GUM in the production of CRMs. In particular the inclusion of the impact of inhomogeneity and instability in the uncertainty budget is often missing. The ongoing revision of ISO Guide 35 aims to fill this gap in providing guidance how (batch) inhomogeneity and instability can be translated into measurement uncertainty. The structure of the current ISO Guide 35 has been maintained as far as possible, but major parts underwent revision to become better aligned with GUM and ISO Guide 34 (2000). Received: 9 April 2001 Accepted: 22 October 2001     

Comments to EURACHEM/CITAC guide “Measurement uncertainty arising from sampling”

The EURACHEM/CITAC Guide “Measurement uncertainty arising from sampling” deals with the design and analysis of experiments for the evaluation of the sampling and analytical standard deviation when a defined sampling and analytical method is used for the determination of the concentration, expressed as mass fraction (mg/kg), of an analyte in a specified material. The Guide recommends reporting the relative expanded uncertainty and using it directly, i.e. it implicitly assumes that the standard deviation is proportional to the mass fraction even in case the experimental data do not support this assumption. Example A1 (and some of the other examples of the Guide) demonstrates that this can result in extreme levels of underestimation or overestimation of the uncertainty of measurement results. Hence, such recommendations should be avoided!