Total error and uncertainty: Friends or foes?

Guidelines ISO 17025 and ISO 15189 aim to improve the quality-assurance scheme of laboratories. Reliable analytical results are of central importance due to the critical decisions that are taken with them. ISO 17025 and ISO 15189 therefore require that analytical methods be validated and that laboratories can routinely provide the measurement uncertainty of the results of measurements. To evaluate the fitness of purpose of analytical methods, total error is increasingly applied to assess the reliability of results generated by analytical methods. However, the ISO requirement to estimate measurement uncertainty seems opposed to the concept of total error, leading to delays in laboratories implementing ISO 17025 and ISO 15189 and confusion for the analysts. This article therefore aims to clarify the divergences between total error and measurement uncertainty, but also to discuss their main similarities and emphasize their implementation.     

Software support for the Nordtest method of measurement uncertainty evaluation

Software support for the Nordtest method of measurement uncertainty evaluation is described. According to the Nordtest approach, the combined measurement uncertainty is broken down into two main components??the within-laboratory reproducibility (intermediate precision) s _Rw and the uncertainty due to possible laboratory bias u(bias). Both of these can be conveniently estimated from validation and quality control data, thus significantly reducing the need for performing dedicated experiments for estimating detailed uncertainty contributions and thereby making uncertainty estimation easier for routine laboratories. An additional merit of this uncertainty estimation approach is that it reduces the danger of underestimating the uncertainty, which continues to be a problem at routine laboratories. The described software tool??MUkit (measurement uncertainty kit)??fully reflects the versatility of the Nordtest approach: it enables estimating the uncertainty components from different types of data, and the data can be imported using a variety of means such as different laboratory data systems and a dedicated web service as well as manual input. Prior to the development of the MUkit software, a laboratory survey was carried out to identify the needs of laboratories related to uncertainty estimation and other quality assurance procedures, as well as their needs for a practical tool for the calculation of measurement uncertainty.     

Do we need to consider metrological meanings of different measurement uncertainty estimations?

Along the years, several approaches for measurement uncertainty estimation have been suggested. Emphasis has been put on the general metrological interpretation of measurement uncertainty, but not on its different meanings when it is associated to given conditions of measurement where analytical work is performed and errors are originated. Three different definitions for uncertainty are proposed for reproducibility and intermediate precision conditions of measurement. These definitions inherit features from the VIM 3 definition of measurement uncertainty. It is argued that if a high performance laboratory keeps errors under control with proper validation and quality assurance programs, measurement uncertainty from intermediate precision condition of measurement is justified as a suitable estimation of its capability to attribute values to a measurand. Alternatively, a laboratory that does not keep errors under control should use uncertainty from reproducibility condition of measurement as the cost of its imperfections. Selection of information sources for measurement uncertainty estimation should be in harmony with its metrological meaning.     

A case study in the practical estimation of measurement uncertainty

This case study is written for analytical laboratories, in order to give support to the implementation of the concept of measurement uncertainty for routine measurements. The aim is to provide a practical, understandable and common way of performing measurement uncertainty calculations, based mainly on pre-existing quality control and validation data. Practical examples taken directly from environmental laboratory monitoring are presented and explained. However, the approach is very general and should be applicable to most testing laboratories in the chemical field. Following the protocol of evaluation illustrated in the case study, it is possible to ensure that most relevant uncertainty components associated with the method are covered. Contributions associated with sampling, homogenisation, sub-sampling, and so on, are, however, excluded.     

Different approaches to legal requirements on validation of test methods for official control of foodstuffs and of veterinary drug residues and element contaminants in the EC

 In order to ensure food consumer protection as well as to avoid barriers to trade and unnecessary duplications of laboratory tests and to gain mutual recognition of results of analyses, the quality of laboratories and test results has to be guaranteed. For this purpose, the EC Council and the Commission have introducedprovisions – on measures for quality assurance for official laboratories concerning the analyses of foodstuffs on the one hand and animals and fresh meat on the other, – on the validation of test methods to obtain results of sufficient accuracy. This article deals with legal requirements in the European Union on basic principles of laboratory quality assurance for official notification to the EC Commission and on method validation concerning official laboratories. Widespread discussions and activities on measurement uncertainty are in progress, and the European validation standards for official purposes may serve as a basis for world-wide efforts on quality harmonization of analytical results. Although much time has already been spent, definitions and requirements have to be revised and further additions have to be made.     

Reproducibility estimates from single-laboratory ruggedness tests compared with those derived from collaborative trials of six microbiological measurement methods

This work investigated the feasibility of single-laboratory ruggedness experiments to estimate between-laboratory reproducibility. Six microbiological measurement methods that had already been subject to multi-laboratory validation were studied. Ruggedness experiments were designed and executed to determine sensitivity coefficients for factors judged likely to vary between laboratories. These were combined with estimates of factor variation to give reproducibility estimates. The single-laboratory estimates of reproducibility were generally not similar to those generated from multi-laboratory work. The experimental plans were difficult to design and execute, and were only partially successful in producing useful sensitivity coefficients. The authors conclude that ruggedness tests do not offer a single-laboratory alternative to multi-laboratory method validation or a practical approach to the evaluation of measurement uncertainty for microbiological methods. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users. Campden BRI was previously known as the Campden and Chorleywood Food Research Association.     

Measurement uncertainty in microbiology

Testing laboratories wishing to comply with the requirements of ISO/IEC 17025:1999 need to estimate uncertainty of measurement for their quantitative methods. Many microbiological laboratories have had procedures available for monitoring variability in duplicate results generated by laboratory analysts for some time. These procedures, however, do not necessarily include all possible contributions to uncertainty in the calculations. Procedures for estimating microbiological method uncertainty, based on the Poisson distribution, have been published but, at times, the procedures can either underestimate uncertainty or require laboratories to undertake considerable experimental studies and more complex statistical calculations. This paper proposes procedures for estimating uncertainty of measurement in microbiology, whereby routine laboratory quality control data can be analyzed with simple statistical equations. The approaches used in these procedures are also applied to published data and examples, demonstrating that essentially equivalent results can be obtained with these procedures.     

Quantifying the measurement uncertainty of results from environmental analytical methods

The Eurachem–CITAC Guide Quantifying Uncertainty in Analytical Measurement was put into practice in a public laboratory devoted to environmental analytical measurements. In doing so due regard was given to the provisions of ISO 17025 and an attempt was made to base the entire estimation of measurement uncertainty on available data from the literature or from previously performed validation studies. Most environmental analytical procedures laid down in national or international standards are the result of cooperative efforts and put into effect as part of a compromise between all parties involved, public and private, that also encompasses environmental standards and statutory limits. Central to many procedures is the focus on the measurement of environmental effects rather than on individual chemical species. In this situation it is particularly important to understand the measurement process well enough to produce a realistic uncertainty statement. Environmental analytical methods will be examined as far as necessary, but reference will also be made to analytical methods in general and to physical measurement methods where appropriate. This paper describes ways and means of quantifying uncertainty for frequently practised methods of environmental analysis. It will be shown that operationally defined measurands are no obstacle to the estimation process as described in the Eurachem/CITAC Guide if it is accepted that the dominating component of uncertainty comes from the actual practice of the method as a reproducibility standard deviation.     

The application of the measurement uncertainty concept to in-process control in pharmaceutical development

 Any analytical data is used to provide information about a sample. The "possible error" of the measurement can be of extreme importance in order to have complete information. The measurement uncertainty concept is a way to achieve quantitative information about this "possible error" using an estimation procedure. On the basis of the analytical result, the chemist makes a decision on the next step of the development process. If the uncertainty is unknown, the information is not complete; therefore this decision might be impossible. The major problem for the in-process control (IPC) procedure is that not only the repeatability but also the intermediate precision (which expresses the variations within laboratories related to different days, different analysts, different equipment, etc.) has to be good enough to make a decision. Unfortunately, the statistical information achieved from one single analytical run only gives information about the repeatability. This paper shows that the estimation of the measurement uncertainty for IPC is a way to solve the problem and gives the necessary information about the quality of the procedure. An example demonstrates that an estimate of uncertainty based on the standard deviations of an analytical method gives a value similar to one based on the standard deviations obtained from a control chart. Therefore, the estimation is both a very useful and also a very cost-effective tool. Though measurement uncertainty cannot replace validation in general, it is a viable alternative to validation for all methods that will never be used routinely. Received: 24 May 1996 Accepted: 10 August 1996     

Method validation and verification in liquid scintillation counting using the long-term uncertainty method (LTUM) on two decades of proficiency test data

Results from proficiency tests gathered over the past two decades by the laboratory for low level radioactivity measurements for liquid scintillation counting of ³H (184 results) and ¹⁴C (74 results) are used to verify the validated measurement methods used by the laboratory, in particular the estimated uncertainty budget of the method and its reproducibility and stability. A linear regression approach is used for the analysis of the results, described in the literature as the long term uncertainty in measurement method. The present study clearly indicates the advantages of using proficiency test results in identifying possible constant or proportional bias effects as well as the possibility to compare the laboratory performance with the performance of peer laboratories.

     

Uncertainty profiles for the validation of analytical methods

This article aims to expose a new global strategy for the validation of analytical methods and the estimation of measurement uncertainty. Our purpose is to allow to researchers in the field of analytical chemistry get access to a powerful tool for the evaluation of quantitative analytical procedures. Indeed, the proposed strategy facilitates analytical validation by providing a decision tool based on the uncertainty profile and the β-content tolerance interval. Equally important, this approach allows a good estimate of measurement uncertainty by using data validation and without recourse to other additional experiments.In the example below, we confirmed the applicability of this new strategy for the validation of a chromatographic bioanalytical method and the good estimate of the measurement uncertainty without referring to any extra effort and additional experiments. A comparative study with the SFSTP approach [1] showed that both strategies have selected the same calibration functions.The holistic character of the measurement uncertainty compared to the total error was influenced by our choice of profile uncertainty. Nevertheless, we think that the adoption of the uncertainty in the validation stage controls the risk of using the analytical method in routine phase.     

Inter-laboratory comparisons: Determination of actinides in excreta

Inter-laboratory tests are a means of assessing the analytical coherence of medical laboratories. In radiotoxicology, this kind of exercise makes it possible to keep up with laboratory know-how and with the evolution and relative performances of analytical techniques (precision and reproducibility). However, the goal of the laboratories taking part in these annual exercises is not only to check the accuracy of their results. The analytical discussions and the chance to compare experiences enrich the group's general competence.French biologists have been organizing annual radiotoxicology intercomparison exercises since 1978. The exercises are carried out within the framework of a working group (GT1) operating under the aegis of the French Atomic Energy Commission's (CEA) Medical Coordinator. Using reports and diagrams which present the results obtained by the participants in the form of syntheses, the authors describe how the exercises for determining actinides in excreta (urine and faeces) are organized, how the results are evaluated in terms of the analytical methods used, and the improvements made in analytical and metrological performance.Up until 1985, these exercises were limited to French laboratories. Since then, the exercises have acquired an international dimension, opening up to include interested foreign radiochemists, initially from European laboratories, and now from laboratories worldwide. At the present time, 35 laboratories representing 9 countries take part regularly in these intercomparison exercises.     

Quality assurance in analytical measurement

 The peculiarities of analytical measurement require to check characteristics of the error (its components) of the obtained analysis results to assure the quality of the measurements. This article deals with the various quality assurance procedures and algorithms which are used to check the quality indices, i.e. the accuracy, reproducibility, certainty and repeatability of analytical measurements: These procedures include: laboratory rapid control; Intra-laboratory statistical control (statistical selection control by alternative attribute, statistical selection control by quantity method of periodic check of the analysis procedure for conformity to the specified requirements) and external control (inter-laboratory control checks, inter-laboratory comparison tests, and intra-laboratory control algorithms carried out by the appropriate supervisory body.) in the separately taken laboratory. The respective algorithms, control plans and control requirements, specified according to the different control aims and assurance tasks, enable the quality and certianty of analytical information obtained in laboratories in Russia to be assured. Received: 9 November 1998 / Accepted: 24 November 1998     

A global approach to method validation and measurement uncertainty

The enforcement of legal limits for food safety raises the question of decision-making in the context of uncertain measurements. It also puts the question of demonstrating that measurement technique that is used is fit for the purpose of controlling legal limits. A recent European Commision (EC) decision gives some indications how to deal with this question. In the meantime, the implementation of quality systems in analytical laboratories is now a reality. While these requirements deeply modified the organization of the laboratories, it has also improved the quality of the results. The goal of this communication is to describe how two fundamental requirements of ISO 17025 standard, i.e. validation of the methods and estimation of the uncertainty of measurements, can give a way to check whether an analytical method is correctly fit for the purpose of controlling legal limits. Both these requirements are not independent and it will be shown how they can be combined. A recent approach based on the “accuracy profile” of a method was applied to the determination of acrylamide and illustrates how uncertainty can be simply derived from the data collected for validating the method. Moreover, by basing on the β-expectation tolerance interval introduced by Mee [Technometrics (1984) 26(3): 251–253], it is possible to unambiguously demonstrate the fitness for purpose of a method. Remembering that the expression of uncertainty of the measurement is also a requirement for accredited laboratories, it is shown that the uncertainty can be easily related to the trueness and precision issuing from the data collected to build the method accuracy profile. The example presented here consists in validating a method for the determination of acrylamide in pig plasma by liquid chromatography–mass spectromery (LC–MS). Concentrations are expressed as mg/l and instrumental response is peak surface. The calibration experimental design included 5×5×2 measurements and namely consisted in preparing duplicate standard solutions at five concentration levels ranging from 10 to about 5000 mg/l. This was repeated for 5 days. The validation experimental design was similar.     

Interlaboratory study of a liquid chromatography method for erythromycin: determination of uncertainty

Erythromycin is a mixture of macrolide antibiotics produced by Saccharopolyspora erythreas during fermentation. A new method for the analysis of erythromycin by liquid chromatography has previously been developed. It makes use of an Astec C18 polymeric column. After validation in one laboratory, the method was now validated in an interlaboratory study. Validation studies are commonly used to test the fitness of the analytical method prior to its use for routine quality testing. The data derived in the interlaboratory study can be used to make an uncertainty statement as well. The relationship between validation and uncertainty statement is not clear for many analysts and there is a need to show how the existing data, derived during validation, can be used in practice. Eight laboratories participated in this interlaboratory study. The set-up allowed the determination of the repeatability variance, s(2)r and the between-laboratory variance, s(2)L. Combination of s(2)r and s(2)L results in the reproducibility variance s(2)R. It has been shown how these data can be used in future by a single laboratory that wants to make an uncertainty statement concerning the same analysis.     

Empirical versus modelling approaches to the estimation of measurement uncertainty caused by primary sampling

Measurement uncertainty is a vital issue within analytical science. There are strong arguments that primary sampling should be considered the first and perhaps the most influential step in the measurement process. Increasingly, analytical laboratories are required to report measurement results to clients together with estimates of the uncertainty. Furthermore, these estimates can be used when pursuing regulation enforcement to decide whether a measured analyte concentration is above a threshold value. With its recognised importance in analytical measurement, the question arises of 'what is the most appropriate method to estimate the measurement uncertainty?'. Two broad methods for uncertainty estimation are identified, the modelling method and the empirical method. In modelling, the estimation of uncertainty involves the identification, quantification and summation (as variances) of each potential source of uncertainty. This approach has been applied to purely analytical systems, but becomes increasingly problematic in identifying all of such sources when it is applied to primary sampling. Applications of this methodology to sampling often utilise long-established theoretical models of sampling and adopt the assumption that a 'correct' sampling protocol will ensure a representative sample. The empirical approach to uncertainty estimation involves replicated measurements from either inter-organisational trials and/or internal method validation and quality control. A more simple method involves duplicating sampling and analysis, by one organisation, for a small proportion of the total number of samples. This has proven to be a suitable alternative to these often expensive and time-consuming trials, in routine surveillance and one-off surveys, especially where heterogeneity is the main source of uncertainty. A case study of aflatoxins in pistachio nuts is used to broadly demonstrate the strengths and weakness of the two methods of uncertainty estimation. The estimate of sampling uncertainty made using the modelling approach (136%, at 68% confidence) is six times larger than that found using the empirical approach (22.5%). The difficulty in establishing reliable estimates for the input variable for the modelling approach is thought to be the main cause of the discrepancy. The empirical approach to uncertainty estimation, with the automatic inclusion of sampling within the uncertainty statement, is recognised as generally the most practical procedure, providing the more reliable estimates. The modelling approach is also shown to have a useful role, especially in choosing strategies to change the sampling uncertainty, when required.     

Production of certified reference materials for mycotoxins: IRMM’s view on the assessment of uncertainties

Analytical difficulty and the economic importance of controlling mycotoxin levels in food and feed led the Community Bureau of Reference (BCR) to prepare a series of certified reference materials (CRM) for various mycotoxins. Because of the wide acceptance of these CRM and the need to ensure the comparability and traceability of measurements in the future it is necessary to prepare and certify new batches of mycotoxin reference materials (RM). In the following text two different approaches for evaluation of the characterisation uncertainty of CRM will be compared using the certification of aflatoxin M₁ (AfM₁) in milk powder as an example. The conventional approach is based on evaluation of characterisation exercise data; the alternative approach is based on measurement uncertainties of the employed analytical methods. Because laboratories are using totally different approaches to estimate the measurement uncertainties, combination of the uncertainties obtained from the participating laboratories was not recommended. Therefore, a new integrated approach for assessment of the measurement uncertainties of the analytical methods on the basis of additional data collected during the characterisation exercise will be described. The conventional approach was found to be the most appropriate and economical approach to evaluate the characterisation uncertainty as a characterisation exercise must be performed anyway to establish the property values of candidate (C)RM, irrespective of whether or not reliable measurement uncertainties can be provided by the laboratories. An integrated approach for assessment of measurement uncertainties based on additional characterisation data as applied here to enable use of an uncertainty-based approach provides more information but is too time-consuming and cost-intensive to become common practice.     

Uncertainty in quantitative thin-layer chromatography

Globalization forces analysts to demand extended control of variability in analytical measurements. A calculation procedure named the "error budget model" following recommendations proposed more than 20 years ago by the Bureau International des Poids et Mesures is established as a rule for evaluating and expressing the measurement uncertainty across a broad spectrum of measurements. This metrological approach common in physical measurement is not applicable in separation techniques and cannot quantitate measurement uncertainty. Our experiments show that it can be used as a planning tool in the validation of thin-layer chromatographic (TLC) methods. A computer program that quantitates uncertainty components associated with potential sources of uncertainty in quantitative TLC is prepared and tested with experimental data. TLC plates with different qualities of stationary phases (TLC and high-performance TLC) spotted with different types of samples are measured. Application is performed manually and automatically. Plates are scanned with UV-vis scanners and a video documentation system in remission and transmission mode and fluorescence. Although the calculated values are close to the values obtained with validation procedures, the error budget approach cannot substitute validation. Calculated results can predict critical points in real quantitative TLC, but they cannot confirm the validity of a selected chromatographic procedure.     

Method validation and measurement uncertainty evaluation for measurement of mass concentration of organic acids in fermentation broths by using ion chromatography

In the present paper, a methodology for method validation and measurement uncertainty evaluation for the measurement of mass concentration of organic acids in fermentation broths was developed. Acetic acid was selected as a representative of organic acids. A detailed procedure for in-house method validation based on simple experimental design and consistent statistics is presented. In addition, a step-by-step illustration of ??Bottom-Up?? approach for measurement uncertainty evaluation of acetic acid in fermentation broths is also provided. The major sources of uncertainty of the result of measurement were identified and the combined uncertainty was calculated. Our analytical protocol allowed us to quantify acetic acid in fermentation broths in mass concentrations up to 75?g?L^?1 with satisfactory recovery (102.3%) and repeatability lower than 2%. We also estimated within-laboratory reproducibility over 3-month period, which was 2.3%. We proved that the method was selective for the measurement of mass concentration of acetic acid in fermentation broths. Measurement uncertainty of results was evaluated to be 6.2% with 95% confidence level. After validation and measurement uncertainty evaluation steps, results obtained showed that the method can be applied to efficiently monitor fermentation processes.     

Calculation of measurement uncertainty in quantitative analysis of genetically modified organisms using intermediate precision--a practical approach

Quantitative characterization of nucleic acids is becoming a frequently used method in routine analysis of biological samples, one use being the detection of genetically modified organisms (GMOs). Measurement uncertainty is an important factor to be considered in these analyses, especially where precise thresholds are set in regulations. Intermediate precision, defined as a measure between repeatability and reproducibility, is a parameter describing the real situation in laboratories dealing with quantitative aspects of molecular biology methods. In this paper, we describe the top-down approach to calculating measurement uncertainty, using intermediate precision, in routine GMO testing of food and feed samples. We illustrate its practicability in defining compliance of results with regulations. The method described is also applicable to other molecular methods for a variety of laboratory diagnostics where quantitative characterization of nucleic acids is needed.