From GUM to alternative methods for measurement uncertainty evaluation

Since the advent of the Guide to the expression of Uncertainty in Measurement (GUM) in 1995 laying the principles of uncertainty evaluation numerous projects have been carried out to develop alternative practical methods that are easier to implement namely when it is impossible to model the measurement process for technical or economical aspects. In this paper, the author presents the recent evolution of measurement uncertainty evaluation methods. The evaluation of measurement uncertainty can be presented according to two axes based on intralaboratory and interlaboratory approaches. The intralaboratory approach includes “the modelling approach” (application of the procedure described in section 8 of the GUM, known as GUM uncertainty framework) and “the single laboratory validation approach”. The interlaboratory approaches are based on collaborative studies and they are respectively named “interlaboratory validation approach” and “proficiency testing approach”.     

A model to set measurement quality objectives and to establish measurement uncertainty expectations in analytical chemistry laboratories using ASTM proficiency test data

A model is presented that correlates historical proficiency test data as the log of interlaboratory standard deviations versus the log of analyte concentrations, independent of analyte (measurand) or matrix. Analytical chemistry laboratories can use this model to set their internal measurement quality objectives and to apply the uncertainty budget process to assign the maximum allowable variation in each major step in their bias-free measurement systems. Laboratories that are compliant with this model are able to pass future proficiency tests and demonstrate competence to laboratory clients and ISO 17025 accreditation bodies. Electronic supplementary material to this paper can be obtained by using the Springer LINK server located at http://dx.doi.org/ 10.1007/s007690100398-y. Received: 31 March 2001 Accepted: 11 September 2001     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working group discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the joint EURACHEM/CITAC/EQALM workshop, Bracknell, UK, 16–18 February 2003 is provided. The nine WGs covered a range of issues concerned with current practice and future directions; PT/EQA as a tool for regulators (WG1); PT/EQA as a tool for accreditation (WG2); evaluation of performance and uncertainty (WG3); frequency of PT/EQA participation (WG4); selection of appropriate PT/EQA schemes (WG5); added value of PT/EQA and cost benefit evaluation (WG6); global harmonisation and rationalisation (WG7); new technical areas and challenges in PT/EQA (WG8); and accreditation of PT/EQA providers (WG9). Participants with different backgrounds were on each WG in order to capture a range of views and experience from different sectors. The discussions reflected on the keynote lectures and built, in many cases, on discussions at previous workshops in 2000 and 2002.     

Revisiting type-A uncertainties relating to the measurement of mass fraction of lead using isotope-dilution inductively coupled plasma mass spectrometry: a way of improving measurement precision and expanded uncertainty

A quadrupole inductively coupled plasma mass spectrometer (Q-ICP-MS) has been used for determination of lead in plant materials using isotope-dilution inductively coupled plasma mass spectrometry. The accuracy of the method was demonstrated by analysis of a matrix certified reference material, NIST SRM 1547 Peach Leaves. Specific instrumental parameters of Q-ICP-MS, including isotope analysis mode, integration time per point, number of points per mass, and number of measurements, were optimized to obtain the best measurement precision. The precision (expressed as relative standard deviation) associated with replicate measurement of the ²⁰⁸Pb/²⁰⁶Pb isotope ratio and its mass-bias correction factor was <0.2%. Following “Example A7” of the Eurachem/CITAC Guide, the relative expanded uncertainty, U _rel, (coverage factor k = 2) was found to be ±1.1%, which fulfilled the target value of ±2% maximum and was lower than the uncertainty of ±3.4% reported by NIST based on isotope-dilution thermal ionization mass spectrometry. Sample recovery of 99% was obtained.     

Comparison of three strategies to evaluate uncertainty from in-house validation data. A case study: mercury determination in sediments

In the present paper, three approaches are compared for the evaluation of the combined uncertainty in the determination of mercury in aquatic sediments by an aqua regia extraction procedure. For this, the data obtained in validation studies from five certified reference materials (CRMs), covering a range of concentrations from 0.8 to 130 mg kg⁻¹ of mercury and analysed by three atomic spectroscopic techniques (cold vapour generation atomic fluorescence spectrometry, CV-AFS, cold vapour generation atomic absorption spectroscopy, and inductively coupled plasma mass spectroscopy), were considered. The combined uncertainty was firstly assessed by considering separately the data obtained for each CRM analysed (approach A). Moreover, this assessment was also performed with two other calculation approaches (B and C) based on the pooled data obtained from the validation step. The comparison of the results obtained for the different techniques showed a clear bias effect when using CV-AFS with nitric acid as a diluent. In relation to the strategies tested for the combined uncertainty assessment, approach C proved to be the easiest and friendliest method for uncertainty assessment.      

Advances for proficiency testing for genetic laboratory science

Nucleic acid based clinical genetic testing has undergone explosive growth in recent years due in large part to the human genome project. Characterization of the human genome has led to a molecular understanding of the pathogenesis of many human diseases, and ultimately to clinical molecular tests becoming routinely used to diagnose a wide diversity of diseases. This rapid growth in clinical molecular genetic testing coupled with the complexity of the analytical procedures underscores the necessity for proficiency testing (i.e. external quality assessment) to allow laboratories offering such services the ability to evaluate their analytical procedures via inter-laboratory comparisons. The American College of Medical Genetics (ACMG) in partnership with the College of American Pathologists (CAP) have been offering proficiency testing for clinical molecular genetics laboratories since 1995, and presently have more than 230 laboratories from 11 countries enrolled in this program. This paper describes the evolution of this program and several challenges encountered in the delivery of a proficiency testing program for laboratories offering clinical molecular genetic services. Received: 13 April 2002 Accepted: 18 July 2002     

Approaching target uncertainty in proficiency testing schemes: experience in the field of water measurement

Good correspondence between evaluations of the proficiency testing data for water measurements according to different scoring schemes was found as a result of implementation of the measurement quality according to the target uncertainty defined in the Drinking Water Directive. The accuracy of the evaluation is determined by the uncertainty of the assigned value, and this uncertainty should be in correspondence with the target uncertainty. Modification of the zeta-score by the thorough application of the target uncertainty is presented. It is convenient for generalised presentation of the proficiency testing data. The allowed bias is included as a linear term on the measured value scale, as it is treated in uncertainty analysis. The importance of implementation of the target measurement uncertainty in other fields of the routine measurements is indicated.     

Analysis of the relationships between proficiency testing,validation of methods and estimation of measurement uncertainty: a qualitative study with experts

Proficiency testing (PT) is an important part in the process ensuring quality of results. Validation of methods and estimation of measurement uncertainty are also vital elements in a laboratory qualification process. The main objective of this paper is to identify the relationships between PT, validation of methods and calculation of measurement uncertainty through the drawing up of a Mind Map. The method proposed used the Web-based Delphi method to conduct a qualitative research with experts of the area (laboratory assessors and accreditation body members). Experts from five accreditation bodies (Brazil, USA, France, Portugal and China) took part in the research. The study led to the drawing up of a Mind Map showing the connections of cause and effect between the factors under study. It emerged that PT effectively contributes to the reliability of results, since it is directly related to the validation of methods and to measurement uncertainty, so as to increase laboratory’s competitiveness.     

Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISO/IEC 17025

ISO/IEC 17025 requests laboratories to estimate the overall uncertainty associated with a test result. In many cases this means a new requirement to laboratories. It is essential to understand the importance of the knowledge of the uncertainty of measurements as well as the principle that not always the smallest possible uncertainty is needed, but fitness for purpose has to be achieved. The paper suggests a policy on the implementation of the concept of uncertainty and guidance on how to proceed, taking into account the present state of the art of understanding the problem.     

Relationship between performance characteristics obtained from an interlaboratory study programme and combined measurement uncertainty: a case study

 In the interlaboratory study programme "ILS Coal Characterisation", eight interlaboratory studies were organised based on the ISO standards for coal analysis. The use of blind samples in each round allows comparability of measurement results between rounds to be assessed. Based on the results, it could be demonstrated that the vast majority of the measurement results of the laboratories were traceable to results obtained in previous rounds of this programme. The hypothesis has been formulated that the combined standard uncertainty obtained from an interlaboratory study is equal to the reproducibility standard deviation. Whether the reproducibility can be used as the basis for the certification depends on whether the interlaboratory study includes all effects to be taken into account for establishing an uncertainty statement. Received: 12 April 1998 · Accepted: 2 July 1998     

Evaluation of the stability of ethanol in water certified reference material: measurement uncertainty under transport and storage conditions

This study simulated the transport and storage conditions of ethanol in water certified reference material (CRM) produced by the Chemical Metrology Division of Inmetro—DQUIM with the purpose of estimating the measurement uncertainty related to stability. The short-term stability study was performed on five different mass fractions (w) in terms of mg ethanol/g solution of the ethanol in water CRM. The nominal values are w = 0.5, 0.9, 1.1, 3.8 and 4.6 mg/g, at temperatures of 4 and 60 °C. On the other hand, the long-term stability study was developed on four different mass fractions (nominal values): w = 0.5, 0.9, 1.1 and 4.6 mg/g, at a temperature of 20 °C. This paper will show the data from the long-term stability study that took place over 52 weeks. The method used complies with ISO Guide 35, the BCR Guideline for Feasibility Studies and ISO Guide 34. According to the statistical parameters used in both studies, the stability of ethanol in water CRM was confirmed for all of the mass fractions studied.     

Reflections regarding uncertainty of measurement, on the results of a Nordic fatigue test interlaboratory comparison

This paper presents the experiences of calculation and reporting uncertainty of measurement in fatigue testing. Six Nordic laboratories performed fatigue tests on steel specimens. The laboratories also reported their results concerning uncertainty of measurement and how they calculated it. The results show large differences in the way the uncertainties of measurement were calculated and reported. No laboratory included the most significant uncertainty source, bending stress (due to misalignment of the testing machine, incorrect specimens and/or incorrectly mounted specimens), when calculating the uncertainty of measurement. Several laboratories did not calculate the uncertainty of measurement in accordance with the Guide to the Expression of Uncertainty in Measurement (GUM) [1].     

Quality assurance programs for countries in need: a global view from the College of American Pathologists

Providing laboratory external quality assessment (EQA) programs for countries in need requires special considerations not ordinarily part of EQA for laboratories in industrialized countries. Cultural, professional, service and economic factors must be understood and accommodated in order to carry out successful programs. Coordination of worldwide efforts for countries in need requires more resources and planning than have thus far been devoted to the enterprise.     

Interlaboratory study of a liquid chromatography method for erythromycin: determination of uncertainty

Erythromycin is a mixture of macrolide antibiotics produced by Saccharopolyspora erythreas during fermentation. A new method for the analysis of erythromycin by liquid chromatography has previously been developed. It makes use of an Astec C18 polymeric column. After validation in one laboratory, the method was now validated in an interlaboratory study. Validation studies are commonly used to test the fitness of the analytical method prior to its use for routine quality testing. The data derived in the interlaboratory study can be used to make an uncertainty statement as well. The relationship between validation and uncertainty statement is not clear for many analysts and there is a need to show how the existing data, derived during validation, can be used in practice. Eight laboratories participated in this interlaboratory study. The set-up allowed the determination of the repeatability variance, s(2)r and the between-laboratory variance, s(2)L. Combination of s(2)r and s(2)L results in the reproducibility variance s(2)R. It has been shown how these data can be used in future by a single laboratory that wants to make an uncertainty statement concerning the same analysis.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

Determination of uncertainty in analytical measurements from collaborative study results on the analysis of a phenoxymethylpenicillin sample

The correct interpretation of a measurement result requires knowledge about its uncertainty. Depending on the conditions under which the analyst is operating, different operational definitions of uncertainty have been proposed. They include: within-laboratory uncertainty, reproducibility uncertainty, bias-included uncertainty and absolute uncertainty. Here we consider the evaluation of the reproducibility uncertainty derived from the results obtained in an inter-laboratory experiment. Nine laboratories participated in an inter-laboratory study for the analysis of phenoxymethylpenicillin. The analyses consisted of a Karl-Fischer water determination, an acid-base titration to assay phenoxymethylpenicillin and a liquid chromatography (LC) method to determine 4-hydroxyphenoxymethylpenicillin and other impurities. The experimental set-up allowed to obtain for each determination s_r² and s_L² as estimates of the repeatability variance (σ_r²) and the between-laboratory variance (σ_L²), respectively. The reproducibility uncertainties for the different assays were then derived from these estimates.     

Problems and risks occurred during uncertainty evaluation of a quantity calculated from correlated parameters: a case study of pH measurement

Parameters of a model describing a measurement process obtained during a calibration experiment allow one to calculate a measurement result, but a simple estimation of measurement uncertainties of the parameters is not sufficient to assess the uncertainty of the result. In this paper, an example of a pH measurement conducted using an ion-selective electrode is presented, in which the uncertainty is evaluated taking into consideration the existing correlation between the parameters of the electrode. The calculations apply either covariances or correlation coefficients that have to be computed additionally. The example presented in this paper illustrates that there are some problems with rounding of variables which, because of the existing very strong correlations, significantly changes the sought uncertainty. This approach is compared with other approaches, that is, usage of uncorrelated variables and Monte Carlo simulations that are described in an earlier work. It is concluded that the approach of uncertainty evaluation, in which covariances or correlation coefficients are explicitly calculated, is work-consuming and may cause significant discrepancies between correct and obtained assessments if some roundings or approximations are done, or if the correlation coefficient is obtained experimentally based on data including random errors.     

The determination of an accepted reference value from proficiency data with stated uncertainties

Proficiency data with stated uncertainties represent a unique opportunity for testing that the reported uncertainties are consistent with the Guide to the expression of uncertainty in measurement (GUM). In most proficiency tests, however, this opportunity is forfeited, because proficiency data are processed without regard to their uncertainties. In this paper we present alternative approaches for determining a reference value as the weighted mean of all mutually consistent results and their stated uncertainties. Using an accepted reference value each reported uncertainty estimate can be expressed as an E _n number, but a value of confirms its validity only if the uncertainty of the reference value is negligible in comparison.Reference values calculated for results from an International Measurement Evaluation Programme (IMEP-9) by “bottom up” as well as “top down” methods were practically identical, although the first strategy yielded the lowest uncertainty. A plot of individual coefficients of variation (CV) versus E _n numbers helps interpretation of the proficiency data, which could be used to validate relative uncertainties down to <1%.