Using target measurement uncertainty to determine fitness for purpose

The methods an analytical laboratory uses must be validated to be fit for purpose. The fitness for purpose of a quantitative method used to determine the concentration of a substance when assessing compliance to requirements can be described by the maximum measurement uncertainty. This is called the target measurement uncertainty. Acceptance criteria for precision and bias in the method validation are then established in terms of the target measurement uncertainty. The target measurement uncertainty can be decided by following a process which involves determining the required concentration range of the measurand; determining the acceptable level of risks of incorrect decisions of compliance; developing a suitable decision rule, with guard bands if appropriate; using the probability of making an incorrect decision of compliance based on the decision rule; and assessing the impact of bias. A key participant in this process is the end user of the data, the laboratory customer. This paper presents the concepts concerning target measurement uncertainty introduced in recently published international guidelines to the practicing analytical chemist who is not generally familiar with these concepts. Three examples are used to illustrate the process.     

Proficiency testing in the light of a new rationale in metrology

The novel proposed definition of measurement result in the international metrology vocabulary requires a revision of standards and guidelines for proficiency testing (PT), and a new approach to processing proficiency data is needed to test the ability of laboratories to present not only unbiased quantity values, but reliable estimates of their uncertainty. Hence, an accepted reference value with the smallest possible uncertainty is needed to ascertain the proficiency of laboratories reporting results with lower than average uncertainty. A strategy based on the T-statistic is proposed leading to an accepted reference value that fully reflects the uncertainties reported by participants in a PT scheme and permits calculation of E _n numbers to distinguish whether or not measurement results are consistent with the accepted definition of the measurand. The strategy is applied to PT data from a recent international laboratory intercomparison of uranium isotopic ratios.     

Comparability of measurement results for pesticide residues in foodstuffs: an open issue?

The dispersion of results from proficiency tests for the analysis of pesticide residues in foodstuffs suggests that improvements in the compatibility of measurement results are needed. Currently observed divergences can make the evaluation conclusion on foodstuffs compliance with certain legislation dependent on the consulted laboratory. This work discusses the origin and impact of this lack of compatibility, following the metrological concepts presented at the latest version of the “International Vocabulary of Metrology” (VIM3), thus allowing for a clear diagnostic of the problem. The reporting of results from different measurement methods uncorrected for the observed analyte recovery makes them traceable to different “operationally defined measurement procedures” (VIM3) and, therefore, not comparable. When results from different measurement methods are reported corrected for analyte recovery, R, and R is different for spiked and incurred residues, measurement results may be not compatible if this effect is not considered on the uncertainty budget. This discussion is illustrated with metrological models for any possible combination of “measurement performance” and “maximum residue level”. These models are complemented with experimental data of the analysis of pesticide residues in a sample of ginseng powder from a proficiency test. The adopted experimental design allowed the identification of additional threats to metrological compatibility in this field. Solutions to the faced problem are discussed for practicability and impact on regulatory issues. The use of a universal “reference measurement procedure” proves to be the most feasible way of ensuring comparability of measurements in this field.     

Measurement uncertainty and its meaning in legal metrology of environmental chemistry and public health

 The need for reliability of measurements supporting legal decisions in environmental policy or medical diagnosis and treatment is well known and widely accepted. This prerequisite can be met only by ensuring that legal measurements are accurate and traceable to national or international standards. Consequently, an outline of the organizational structure of the Romanian National Institute of Metrology (INM) for ensuring uniformity, consistency and accuracy of all measurements including legal measurements performed in chemical laboratories is presented. Since reliable measurements can only be accomplished within an appropriate traceability chain, the experience of the INM in identification and evaluation of measurement uncertainty in legal activities concerning the environment and health is reviewed. Practical examples of measurement uncertainty evaluation in spectrophotometric determination of five analytes, commonly determined in environmental and clinical chemistry are described. The implications of measurement uncertainty for interpretation of regulatory compliance are discussed. Received: 3 January 1998 · Accepted: 9 June 1998     

Uncertainty-based measurement quality control

According to a simple acceptance decision rule for measurement quality control, a measured value will be accepted if the expanded uncertainty of the measurements is not greater than a preset maximum permissible uncertainty. Otherwise, the measured value will be rejected. The expanded uncertainty may be calculated as the z-based uncertainty (the half-width of the z-interval) when the measurement population standard deviation σ is known or the sample size is large (30 or greater), or by a sample-based uncertainty estimator when σ is unknown and the sample size is small. The decision made based on the z-based uncertainty will be deterministic and may be assumed to be correct. However, the decision made based on a sample-based uncertainty estimator will be uncertain. This paper develops the mathematical formulations for computing the probability of acceptance for two sample-based uncertainty estimators: the t-based uncertainty (the half-width of the t-interval) and an unbiased uncertainty estimator. The risk of incorrect decision-making, in terms of the false acceptance probability and false rejection probability, is derived from the probability of acceptance. The theoretical analyses indicate that the t-based uncertainty may result in significantly high false rejection probability when the sample size is very small (especially for samples of size 2). For some applications, the unbiased uncertainty estimator may be superior to the t-based uncertainty for measurement quality control. Several examples from acoustic Doppler current profiler streamflow measurements are presented to demonstrate the performance of the t-based uncertainty and the unbiased uncertainty estimator.     

Chemical metrology, chemistry and the uncertainty of chemical measurements

Chemical results normally involve traceability to two reference points, the specific chemical entity and the quantity of this entity. Results must also be traceable back to the original sample. As a consequence, any useful estimation of uncertainty in results must include components arising from any lack of specificity of the method, the variation between repeats of the measurement and the relationship of the result to the original sample. Chemical metrology does not yet incorporate uncertainty arising from any lack of specificity from the method selected or the traceability of the result to the original sample. These sources of uncertainty may however have much more impact on the reliability of the result than will any uncertainty associated with the repeatability of the measurement. Uncertainty associated with sampling may amount to 50–1000% of the reported result. Chemical metrology must be expanded to include estimations of uncertainty associated with lack of specificity and sampling. Received: 29 May 2001 Accepted: 17 December 2001     

Optimising calibration and measurement capabilities in terms of economics in conformity assessment

Calibration measurement capabilities (CMC) are key factors in declaring the metrological performance of national metrology institutes (NMIs). Different countries have different CMC capabilities, reflecting both the existing measurement science competence as well as the perceived national needs for traceable calibration. This paper deals with increasing interest in decision-making in conformity assessment in terms of effective costs associated with measurement, testing and incorrect decision-making. The work examines the CMCs of calibration laboratories and NMIs with economic decision theory, in particular, in terms of customer satisfaction and with respect to conformity assessment issues. Optimal strategies for calibration costs, maintenance of national measurement standards, testing and production costs are illustrated in practical examples. CMCs are an essential instrument to enable conformity assessment both for product safety, legal metrology, quality requirements as well as scientific research. The newly defined term “target measurement uncertainty”, introduced in the latest international metrology vocabulary (VIM), should be therefore always related to appropriate CMCs and related dissemination paths in the whole conformity assessment procedure. These requirements are clear and transparent justification for the development of required national metrological infrastructures, in order to fulfil the requirements of target measurement uncertainty for intended use or application in the particular conformity assessment procedure.     

Systematic errors in analytical measurement results

Definitions of the concepts of bias and recovery are discussed and approaches to dealing with them described. The Guide To Uncertainty in Measurement (GUM) recommends correction for all significant systematic effects, but it is also possible to expand measurement uncertainty to take account of uncorrected bias. Run, laboratory and method bias can be defined as components of the bias of a particular measurement result, and can be useful as concepts used in method validation. Estimation of run bias allows a simplification of the estimation of measurement uncertainty. Multivariate calibration brings its own biases that must be quantified and minimised.     

Conceptual thinking and metrology concepts

In introducing the term ‘concept’, the authors of the 2008 International vocabulary of metrology ‘Basic and general concepts and associated terms’ (VIM, 2008) recognize that in order to operationalize a globally accepted set of metrology terms, one requires to deal with a higher level of abstraction. Concepts are obviously not specific to metrology–handling complex tasks in any domain of knowledge that requires conceptual thinking abilities. In this short white paper, we discuss how to assess and develop conceptual thinking of professionals in service, business, and industrial environments. The approach builds on a proven methodology called MERLO that has been developed in the last 15 years by experts in psychology and education with adaptation to new interactive technologies such as clickers and internet-based formative assessments. MERLO pedagogy can be used to assess individuals’ inherent conceptual thinking abilities and train them to enhance their competence in analyzing complex conceptual situations. This is pertinent to the education of metrology, quality, and statistical thinking. We suggest that MERLO can be considered as a complementary enabler to VIM, so that this fundamental work can enhance its impact and applicability.     

Do we need to consider metrological meanings of different measurement uncertainty estimations?

Along the years, several approaches for measurement uncertainty estimation have been suggested. Emphasis has been put on the general metrological interpretation of measurement uncertainty, but not on its different meanings when it is associated to given conditions of measurement where analytical work is performed and errors are originated. Three different definitions for uncertainty are proposed for reproducibility and intermediate precision conditions of measurement. These definitions inherit features from the VIM 3 definition of measurement uncertainty. It is argued that if a high performance laboratory keeps errors under control with proper validation and quality assurance programs, measurement uncertainty from intermediate precision condition of measurement is justified as a suitable estimation of its capability to attribute values to a measurand. Alternatively, a laboratory that does not keep errors under control should use uncertainty from reproducibility condition of measurement as the cost of its imperfections. Selection of information sources for measurement uncertainty estimation should be in harmony with its metrological meaning.     

Sampling and metrology

The result of a measurement refers in principle only to the amount of substance actually contributing to the analytical signal. However, an appropriate definition of the measurand must include a specification of the system for which the result of the measurement should apply. All systems being inherently heterogeneous, representativity assumes importance for the metrological quality of a measurement, and the process needed to ascertain representativity is sampling. The contribution from this characteristic must be included when expressing the uncertainty of the reported value of the measurand. Representative sampling of systems that are infinite or non-uniform was developed by Pierre Gy in his Theory of Sampling. Finite systems can achieve uniformity by mechanical treatment and mixing; the heterogeneity of these systems can be characterized by a sampling constant, expressed in units of weight, for each particular species being determined. Examples of the contribution of sampling to the uncertainty of analytical results are discussed for some biological materials. Presented at the 2nd International Conference on Metrology – Trends and Applications in Calibration and Testing Laboratories, November 4–6, 2003, Eilat, Israel.     

The use of certified reference materials in the Romanian traceability scheme

 For ensuring the traceability and uniformity of measurement results, the main objectives of national metrology programmes in chemistry are to calibrate and verify measuring instruments, to evaluate the uncertainty of measurement results and to intercompare the analytical results, etc. The concept of traceability has developed recently in chemical measurements, thus, an attempt to implement the principles of metrological traceability especially by appropriateness calibration using composition certified reference materials (CRMs) is underlined. Interlaboratory comparisons are also a useful response to the need for comparable results. The paper presents some aspects and practices in the field of spectrometric measurement regarding the metrological quality of the traceability by calibrating the instruments using suitable and reliable CRMs. The uncertainty of results, as a measure of the reliability that can be placed on them, has been adequately described in different documents and, as a consequence, some examples of evaluating the measurement uncertainty are described. The relationship between uncertainty and traceability, as two fundamental concepts of metrology which are intimately linked, is underlined. Received: 12 November 1999 / Accepted: 10 December 1999     

Appropriate rather than representative sampling, based on acceptable levels of uncertainty

Appropriate sampling, that includes the estimation of measurement uncertainty, is proposed in preference to representative sampling without estimation of overall measurement quality. To fulfil this purpose the uncertainty estimate must include contribution from all sources, including the primary sampling, sample preparation and chemical analysis. It must also include contributions from systematic errors, such as sampling bias, rather than from random errors alone. Case studies are used to illustrate the feasibility of this approach and to show its advantages for improved reliability of interpretation of the measurements. Measurements with a high level of uncertainty (e.g. 50%) can be shown to be fit for some specified purposes using this approach. Once reliable estimates of the uncertainty are available, then a probabilistic interpretation of results can be made. This allows financial aspects to be considered in deciding upon what constitutes an acceptable level of uncertainty. In many practical situations ”representative” sampling is never fully achieved. This approach recognises this and instead, provides reliable estimates of the uncertainty around the concentration values that imperfect appropriate sampling causes. Received: 28 December 2001 Accepted: 25 April 2002     

From a single analyzer to multiple instruments—The GUM approach

Assessment and expression of analytical quality have become novel spotlights in medical laboratories since accreditation began in the early 1990s, in Europe. Evaluation of uncertainty of measurement by definition was launched in Finland when the Finnish Accreditation Service (FINAS) accredited the first medical laboratories in the mid 1990s. In spite of all the analytical and statistical knowledge which has been available in medical laboratories for years, evaluation of total uncertainty of measurement has not yet caught on. The concept is still unfamiliar to experts and, indeed, little guidance has been available. National and international activities, with good results, can be shown when the educational aspect is considered. The Guide to the Expression of Uncertainty in Measurement (GUM) remains the main document for uncertainty evaluation. Uncertainty of measurement together with target value of uncertainty can be used as a good measure for analytical quality in large or smaller laboratories over time, because it is a quantitative indication and the evaluation is easy to repeat as running practical tools are available.Presented at the 8th Conference on Quality in the Spotlight, 17–18 March 2003, Antwerp, Belgium     

Practical considerations on the traceability to conventional scales

 The basic concepts of traceability as they are defined by the Comité Consultatif pour la Quantité de Matière (CCQM) are difficult to apply to some chemical results. For instance, for some environments or chemical analyses measurement results are expressed in conventional units. Such units are realized on conventional scales relying on two fundamental pillars: reference materials and standard specification. The octane number of fuel or water turbidity measurements are typical examples of such units. Traceability concepts are discussed in terms of their practical applicability for turbidimetric analysis. Some outcomes on the validation of the metrological performance of turbidimeters and the comparability of turbidity measurement results are also presented. Received: 8 June 1999 / Accepted: 13 December 1999     

Guidelines and recommended terms for expression of stable-isotope-ratio and gas-ratio measurement results

To minimize confusion in the expression of measurement results of stable isotope and gas-ratio measurements, recommendations based on publications of the Commission on Isotopic Abundances and Atomic Weights of the International Union of Pure and Applied Chemistry (IUPAC) are presented. Whenever feasible, entries are consistent with the Système International d'Unités, the SI (known in English as the International System of Units), and the third edition of the International Vocabulary of Basic and General Terms in Metrology (VIM, 3rd edition). The recommendations presented herein are approved by the Commission on Isotopic Abundances and Atomic Weights and are designed to clarify expression of quantities related to measurement of isotope and gas ratios to ensure that quantity equations instead of numerical value equations are used for quantity definitions. Examples of column headings consistent with quantity calculus (also called the algebra of quantities) and examples of various deprecated usages connected with the terms recommended are presented.     

Probabilistic uncertainty estimator for gamma-spectroscopy measurements

To properly interpret the quality of a gamma-spectroscopy measurement, an uncertainty estimate must be made. The uncertainty in the efficiency calibration is the dominant component to the total propagated measurement uncertainty for many types of measurements. Any deviations between the as-calibrated geometry and the as-measured geometry contribute to the total uncertainty. A mathematical technique has been developed to evaluate the variations between calibration and measurement conditions. A sensitivity analysis mode identifies those variables with the largest contribution to the uncertainty. The uncertainty mode uses probabilistic techniques for the combined variables to compute average efficiency and uncertainty, and then to propagate those values with the gamma-spectroscopic analysis into the final result for that sample.     

Outline for the revision of ISO Guide 35

The production of reference materials (RMs) is a key activity for the improvement and maintenance of a worldwide coherent measurement system. As detailed in ISO Guide 33, RMs with different characteristics are used in measurements, such as calibration, quality control and method validation, as well as for the assignment of values to other materials. Currently, ISO Guide 35 is in its third edition after it was revised in 2006. The Guide was developed to support best practices in the value assignment to specified properties of Certified Reference Materials (CRMs). This Guide gives general guidance and explains concepts to assist the understanding and development of valid methods to assign values to the properties of a reference material, including the evaluation of their associated measurement uncertainties, and the establishment of their metrological traceability. From the outcome of a systematic review of ISO Guide 35 among the members of ISO/REMCO, the ISO Committee on Reference Materials, it followed that there is a need for revising the current edition of ISO Guide 35. The mandate for the revision is focused on editorial updates to explain the concepts in more detail. It is not envisaged that major technical changes will be introduced. This paper explains the approach and rationale for the revision of ISO Guide 35 and invites comments from the users of the current edition of ISO Guide 35.