Design of experiment for evaluation of uncertainty from sampling in the framework of the fitness for purpose concept: a case study

The design of an experiment for the evaluation of sampling uncertainty in the framework of the fitness for purpose concept is described in terms of probabilities (risks of the user) of type 1 and type 2 errors in decisions concerning the significance of effects influencing the sampling uncertainty and the measurement results. As a case study, an experiment based on the duplicate method for quantification of the sampling uncertainty and inhomogeneity (heterogeneity) of a melt of tin-free bronze produced in a 10-ton reflective oven is analyzed. The melt is defined as the sampling target. It is shown that the number of such targets (melts), the number of samples under analysis and the number of replicate analyses can be minimized, i.e., the size and cost of the experiment can be reduced, if the user knows which risks are acceptable. When inhomogeneity of the sampling target has a systematic character, like the decrease of the mass fraction of aluminum from the beginning to the end of the melt pouring in the case study, the inhomogeneity effect can be separated from the sampling uncertainty and evaluated according to the user’s needs.     

Using target measurement uncertainty to determine fitness for purpose

The methods an analytical laboratory uses must be validated to be fit for purpose. The fitness for purpose of a quantitative method used to determine the concentration of a substance when assessing compliance to requirements can be described by the maximum measurement uncertainty. This is called the target measurement uncertainty. Acceptance criteria for precision and bias in the method validation are then established in terms of the target measurement uncertainty. The target measurement uncertainty can be decided by following a process which involves determining the required concentration range of the measurand; determining the acceptable level of risks of incorrect decisions of compliance; developing a suitable decision rule, with guard bands if appropriate; using the probability of making an incorrect decision of compliance based on the decision rule; and assessing the impact of bias. A key participant in this process is the end user of the data, the laboratory customer. This paper presents the concepts concerning target measurement uncertainty introduced in recently published international guidelines to the practicing analytical chemist who is not generally familiar with these concepts. Three examples are used to illustrate the process.     

Uncertainty from sampling,in the context of fitness for purpose

Sampling is an integral part of nearly all chemical measurement and often makes a substantial or even a dominant contribution to the uncertainty of the measurement result. In contrast with analysis, however, the uncertainty contribution from sampling has usually been ignored. Indeed, far less is known about sampling uncertainty, although in some application sectors it is known to exceed the analytical uncertainty, especially when raw materials (natural or industrial) are under test. In 1995 the authors of this paper proposed a framework of concepts and procedures for studying, quantifying, and controlling the uncertainty arising from the sampling that normally precedes analysis. Many of the ideas were based on analogy with well-established procedures and considerations relating to quality of analytical measurement, ideas such as validation of the sampling protocol, sampling quality control and fitness for purpose. Since that time many of these ideas have been explored experimentally and found to be effective. This paper is a summary of progress to date. Presented at the AOAC Europe Workshop, November 2006, Limassol, Cyprus.

---

Modifying conventional high-performance liquid chromatography systems to achieve fast separations with Fused-Core columns: a case study

The theoretical increase in performance from the use of high efficiency columns with conventional HPLC equipment is generally not observed due to the design limitations of such equipment, particularly with respect to extra-column dispersion (ECD). This study examines the impact of ECD from a Waters Alliance 2695 system on the performance of 2.7 μm HALO C(18) Fused-Core superficially porous particle columns of various dimensions. The Alliance system was re-configured in different ways to reduce extra-column volume (ECV) and the ECD determined in each case as a function of flow rate up to a maximum of 2 mL/min. The results obtained showed a progressive decrease in ECD as the ECV was reduced, irrespective of the flow rate employed. However, this decrease in ECD was less than theoretically expected for the lower ECV configurations. The inability to reduce the actual extra-column dispersion further was attributed to additional dispersion associated with the design/volume of the auto-injector. This was confirmed by making sample injections with a low dispersion manual injection valve, instead of auto-injection, for the two lowest ECV configurations studied. In each case, the measured and predicted ECD values were in good agreement. The auto-injector module is an integral part of the Alliance 2695 instrument and cannot be easily modified. However, even with autosampler injection, for a 3mm ID × 100 mm Fused-Core column approximately 70% of the maximum plate count (～84% of the resolution or more) could still be obtained in isocratic separations for solutes with k ≥ ～4.5 when using the lowest ECV configuration. This study also highlights some of the problems inherent in trying to measure accurately the true extra-column dispersion of a chromatographic system and compares the results obtained to those theoretically predicted. Using this same lowest volume instrument configuration, two real-world pharmaceutical methods were scaled to separations that are ～3-3.5-fold faster, while still maintaining comparable data quality (resolution and signal-to-noise ratios).     

Estimating sampling and analysis uncertainties to assess the fitness for purpose of a water quality monitoring network

In order to verify that the protocols used for water quality monitoring of surface waters within the Long-term Environmental Research Monitoring and Testing System (OPE), located in the north-eastern part of France in relation with a geological disposal for radioactive waste project, are fit for purpose, a validation study was conducted following the methodology described in the Eurachem/citac and Nordtest guidance documents on uncertainty arising from sampling. As one of the objectives of the OPE water monitoring programme was to investigate the spatial and temporal variability of water quality, quality requirements were set to having a measurement variance, including sampling and analytical contributions, less than 20 % of the total variance to minimise the impact of measurement over the observed environmental variability. The replicate method was then selected in order to estimate the measurement uncertainty, including the sampling contribution, as well as the spatial and temporal variability of water quality of surface waters. To minimise costs, a single-split level was selected. Analytical uncertainties were assessed from inter-laboratory data and/or internal quality control data from the last 2–5 years. Finally, ANOVA was applied to the data sets after elimination of outliers. Results showed that for pH, electrical conductivity, turbidity and nitrate, the sampling uncertainty was negligible, whereas for other parameters such as dissolved oxygen, total suspend solids, total organic carbon, nitrite and phosphate, the sampling contribution to the measurement uncertainty was largely significant. For all parameters except calcium, the sampling and analytical protocols were considered fit for purpose.     

Uncertainty from sampling: workshop to launch a Nordtest handbook on sampling uncertainty estimation and control

A workshop on uncertainty in sampling was held in Hillerød, Denmark, on 12–13 April 2007 to launch a new handbook on sampling quality assurance and uncertainty estimation. The participants of the workshop were approximately 60 delegates from 15 European countries, representing institutions performing sampling, users of the data, research institutions, as well as accreditation bodies. Materials from the workshop, including examples, tools, and calculation aids for the work can be found at http://www.samplersguide.com. The Nordtest handbook Uncertainty from sampling will be made available on the Nordtest web site at http://www.nordicinnovation.net/nordtest.cfm under NT technical reports, report number NT tec 604. Until the final report is available on the Nordtest web site, an advance draft of the Nordtest handbook is available from http://www.samplersguide.com.     

A decision theory approach to fitness for purpose in analytical measurement

The choice of an analytical procedure and the determination of an appropriate sampling strategy are here treated as a decision theory problem in which sampling and analytical costs are balanced against possible end-user losses due to measurement error. Measurement error is taken here to include both sampling and analytical variances, but systematic errors are not considered. The theory is developed in detail for the case exemplified by a simple accept or reject decision following an analytical measurement on a batch of material, and useful approximate formulae are given for this case. Two worked examples are given, one involving a batch production process and the other a land reclamation site.     

Soil sampling uncertainty on arable fields estimated from reference sampling and a collaborative trial

On three fields of arable land of (3–6)×10⁴ m², simple reference sampling was performed by taking up to 195 soil increments from each field applying a systematic sampling strategy. From the analytical data reference values for 15 elements were established, which should represent the average analyte mass fraction of the areas. A “point selection standard deviation” was estimated, from which a prediction of the sampling uncertainty was calculated for the application of a standard sampling protocol (X-path across the field, totally 20 increments for a composite sample). Predicted mass fractions and associated uncertainties are compared with the results of a collaborative trial of 18 experienced samplers, who had applied the standard sampling protocol on these fields. In some cases, bias between reference and collaborative values is found. Most of these biases can be explained by analyte heterogeneity across the area, in particular on one field, which was found to be highly heterogeneous for most nutrient elements. The sampling uncertainties estimated from the reference sampling were often somewhat smaller compared to those from the collaborative trial. It is suspected that the influence of sample preparation and the variation due to sampler were responsible for these differences. For the applied sampling protocol, the uncertainty contribution from sampling generally is in the same range as the uncertainty contribution from analysis. From these findings, some conclusions were drawn, especially about the consequences for a sampling protocol, if in routine sampling a demanded “certainty of trueness” for the measurement result should be met.     

Correction to: Combined uncertainty factor for sampling and analysis

Accreditation and Quality Assurance - In the penultimate paragraph of the original publication, a confidence interval of 93 mg/kg to 971 mg/kg was reported. These values should be...     

Estimates of uncertainty of measurement from proficiency testing data: a case study

The results obtained by a laboratory over a number of proficiency testing/external quality assessment schemes (PT/EQAS) rounds can give information on the uncertainty of its measurements for a given test, provided that conditions such as full coverage of the routine analytical range, traceability, and small uncertainty of the assigned values (compared to the spread of the results) are met and provided that systematic deviations and any other sources of uncertainty are considered. As organisers of the Italian EQAS (ITEQAS) in occupational and environmental laboratory medicine, we tested this hypothesis using as model data from well-performing laboratories taking part in ITEQAS for lead in blood over the last 2 years. We also investigated how different PT/EQAS features (frequency of trials and number of samples) would affect a laboratory estimate of its uncertainty. Such information can be helpful in improving PT/EQAS organisation and define, for a given test: (a) the state of the art of the uncertainty of current measurement procedures, (b) identify needs for improvement of analytical methodologies and (c) set targets for acceptable uncertainty values.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia.Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher.     

Relationship between performance characteristics obtained from an interlaboratory study programme and combined measurement uncertainty: a case study

 In the interlaboratory study programme "ILS Coal Characterisation", eight interlaboratory studies were organised based on the ISO standards for coal analysis. The use of blind samples in each round allows comparability of measurement results between rounds to be assessed. Based on the results, it could be demonstrated that the vast majority of the measurement results of the laboratories were traceable to results obtained in previous rounds of this programme. The hypothesis has been formulated that the combined standard uncertainty obtained from an interlaboratory study is equal to the reproducibility standard deviation. Whether the reproducibility can be used as the basis for the certification depends on whether the interlaboratory study includes all effects to be taken into account for establishing an uncertainty statement. Received: 12 April 1998 · Accepted: 2 July 1998     

Comparison of three strategies to evaluate uncertainty from in-house validation data. A case study: mercury determination in sediments

In the present paper, three approaches are compared for the evaluation of the combined uncertainty in the determination of mercury in aquatic sediments by an aqua regia extraction procedure. For this, the data obtained in validation studies from five certified reference materials (CRMs), covering a range of concentrations from 0.8 to 130 mg kg⁻¹ of mercury and analysed by three atomic spectroscopic techniques (cold vapour generation atomic fluorescence spectrometry, CV-AFS, cold vapour generation atomic absorption spectroscopy, and inductively coupled plasma mass spectroscopy), were considered. The combined uncertainty was firstly assessed by considering separately the data obtained for each CRM analysed (approach A). Moreover, this assessment was also performed with two other calculation approaches (B and C) based on the pooled data obtained from the validation step. The comparison of the results obtained for the different techniques showed a clear bias effect when using CV-AFS with nitric acid as a diluent. In relation to the strategies tested for the combined uncertainty assessment, approach C proved to be the easiest and friendliest method for uncertainty assessment.      

Preparation of fine powders by clathrate-forming freeze-drying: a case study of ammonium nitrate

1. Download high-res image (245KB)
2. Download full-size image

     

Interferences from oxygen reduction reactions in bioelectroanalytical measurements: the case study of nitrate and nitrite biosensors

Bioelectroanalytical procedures based on cathodic processes are often subject to interference from dissolved oxygen. At the potentials applied for analyte detection, oxygen reduction may occur directly at the electrode or may be catalyzed by the electron mediators or the sensing enzyme of the biosensor. These processes affect the background current and may thus result in erroneous analyte quantification. In this review, current strategies to circumvent these oxygen interferences are presented and critically assessed with respect to their compatibility for on-site monitoring with amperometric biosensing devices operating at low potential. The main strategies consist in (1) use of oxygen scavenging systems to remove dissolved oxygen from the sample, (2) design of bioelectroanalytical approaches to shift the applied potential for analyte detection to more positive values, and (3) development of electrode materials to increase the overpotential for the oxygen reduction reaction. The latest developments in these approaches have recently led to the first biosensing devices based on reductases fully compatible with on-site monitoring requirements and this opens up possibilities for their widespread application.     

Problems and risks occurred during uncertainty evaluation of a quantity calculated from correlated parameters: a case study of pH measurement

Parameters of a model describing a measurement process obtained during a calibration experiment allow one to calculate a measurement result, but a simple estimation of measurement uncertainties of the parameters is not sufficient to assess the uncertainty of the result. In this paper, an example of a pH measurement conducted using an ion-selective electrode is presented, in which the uncertainty is evaluated taking into consideration the existing correlation between the parameters of the electrode. The calculations apply either covariances or correlation coefficients that have to be computed additionally. The example presented in this paper illustrates that there are some problems with rounding of variables which, because of the existing very strong correlations, significantly changes the sought uncertainty. This approach is compared with other approaches, that is, usage of uncorrelated variables and Monte Carlo simulations that are described in an earlier work. It is concluded that the approach of uncertainty evaluation, in which covariances or correlation coefficients are explicitly calculated, is work-consuming and may cause significant discrepancies between correct and obtained assessments if some roundings or approximations are done, or if the correlation coefficient is obtained experimentally based on data including random errors.     

Screening nitrate in forages with a test strip: collaborative study

A collaborative study was conducted for screening nitrate in forages with a commercially available test strip. The method involves extracting a finely ground sample with deionized water. The test strip is dipped in the sample extract. The color of the reaction zone on the test strip changes from white to pink or purple depending on the nitrate concentration in sample extract. The nitrate present in the extract is determined by comparing the color of the test strip to the color scale on the test strip container. Six blind quintuplicates of forage samples were analyzed by 20 collaborators. Nitrate concentrations in forage samples tested ranged from < 1000 ppm nitrate to > 10,000 ppm nitrate on dry matter basis. Each collaborator was asked to assign each sample to one of the 4 following nitrate concentration ranges: (1) < 1000 ppm, (2) 1000 to 5000 ppm, (3) > 5000 ppm to 10,000 ppm, and (4) > 10,000 ppm. Nineteen of 20 collaborators reported results. Results from 2 laboratories were rejected as outliers by inspection and chi 2 test. Sensitivity rates (p+) ranged from 0.965 to 0.998, with standard errors of 0.006 to 0.16. Specificity rates (p-) ranged from 0.991 to 0.997 for the 4 ranges, with standard errors of 0.003 to 0.006. False-positive rates (pf+) ranged from 0.006 to 0.046, with standard errors of 0.006 to 0.025. False-negative rates (pf-) ranged from 0.003 to 0.007, with standard errors of 0.003 to 0.006. Screening nitrate in forages with a test strip has been adopted first action by AOAC INTERNATIONAL.     

Interlaboratory study of a liquid chromatography method for erythromycin: determination of uncertainty

Erythromycin is a mixture of macrolide antibiotics produced by Saccharopolyspora erythreas during fermentation. A new method for the analysis of erythromycin by liquid chromatography has previously been developed. It makes use of an Astec C18 polymeric column. After validation in one laboratory, the method was now validated in an interlaboratory study. Validation studies are commonly used to test the fitness of the analytical method prior to its use for routine quality testing. The data derived in the interlaboratory study can be used to make an uncertainty statement as well. The relationship between validation and uncertainty statement is not clear for many analysts and there is a need to show how the existing data, derived during validation, can be used in practice. Eight laboratories participated in this interlaboratory study. The set-up allowed the determination of the repeatability variance, s(2)r and the between-laboratory variance, s(2)L. Combination of s(2)r and s(2)L results in the reproducibility variance s(2)R. It has been shown how these data can be used in future by a single laboratory that wants to make an uncertainty statement concerning the same analysis.     

Enveloping distribution sampling: a method to calculate free energy differences from a single simulation

The authors present a method to calculate free energy differences between two states A and B "on the fly" from a single molecular dynamics simulation of a reference state R. No computer time has to be spent on the simulation of intermediate states. Only one state is sampled, i.e., the reference state R which is designed such that the subset of phase space important to it is the union of the parts of phase space important to A and B. Therefore, an accurate estimate of the relative free energy can be obtained by construction. The authors applied the method to four test systems (dipole inversion, van der Waals interaction perturbation, charge inversion, and water to methanol conversion) and compared the results to thermodynamic integration estimates. In two cases, the enveloping distribution sampling calculation was straightforward. However, in the charge inversion and the water to methanol conversion, Hamiltonian replica-exchange molecular dynamics of the reference state was necessary to observe transitions in the reference state simulation between the parts of phase space important to A and B, respectively. This can be explained by the total absence of phase space overlap of A and B in these two cases.     

High-performance sustainable tissue paper from agricultural residue: a case study on fique fibers from Colombia

Cellulose - Global sustainability megatrends are promoting the utilization of sustainably perceived fibers such as recycled and agricultural residue fibers in hygiene tissue applications. Tissue...