Reproducibility estimates from single-laboratory ruggedness tests compared with those derived from collaborative trials of six microbiological measurement methods

This work investigated the feasibility of single-laboratory ruggedness experiments to estimate between-laboratory reproducibility. Six microbiological measurement methods that had already been subject to multi-laboratory validation were studied. Ruggedness experiments were designed and executed to determine sensitivity coefficients for factors judged likely to vary between laboratories. These were combined with estimates of factor variation to give reproducibility estimates. The single-laboratory estimates of reproducibility were generally not similar to those generated from multi-laboratory work. The experimental plans were difficult to design and execute, and were only partially successful in producing useful sensitivity coefficients. The authors conclude that ruggedness tests do not offer a single-laboratory alternative to multi-laboratory method validation or a practical approach to the evaluation of measurement uncertainty for microbiological methods. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users. Campden BRI was previously known as the Campden and Chorleywood Food Research Association.     

Evaluation of measurement uncertainties for the determination of total metal content in soils by atomic absorption spectrometry

In this work estimation of measurement uncertainties associated with the total metal content in soils was done by an intralaboratory approach based on method validation and quality control data, and using two certified reference materials (CRM). CRM and soil samples were analyzed following procedures based on the methods that are applied to silicate materials. All elements were determined by atomic absorption spectrometry following a quality assurance program previously established. Quality control actions were implemented in order to provide reliable data. The precision under within-laboratory reproducibility conditions was estimated from triplicate analysis. The trueness component was determined as recovery of the analyte from CRMs: soil sample, SO-2 and river clay sediment, LGC 6139. Combined measurement uncertainty was expressed in terms of precision and recovery uncertainties and the later further split on CRM replicate analysis and uncertainty of the certified value components. The results obtained are critically discussed on the basis of the different contributions. For the selection of the reference material, the CRM dependent terms are critically compared in order to fulfill specific requirements. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.     

Speciation analysis of chromium in drinking water samples by ion-pair reversed-phase HPLC–ICP-MS: validation of the analytical method and evaluation of the uncertainty budget

The approach presented in this article refers to the modification of a method for the detection and quantitative determination of chromium species in water by high-performance liquid chromatography inductively coupled plasma mass spectrometry. The main aim of this work was to establish a detailed validation of the analytical procedure and an estimation of the budget of measurement uncertainty which was helpful in recognizing the critical points of the presented method. As a result of the method validation experiment, the obtained limit of quantification, repeatability and intermediate precision were satisfied for the quantification Cr(III) and Cr(VI) in water matrices. The trueness of the method was verified via an estimation of the recovery of the spiked real samples. The recovery rate of both determined analytes was found to be between 93 and 115 %. Considering that the validation of the method and the evaluation of measurement uncertainty are crucial for quantitative analysis, the above-mentioned assessment of the uncertainty budget was performed in two different ways: a modelling approach and a single-laboratory validation approach. The measurement uncertainties of the results were found to be 4.4 and 7.8 % for Cr(III), 4.2 and 7.9 % for Cr(VI) using the classical concept and method validation data, respectively. This paper is the first publication to presenting all the steps needed to evaluate the measurement uncertainty for the speciation analysis of chromium species. In summary, the obtained results demonstrate that the method can be applied effectively for its intended use.     

Measurement uncertainty evaluation and in-house method validation of the herbicide iodosulfuron-methyl-sodium in water samples by using HPLC analysis

A method for separation and quantitative determination of the iodosulfuron-methyl-sodium in water samples by high-performance liquid chromatography (HPLC) was developed and in-house validated in order to demonstrate its performance for monitoring of heterogeneous photocatalytic elimination of the herbicide iodosulfuron-methyl-sodium from water. Surface and ground water samples were used to demonstrate its selectivity, detection and quantification limits, linearity, trueness and precision. In addition, stability of iodosulfuron-methyl-sodium was studied in function of temperature and time. Method accuracy was quantified through measurement uncertainty estimate based on method validation data. The paper gives practical and easy to follow guidance on how uncertainty estimates can be obtained from method validation experiments. It shows that, if properly planned and executed, key precision and trueness studies undertaken for validation purposes can also provide much of the data needed to produce an estimate of measurement uncertainty. Our analytical protocol allowed us to quantify iodosulfuron-methyl-sodium in ground water and surface water in concentration level between 2.50–50.0 μmol L⁻¹ with satisfactory recoveries (99–104%) and repeatability lower or equal than 0.3% for all the matrices. We also estimated within-laboratory reproducibility over 3-month period, which was 0.7%. We proved that the method was selective for determination of iodosulfuron-methyl-sodium in the relevant matrices. Measurement uncertainty of results was evaluated to be 4.0% with 95% confidence level. After validation and measurement uncertainty evaluation steps, results obtained showed that the method can be applied to efficiently monitor heterogenous photocatalytic degradation of the herbicide iodosulfuron-methyl-sodium.     

Estimation of measurement uncertainty for the determination of nonylphenol in water using solid-phase extraction and solid-phase microextraction procedures

We describe an estimation of measurement uncertainty calculated by the “bottom-up” approach for the determination of the oestrogenic compound nonylphenol in treated water samples by solid-phase extraction (SPE) and solid-phase microextraction (SPME) procedures and GC/MS detection. The results were compared and the different contributions to the uncertainty were evaluated. A study of the linear range was established and validation was performed for both methods using statistical analysis of several indicative parameters. In terms of validation data, precision (R.S.D. values <20%) and trueness (relative error <11%) were obtained for both methods under day-to-day conditions. The results of the estimation of measurement uncertainty obtained for both methods for concentrations higher than 1 μg/l have demonstrated that the time-consuming SPE method has a lower relative uncertainty (32%) than the SPME method (42.8%). The chromatographic uncertainty value was the main factor in the SPME method whereas the recovery factor (used to calculate the concentration) was the main contribution to uncertainty in the SPE method.     

Quantifying uncertainty in determination of polysaccharides in glycoconjugate vaccines based on in-house validation data

The measurement uncertainty in quantifying polysaccharides in glycoconjugate vaccines was estimated in order to ascertain compliance of these products in the batch release procedure of an Official Medicines Control Laboratory. A single-laboratory validation approach of an internal analytical procedure to quantify the total saccharide by the way of the sialic acid (N-Acetylneuraminic acid) concentration in a meningococcal glycoconjugate vaccine by High Performance Anion Exchange Chromatography coupled with Pulsed Amperometric Detection is reported. Uncertainty components from precision, bias and other sources were evaluated. It turned out that uncertainty is dominated by the precision contribution, while bias was mainly due to the non-recovery from hydrolysis. As a consequence, a correction factor was considered. At a confidence level of approximately 95%, the expanded uncertainty of the total polysaccharide concentration in a vaccine, when measured using the internal method, was found to be 11.5%.     

Comparison of different approaches to estimate the uncertainty of a liquid chromatographic assay

A measurement result cannot be properly interpreted without knowledge about its uncertainty. Several concepts to estimate the uncertainty of a measurement result have been developed. Here, four different approaches for uncertainty estimation are compared on the example of the RP-high-performance liquid chromatography (HPLC) assay for tylosin for veterinary use: the guide to the expression of uncertainty in measurement (GUM) approach, which derives the uncertainty of a measurement result by combining the uncertainties related to the uncertainty sources of the measurement process; the top-down approach, which uses the reproducibility estimate from an inter-laboratory study as uncertainty estimate; an approach recently presented by Barwick and Ellison, which combines precision, trueness and robustness data to obtain an uncertainty estimate of the measurement result and finally a further approach, which directly estimates the measurement uncertainty from a robustness test. The comparison shows that the different approaches lead to comparable uncertainty estimates.     

A diagnostic tool for determining the quality of accuracy validation. Assessing the method for determination of nitrate in drinking water

Realistic internal validation of a method implies the performance validation experiments under intermediate precision conditions. The validation results can be organized in an X _Nr×Ns (replicates×runs) data matrix, analysis of which enables assessment of the accuracy of the method. By means of Monte Carlo simulation, uncertainty in the estimates of bias and precision can be assessed. A bivariate plot is presented for assessing whether the uncertainty intervals for the bias (E ± U(E)) and intermediate precision (RSDi ± U(RSDi) are included in prefixed limits (requirements for the method). As a case study, a method for determining the concentration of nitrate in drinking water at the official level set by 98/83/EC Directive is assessed by use of the proposed plot.     

New advances in method validation and measurement uncertainty aimed at improving the quality of chemical data

The implementation of quality systems in analytical laboratories has now, in general, been achieved. While this requirement significantly modified the way that the laboratories were run, it has also improved the quality of the results. The key idea is to use analytical procedures which produce results that fulfil the users needs and actually help when making decisions. This paper presents the implications of quality systems on the conception and development of an analytical procedure. It introduces the concept of the lifecycle of a method as a model that can be used to organize the selection, development, validation and routine application of a method. It underlines the importance of method validation, and presents a recent approach based on the accuracy profile to illustrate how validation must be fully integrated into the basic design of the method. Thanks to the -expectation tolerance interval introduced by Mee (Technometrics (1984) 26(3):251–253), it is possible to unambiguously demonstrate the fitness for purpose of a new method. Remembering that it is also a requirement for accredited laboratories to express the measurement uncertainty, the authors show that uncertainty can be easily related to the trueness and precision of the data collected when building the method accuracy profile.     

Uncertainty profiles for the validation of analytical methods

This article aims to expose a new global strategy for the validation of analytical methods and the estimation of measurement uncertainty. Our purpose is to allow to researchers in the field of analytical chemistry get access to a powerful tool for the evaluation of quantitative analytical procedures. Indeed, the proposed strategy facilitates analytical validation by providing a decision tool based on the uncertainty profile and the β-content tolerance interval. Equally important, this approach allows a good estimate of measurement uncertainty by using data validation and without recourse to other additional experiments.In the example below, we confirmed the applicability of this new strategy for the validation of a chromatographic bioanalytical method and the good estimate of the measurement uncertainty without referring to any extra effort and additional experiments. A comparative study with the SFSTP approach [1] showed that both strategies have selected the same calibration functions.The holistic character of the measurement uncertainty compared to the total error was influenced by our choice of profile uncertainty. Nevertheless, we think that the adoption of the uncertainty in the validation stage controls the risk of using the analytical method in routine phase.     

Software support for the Nordtest method of measurement uncertainty evaluation

Software support for the Nordtest method of measurement uncertainty evaluation is described. According to the Nordtest approach, the combined measurement uncertainty is broken down into two main components??the within-laboratory reproducibility (intermediate precision) s _Rw and the uncertainty due to possible laboratory bias u(bias). Both of these can be conveniently estimated from validation and quality control data, thus significantly reducing the need for performing dedicated experiments for estimating detailed uncertainty contributions and thereby making uncertainty estimation easier for routine laboratories. An additional merit of this uncertainty estimation approach is that it reduces the danger of underestimating the uncertainty, which continues to be a problem at routine laboratories. The described software tool??MUkit (measurement uncertainty kit)??fully reflects the versatility of the Nordtest approach: it enables estimating the uncertainty components from different types of data, and the data can be imported using a variety of means such as different laboratory data systems and a dedicated web service as well as manual input. Prior to the development of the MUkit software, a laboratory survey was carried out to identify the needs of laboratories related to uncertainty estimation and other quality assurance procedures, as well as their needs for a practical tool for the calculation of measurement uncertainty.     

Alternative methodology for the analysis of progesterone, testosterone, and epi-testosterone in bovine liver and veal muscle

Research has shown that traditional solvent extraction procedures used for the analysis of endogenous steroids often give inconsistent recoveries and results. However, a single-laboratory validation of a liquid chromatography/tandem mass specrometry method using 2 product ions per transition for progesterone, testosterone, and epi-testosterone in bovine liver and veal muscle showed accuracy and precision to within 23% at concentrations ranging from 0.5 to 2.0 microg/kg. Homogenized samples were pretreated with methanol to denature endogenous enzymes. Following removal of methanol, samples were treated overnight with Helix pomatia beta-glucuronidase to deconjugate glucuronide conjugates. Alkali digestion of the samples in KOH solutions was done under shaking at 37 degrees C for 30 min. The digestate was extracted with methyl tert-butyl ether, and the extracts were cleaned by partitioning between acetonitrile-hexane, followed by solid-phase extraction cleanup on silica cartridges. In bovine liver, average recoveries exceeded 54% for all analytes, and the within-run assay coefficients of variations were < 6 and 13% for high (2.0 microg/kg) and low (0.3 microg/kg) analyte concentrations, respectively. In veal muscle, average recoveries exceeded 60%, and the analysis of blind spikes gave accuracy estimates of over 85%, with coefficients of variation (CVs) < 15% for all analytes. The CVs for the multiple reaction monitoring ion ratios for all compounds were < 22% for all validation data. The method meets the requirements for confirmatory methods as outlined in 2002/657/EC. An analyst is capable of processing up to 20 samples within 5 days.     

Validation of Some Procedures for Quantifying Platinum at sub‐μg/L Level in Some Real Matrices by Catalytic Adsorptive Stripping Voltammetry

The potential toxicological properties of platinum group elements, usually present in real matrices at sub‐ng/g level, explain the increasing interest in developing specific and sensitive analytical methods. In this work, a validation study was performed in order to ensure the fitness‐for‐purposes of procedures developed for quantifying platinum in tap water, beer and lettuce using an extremely sensitive method based on catalytic adsorptive stripping voltammetry. The work involved the estimation of selectivity, detection and quantitation limits, range, accuracy (trueness plus precision), uncertainty of measurement (UOM), robustness and recovery. The results are discussed in the light of the most recent literature findings.     

Measurement uncertainty of shikimic acid in red wines produced in Chile

High-performance liquid chromatography (HPLC) to determine shikimic acid is used as a complementary tool to differentiate wine varieties. In order to correctly classify, measurement uncertainty of shikimic acid by HPLC in red wine was estimated considering the following components: uncertainty associated with the preparation of shikimic acid stock solution, uncertainty associated with quantification using a calibration curve, and uncertainty associated with precision. The most important contribution to total uncertainty was the method precision. The expanded uncertainty (U) for different wine varieties was between 2.6 and 8.5%. The method was applied to determine the concentration of shikimic acid in different emerging wine varieties cultivated in Chile, such as Carmenère, Shiraz, and Pinot Noir, comparing them with classical varieties, such as Cabernet Sauvignon and Merlot. Shiraz wines presented lower shikimic acid concentrations (between 27 and 86 mg L⁻¹ with U _(k=2) = 2.6%) than Cabernet Sauvignon wines (between 41 and 142 mg L⁻¹ with U _(k=2) = 8.1%), but their concentrations were higher than found in Merlot (from 9 to 41 mg L⁻¹ with U _(k=2) = 4.3%) and Carmenère wines (between 7 and 49 mg L⁻¹ with U _(k=2) = 5.8%). Pinot Noir was the variety with the lowest concentration of this acid (7–14 mg L⁻¹ with U _(k=2) = 8.5%). Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

LC method for the determination of R-timolol in S-timolol maleate: Validation of its ability to quantify and uncertainty assessment

This article presents the validation results of a chiral liquid chromatographic (LC) method previously developed for the quantitative determination of R-timolol in S-timolol maleate samples. A novel validation strategy based on the accuracy profiles was used to select the most appropriate regression model, to assess the method accuracy within well defined acceptance limits and to determine the limits of quantitation as well as the concentration range.The validation phase was completed by the investigation of the risk profiles of various acceptable regression models in order to ensure the risk of obtaining the future measurements outside the acceptance limits fixed a priori.On the other hand, the present paper also shows how data used in this validation approach can be used to estimate the measurement uncertainty. The uncertainty derived from β-expectation tolerance interval (), which is equal to the uncertainty of measurements as well as the expanded uncertainty (U_x) using a coverage factor k = 2 was estimated. The uncertainty estimates obtained from validation data were finally compared with those obtained from interlaboratory and robustness studies.     

Some practical aspects of the validation of photometric systems for clinical analyses

 It is well known that erroneous data reported to a physician may strongly affect medical decision making. For routine clinical chemistry purposes, different instrumentation can be used to compare measurements of unknown samples with standard reference materials. Currently, acceptable limits of accuracy and precision are poorly defined in the field of clinical chemistry laboratories. In this article, problems associated with spectrophotometric measurements, both manual and automated, are discussed. The task of the validation of photometric systems for clinical analyses is currently of considerable interest. Some practical aspects of this validation and the use of reference materials for this activity in the national area are discussed. Received: 7 November 1996 Accepted: 14 January 1997     

Rapid method for determination of protein content in cereals and oilseeds: validation, measurement uncertainty and comparison with the Kjeldahl method

The objective of this research was to test suitability of the Dumas combustion method to completely substitute the Kjeldahl method in routine laboratory determination of crude protein content in cereals and oilseeds. The validation of the method demonstrated that it is able to determine crude protein content in cereals and oilseeds in an efficient and accurate manner, with a detection limit w(N) = 0.006%, quantification limit w(N) = 0.019%, repeatability precision RSD _r = 0.41%, intra-laboratory reproducibility precision RSD _R = 0.74%, trueness, expressed in terms of bias b = 0.43%, and linear response between (2.36–19.2) mg N. Measurement uncertainty, expressed as relative expanded uncertainty (coverage factor k = 2, confidence level 95%), was calculated from validation data (U _rel = 2.24%). In order to examine the relationship between two methods, 15 cereal grain and oilseed samples were analyzed using Dumas and Kjeldahl procedure. The Kjeldahl procedure gave slightly lower w(N) values than the Dumas procedure: w _K(N) = 0.9905 w _D(N) = 0.0376 (R ²  = 0.9996). Relative standard deviations and results of homogeneity test obtained during analysis of complex cereal products (cereal breakfast and muesli bars) show that the Dumas combustion method may be less suitable for analysis of such samples compared to Kjeldahl method.     

The evaluation of measurement uncertainty from method validation studies

 A protocol has been developed illustrating the link between validation experiments, such as precision, trueness and ruggedness testing, and measurement uncertainty evaluation. By planning validation experiments with uncertainty estimation in mind, uncertainty budgets can be obtained from validation data with little additional effort. The main stages in the uncertainty estimation process are described, and the use of trueness and ruggedness studies is discussed in detail. The practical application of the protocol will be illustrated in Part 2, with reference to a method for the determination of three markers (CI solvent red 24, quinizarin and CI solvent yellow 124) in fuel oil samples. Received: 10 April 1999 / Accepted: 24 September 1999     

Estimating Reactivity Ratios From Triad Fraction Data

Reactivity ratio estimation is a non-linear estimation problem. Typically, reactivity ratios are estimated using the instantaneous copolymer composition equation, otherwise known as the Mayo-Lewis model, based on low conversion (<5%) copolymer composition data. However, there are other instantaneous models, which can be used to estimate reactivity ratios, such as the instantaneous triad fraction equations. The aim of this paper is to determine the potential improvement in reactivity ratio estimates when triad fraction data is used in place of and in combination with copolymer composition data. The interest in using triad fraction data in parameter estimation, stems from the fact that there are a greater number of responses measured (six triad fractions) compared to composition leading to data with theoretically more information content. In principle this should lead to reactivity ratio estimates having less uncertainty. In this study, the parameter estimates are obtained by employing the error in variables model (EVM), assuming a multiplicative error structure. Several case studies involving published literature data for different copolymer systems are presented. As the case studies demonstrate in general more precise estimates can be obtained from triad fraction data. Combining the triad fraction with composition data leads to little additional improvement. However, discrepancies arise between reactivity ratios estimated from composition data compared with those obtained from triad fraction data depending upon the copolymer system. Those copolymer systems exhibiting more heterogeneity due to phase separation during polymerization may be showing more discrepancy.