A simple method of constructing a confidence interval for the mean probability of detection in collaborative studies of binary measurement methods

We deal with collaborative studies where each of k laboratories performs n repeated binary measurements (measurement result x = 0: “not detected”; measurement result x = 1: “detected”), and present a simple method of constructing a confidence interval for the mean probability of detection of the laboratories. This method is based on an approximation of the distribution of the number y of detections among n independent measurements of a randomly chosen laboratory by a binomial distribution. The confidence interval is not only much easier to calculate but also more accurate than the profile likelihood interval presented by Uhlig et al.     

Multiparameter models and statistical uncertainties

When more than one parameter is found by a least-squares calculation, the statistical uncertainties of the parameters are generally interdependent. If the uncertainty of one such parameter is quoted as a confidence interval based on the standard error estimate of that parameter and the Student t-statistic, this interval tends to be an underestimate. A suggestion is made to quote more conservative parameter uncertainties as the extreme points on the 95% joint confidence ellipsoid and it is shown that these joint parametric uncertainties are easily calculated from the standard error estimates. Both linear and nonlinear multiple regression are discussed. Nonlinear parameter uncertainties are found after an iterative search for the minimum sum-of-squares of residuals; searches by the Gauss and simplex methods are considered. A joint parametric uncertainty calculation is illustrated by a four-parameter nonlinear regression involving a pH potentiometric titration.     

A DANREF certified reference plastic for measurement of overall migration into the food simulant olive oil by single sided testing

A reference material for the determination of overall migration from a plastic coextrudate into the fatty food simulant olive oil was produced and certified in an interlaboratory study. The analyses were carried out according to the ENV 1186 standard from the European Committee for Standardization (CEN) [1, 2, 3] with exposure of the coextrudate to olive oil for 10 days at 40?°C. After an initial preliminary interlaboratory study eight laboratories participated in the certification round, and two different methods were used to obtain single sided exposure of the plastic to the oil. The certified value was determined as the mean of laboratory mean values. No outliers were found. A reference value of 8.6 mg/dm²± 1.4 mg/dm² (± half width of the 95% confidence interval) was obtained which is within the range relevant for the regulatory limit (10 mg/ dm²), making this reference material suitable for laboratories measuring according to the EU overall migration limit [4]. The material has been found stable over 45 months.     

Assessment of quality performance parameters for straight line calibration curves related to the spread of the abscissa values around their mean

In validation of quantitative analysis methods, knowledge of the response function is essential as it describes, within the range of application, the existing relationship between the response (the measurement signal) and the concentration or quantity of the analyte in the sample. The most common response function used is obtained by simple linear regression, estimating the regression parameters slope and intercept by the least squares method as general fitting method. The assumption in this fitting is that the response variance is a constant, whatever the concentrations within the range examined.The straight calibration line may perform unacceptably due to the presence of outliers or unexpected curvature of the line. Checking the suitability of calibration lines might be performed by calculation of a well-defined quality coefficient based on a constant standard deviation.The concentration value for a test sample calculated by interpolation from the least squares line is of little value unless it is accompanied by an estimate of its random variation expressed by a confidence interval. This confidence interval results from the uncertainty in the measurement signal, combined with the confidence interval for the regression line at that measurement signal and is characterized by a standard deviation s_x0 calculated by an approximate equation. This approximate equation is only valid when the mathematical function, calculating a characteristic value g from specific regression line parameters as the slope, the standard error of the estimate and the spread of the abscissa values around their mean, is below a critical value as described in literature.It is mathematically demonstrated that with respect to this critical limit value for g, the proposed value for the quality coefficient applied as a suitability check for the linear regression line as calibration function, depends only on the number of calibration points and the spread of the abscissa values around their mean.     

Working Methods Paper: Certification of methylmercury compounds concentration in marine sediment reference material,IAEA-356

An intercomparison exercise was organized between seven laboratories using various isolation procedures (extraction, distillation, ion-exchange and alkaline digestion) and detection systems (CV AAS, cold vapour atomic absorption spectroscopy; CV AFS, cold vapour atomic fluorescence spectroscopy; GC, ECD, gas chromatography electron capture detector and HPLC with CV AFS detection) for determination of methylmercury compounds in sediment sample. All certification criteria were fulfilled and therefore the value for total concentration of methylmercury compounds was certified to be 5.46 ng g^?1, with a 95% confidence interval from 4.07–5.84 ng g^?1. The acceptable range, calculated as two times the confidence interval of the mean is therefore from 4.68–6.23 ng g^?1. This is the first sediment reference material ever to be certified for concentration of methylmercury compounds. Comparison of the data obtained by various methodologies has shown that the most critical step is the isolation of methylmercury compounds from binding sites. Acid leaching only cannot release methylmercury compounds quantitatively. Total release of methylmercury compounds could only be achieved by alkaline digestion or distillation. This simple intercomparison exercise has shown that since large numbers of laboratories world-wide are performing methylmercury compound analyses using various improved and specific separation methods and sensitive detection systems, certification of methylmercury compounds in different biological and environmental samples should not be a problem in the future.     

A DANREF certified reference plastic for measurement of overall migration into the food simulant olive oil by single sided testing

A reference material for the determination of overall migration from a plastic coextrudate into the fatty food simulant olive oil was produced and certified in an interlaboratory study. The analyses were carried out according to the ENV 1186 standard from the European Committee for Standardization (CEN) [ 1, 2, 3] with exposure of the coextrudate to olive oil for 10 days at 40 degrees C. After an initial preliminary interlaboratory study eight laboratories participated in the certification round, and two different methods were used to obtain single sided exposure of the plastic to the oil. The certified value was determined as the mean of laboratory mean values. No outliers were found. A reference value of 8.6 mg/dm2 +/- 1.4 mg/dm2 (+/- half width of the 95% confidence interval) was obtained which is within the range relevant for the regulatory limit (10 mg/ dm2), making this reference material suitable for laboratories measuring according to the EU overall migration limit [4]. The material has been found stable over 45 months.     

Recent certification of reference materials by the Community Bureau of Reference for,in particular,environmental analysis

Reference materials are necessary to improve or to maintain a high quality of analysis. To avoid bias as far as possible, the BCR certifies reference materials on the basis of results of different methods used by different laboratories. A proper selection of good methods and laboratories makes it possible to obtain excellent agreement and narrow confidence intervals. Neutron activation techniques are suited for many key elements; their results are in good agreement with those of other techniques.     

Auswertung eines Ringversuches im Spurenbereich

Summary The evaluation of an interlaboratory comparison in trace analysis has to take account of the log-normal distribution of the results. This distribution leads to the geometric mean as a measure for the content of the investigated sample, and to a relative error for the confidence interval for the reproducibility and the repeatibility. The log-normal distribution allows to pool together the relative standard deviations from the different components of the sample. In this way the confidence interval gets narrower and the criteria for reproducibility and repeatibility can be improved, even when there was an insufficient number of laboratories taking part in the interlaboratory comparison.     

Statistical analysis of measured actinide concentration bias in a Mixed Analyte Performance Evaluation Program

Data from seven Mixed Analyte Performance Evaluation Program studies have been used to identify specific problems with results and/or laboratories. These data have never been used to determine the bias or variability in data reporting for these laboratories in general. These historical analytical data were analyzed for accuracy and precision overall and separated by method of sample preparation, method of sample pretreatment, method of detection and sample size. The calculated mean percent relative bias was analyzed for the actinides (²⁴¹Am, ²³⁸Pu, ^239/240Pu, ^234/233U, and ²³⁸U) in the soil and water standards. Results from most analysis methods were at least slightly negatively biased. For the water standards, the overall 95% confidence interval for the mean percent relative bias was −1.6% to −0.5% indicating that the average actinide results slightly underestimated the true concentration. For the soil standards, the overall 95% confidence interval for the mean percent relative bias was −8.1% to −6.6% indicating that the average actinide results moderately underestimated the true concentration. This revised version was published online in July 2006 with corrections to the Cover Date.     

Is the assessment of interlaboratory comparison results for a small number of tests and limited number of participants reliable and rational?

Tests and/or test items can sometimes be expensive, unique, or only performed in a few laboratories. There can be cases where assigned values are unknown, there is no information, or only poor information on the probability density function attributed to the test result. Sometimes there are neither reference materials nor the ability to establish consensus values due to a lack of experts. It can be impossible to repeat a test on the same item because it is destroyed during the test itself, or the homogeneity of tested items is unknown and no criteria can be established. Specified technical requirements concerning proficiency testing and interlaboratory comparison schemes are generally not applicable in this situation. However, interlaboratory comparison could allow laboratories to have more confidence in their results. The present paper discusses three statistical methods of assessing interlaboratory comparison results obtained in such conditions. Two methods are based on an assigned value determined from participant results through robust analysis. The third is based on the compatibility of results assessed using the ζ parameter. This paper focuses on an interlaboratory comparison for two laboratories, each testing three samples. The use of statistical methods turns out to be high risk, particularly in terms of falsely accepting results. Additionally, is shown that methods dedicated to small samples are also not efficient in detecting discrepancies of test results.     

The influence of experience on the quality of chemical measurements made on soils from former gasworks

 A strategy for ensuring that measurements of ten common contaminants found in soils from former gasworks are of optimum quality over long timescales is described. In particular, the performance of laboratories in applying validated measurement methods in both small- and large-scale inter-laboratory trials over a 5-year period is assessed. It is shown that a production engineering concept, the experience curve, has a significant influence on the quality of some of the measurements made and that this concept can be used to model the performance of laboratories. Complexity of the measurement procedure and familiarity of analysts with the measurement operations are suggested as the main drivers of method error. The implications of the achievable quality of measurement on the costs of land re-development and on confidence in the quality of the rejuvenated land are discussed. Received: 31 August 1996 Accepted: 5 February 1997     

“Demonstration” vs. “designation” of measurement competence: the need to link accreditation to metrology

Formal acceptance of the results of chemical laboratories is increasingly organized through a) accreditation of measuring laboratories nationally and b) mutual recognition of accreditation internationally (through formal Multilateral Recognition Agreements, MRAs). However, real comparability of results of measurements is realized by using common (internationally agreed) measurement scales which make these results traceable to this scale, i.e. “traceable” to the same (internationally agreed) value of the unit of that scale. In addition, the criterion against which the evaluation is done, should be “external” to the measurement laboratories which are being evaluated. This is realized in IRMM’s International Measurement Evaluation Programme (IMEP) where evaluation is performed against values which are anchored using “metrology”, the science of measurement with its own rules, which offers a sound foundation for measurement in all scientific disciplines. It is argued in this paper that the demonstration of measurement capability against values on such scales provides a result-oriented rather than a procedure-oriented evaluation. Thus, competence can be “demonstrated” rather than just “designated” and this can be shown to both customers and regulators. It inspires more confidence.     

Certification of trace and major elements and methylmercury concentrations in a macroalgae (Fucus sp.) reference material, IAEA-140

A marine reference material, IAEA-140, prepared with a macroalgae (Fucus sp.) was recently produced by the International Atomic Energy Agency and certified for trace and major elements and for methylmercury (MeHg). Certification of this material was achieved as an outcome of an international analytical intercomparison study which resulted in 116 independent sets of results reported by participants from 54 countries. The statistical evaluation of the collected data and the criteria used for assignment of the mean and uncertainty values are described. The analysis of data allowed to certify concentration values for 24 elements and MeHg, and to provide information values for another 10 elements. Regarding the elements which could be given certified values, between two and eight different instrumental methods were used to measure metal concentrations, and four independent analytical procedures were used to measure MeHg concentrations. In order to assess the results of the certification procedure, a comparison was made between the certified values obtained from the world-wide intercomparison results and the values obtained from a small subgroup of well-qualified laboratories. The means and 95% confidence intervals for reference values obtained by the two methods are similar for most elements showing that the usual method of certification used by the IAEA, based on large groups of participants, is indeed pertinent.     

A global approach to method validation and measurement uncertainty

The enforcement of legal limits for food safety raises the question of decision-making in the context of uncertain measurements. It also puts the question of demonstrating that measurement technique that is used is fit for the purpose of controlling legal limits. A recent European Commision (EC) decision gives some indications how to deal with this question. In the meantime, the implementation of quality systems in analytical laboratories is now a reality. While these requirements deeply modified the organization of the laboratories, it has also improved the quality of the results. The goal of this communication is to describe how two fundamental requirements of ISO 17025 standard, i.e. validation of the methods and estimation of the uncertainty of measurements, can give a way to check whether an analytical method is correctly fit for the purpose of controlling legal limits. Both these requirements are not independent and it will be shown how they can be combined. A recent approach based on the “accuracy profile” of a method was applied to the determination of acrylamide and illustrates how uncertainty can be simply derived from the data collected for validating the method. Moreover, by basing on the β-expectation tolerance interval introduced by Mee [Technometrics (1984) 26(3): 251–253], it is possible to unambiguously demonstrate the fitness for purpose of a method. Remembering that the expression of uncertainty of the measurement is also a requirement for accredited laboratories, it is shown that the uncertainty can be easily related to the trueness and precision issuing from the data collected to build the method accuracy profile. The example presented here consists in validating a method for the determination of acrylamide in pig plasma by liquid chromatography–mass spectromery (LC–MS). Concentrations are expressed as mg/l and instrumental response is peak surface. The calibration experimental design included 5×5×2 measurements and namely consisted in preparing duplicate standard solutions at five concentration levels ranging from 10 to about 5000 mg/l. This was repeated for 5 days. The validation experimental design was similar.     

Comparison of multianalyte proficiency test results by sum of ranking differences, principal component analysis, and hierarchical cluster analysis

Sum of ranking differences (SRD) was applied for comparing multianalyte results obtained by several analytical methods used in one or in different laboratories, i.e., for ranking the overall performances of the methods (or laboratories) in simultaneous determination of the same set of analytes. The data sets for testing of the SRD applicability contained the results reported during one of the proficiency tests (PTs) organized by EU Reference Laboratory for Polycyclic Aromatic Hydrocarbons (EU-RL-PAH). In this way, the SRD was also tested as a discriminant method alternative to existing average performance scores used to compare mutlianalyte PT results. SRD should be used along with the z scores—the most commonly used PT performance statistics. SRD was further developed to handle the same rankings (ties) among laboratories. Two benchmark concentration series were selected as reference: (a) the assigned PAH concentrations (determined precisely beforehand by the EU-RL-PAH) and (b) the averages of all individual PAH concentrations determined by each laboratory. Ranking relative to the assigned values and also to the average (or median) values pointed to the laboratories with the most extreme results, as well as revealed groups of laboratories with similar overall performances. SRD reveals differences between methods or laboratories even if classical test(s) cannot. The ranking was validated using comparison of ranks by random numbers (a randomization test) and using seven folds cross-validation, which highlighted the similarities among the (methods used in) laboratories. Principal component analysis and hierarchical cluster analysis justified the findings based on SRD ranking/grouping. If the PAH-concentrations are row-scaled, (i.e., z scores are analyzed as input for ranking) SRD can still be used for checking the normality of errors. Moreover, cross-validation of SRD on z scores groups the laboratories similarly. The SRD technique is general in nature, i.e., it can be applied to any experimental problem in which multianalyte results obtained either by several analytical procedures, analysts, instruments, or laboratories need to be compared.

Certification of total mercury and methylmercury concentrations in mussel homogenate (Mytilus edulis) reference material, IAEA-142

Due to the increased demand for new reference materials certified for total and methylmercury (MeHg) a sample of mussel homogenate (IAEA-142) has been prepared. Thirteen experienced laboratories reported results for total Hg of which 9 laboratories also reported results for MeHg content. Laboratories reporting MeHg results used various isolation techniques (solvent extraction, saponification, acid leaching, ion-exchange separation, and distillation) and detection systems (cold vapour atomic absorption spectrometry (CV AAS), cold vapour atomic fluorescence spectrometry (CV AFS), gas chromatography with electron capture detector (GC/ECD) and HPLC with CV AAS detector). In the case of total Hg, most of the laboratories used acid digestion, only two used alkaline dissolution, followed either by CV AAS or CV AFS. One laboratory used neutron activation analyses with radiochemical separation. The data received were in good agreement. The value for total Hg was certified to be 126 ng/g, with a 95% confidence interval from 119 to 132 ng/g. For MeHg the certified value of 47 ng/g expressed as Hg was assigned, with a 95% confidence interval from 43 to 51 ng/g. Stability testing has shown that both total and MeHg are stable if samples are stored in a dry and dark place at room temperature. The sample is now available as a certified reference material and is, in particular, useful for quality control measurements of Hg and MeHg in mussel samples at low concentration levels.     

Testing equivalence between two laboratories or two methods using paired-sample analysis and interval hypothesis testing

A modified interval hypothesis testing procedure based on paired-sample analysis is described, as well as its application in testing equivalence between two bioanalytical laboratories or two methods. This testing procedure has the advantage of reducing the risk of wrongly concluding equivalence when in fact two laboratories or two methods are not equivalent. The advantage of using paired-sample analysis is that the test is less confounded by the intersample variability than unpaired-sample analysis when incurred biological samples with a wide range of concentrations are included in the experiments. Practical aspects including experimental design, sample size calculation and power estimation are also discussed through examples.     

Certified reference material IAEA-418: 129I in Mediterranean Sea water

A certified reference material designed for the determination of ¹²⁹I in seawater, IAEA-418 (Mediterranean Sea water) is described and the results of certification are presented. The median of ¹²⁹I concentration with 95% confidence interval was chosen as the most reliable estimates of the true value. The median, given as the certified value, is 2.28 × 10⁸ atom L^?1 (95% confidence interval is (2.16–2.73) 10⁸ atom L^?1), or 3.19 × 10^?7 Bq L^?1 (95% confidence interval is (3.02–3.82) × 10^?7 Bq L^?1). The material is intended to be used for standardization procedures applied in accelerator mass spectrometric laboratories. It is available in 1 L units and may be ordered via IAEA web side (www.iaea.org).     

Proficiency testing schemes for the assessment of Legionella PCR methodologies

Standard operating procedures used for the detection of bacteria in environmental samples are primarily based on bacterial growth on specific culture media and confirmation by biochemical and/or immunological tests. In the case of Legionella, isolation on BCYE-?? medium is the standard method, although it presents a number of drawbacks, and for this reason, the implementation of molecular methods, mainly those based on PCR, has increased over the last years. Following the ISO/IEC 17025, laboratories need an external evaluation of their work to assure the quality of the results they are producing, and the participation in proficiency testing (PT) schemes is compulsory. For those water-testing laboratories using PCR methods for Legionella, we have developed a PT scheme accredited according to ISO/IEC 17043. The preparation and the statistical analysis of the results are performed following this standard and the ISO 13528. The used samples have a very rapid and easy to use format, consisting of tablets with inactivated freeze-dried Legionella cells or freeze-dried Legionella DNA. In this PT scheme, participants evaluate both Legionella pneumophila and Legionella spp. detection systems and control the whole PCR process from the water sample concentration until the PCR results.     

Certification of total mercury and methylmercury concentrations in mussel homogenate (Mytilus edulis) reference material, IAEA-142

Due to the increased demand for new reference materials certified for total and methylmercury (MeHg) a sample of mussel homogenate (IAEA-142) has been prepared. Thirteen experienced laboratories reported results for total Hg of which 9 laboratories also reported results for MeHg content. Laboratories reporting MeHg results used various isolation techniques (solvent extraction, saponification, acid leaching, ion-exchange separation, and distillation) and detection systems (cold vapour atomic absorption spectrometry (CV AAS), cold vapour atomic fluorescence spectrometry (CV AFS), gas chromatography with electron capture detector (GC/ECD) and HPLC with CV AAS detector). In the case of total Hg, most of the laboratories used acid digestion, only two used alkaline dissolution, followed either by CV AAS or CV AFS. One laboratory used neutron activation analyses with radiochemical separation. The data received were in good agreement. The value for total Hg was certified to be 126 ng/g, with a 95% confidence interval from 119 to 132 ng/g. For MeHg the certified value of 47 ng/g expressed as Hg was assigned, with a 95% confidence interval from 43 to 51 ng/g. Stability testing has shown that both total and MeHg are stable if samples are stored in a dry and dark place at room temperature. The sample is now available as a certified reference material and is, in particular, useful for quality control measurements of Hg and MeHg in mussel samples at low concentration levels. Received: 24 September 1996 / Revised: 20 November 1996 / Accepted: 8 December 1996