Comparability of measurement results for pesticide residues in foodstuffs: an open issue?

The dispersion of results from proficiency tests for the analysis of pesticide residues in foodstuffs suggests that improvements in the compatibility of measurement results are needed. Currently observed divergences can make the evaluation conclusion on foodstuffs compliance with certain legislation dependent on the consulted laboratory. This work discusses the origin and impact of this lack of compatibility, following the metrological concepts presented at the latest version of the “International Vocabulary of Metrology” (VIM3), thus allowing for a clear diagnostic of the problem. The reporting of results from different measurement methods uncorrected for the observed analyte recovery makes them traceable to different “operationally defined measurement procedures” (VIM3) and, therefore, not comparable. When results from different measurement methods are reported corrected for analyte recovery, R, and R is different for spiked and incurred residues, measurement results may be not compatible if this effect is not considered on the uncertainty budget. This discussion is illustrated with metrological models for any possible combination of “measurement performance” and “maximum residue level”. These models are complemented with experimental data of the analysis of pesticide residues in a sample of ginseng powder from a proficiency test. The adopted experimental design allowed the identification of additional threats to metrological compatibility in this field. Solutions to the faced problem are discussed for practicability and impact on regulatory issues. The use of a universal “reference measurement procedure” proves to be the most feasible way of ensuring comparability of measurements in this field.     

Laboratory assessment by combined z score values in proficiency tests: experience gained through the European Union proficiency tests for pesticide residues in fruits and vegetables

The obligation for accredited laboratories to participate in proficiency tests under ISO 17025, performing multiresidue methods (MRMs) for pesticide residues, involves the reporting of a large number of individual z scores making the evaluation of the overall performance of the laboratories difficult. It entails, time and again, the need for ways to summarise the laboratory’s overall assessment into a unique combined index. In addition, the need for ways to continually evaluate the performance of the laboratory over the years is equally acknowledged. For these reasons, following 14 years of the European Union Reference Laboratory for Pesticide Residues in Fruits and Vegetables (EUPT-FV), useful formulas have been designed to globally evaluate the assessment of the participating laboratories. The aim is to achieve a formula which is easy to understand, which can be applied and which fits the purposes of long-term evaluation detecting positive and negative trends. Moreover, consideration is needed for a fair compensation of bad results in MRM, taking into account the large number of compounds that are covered. It is therefore important to be aware of the difficulties in getting satisfactory values from a wide range of compounds. This work presents an evaluation of the main well-established combined z score formulas together with those new ones developed here which have been applied to the European proficiency test results (EUPTs) over the years. Previous formulas such as the rescaled sum of z score (RSZ), the sum squared of z score (SSZ) and the relative laboratory performance (RLP) are compared with the newer ones: the sum of weighted z scores (SWZ) and the sum of squared z scores (SZ2). By means of formula comparisons, conclusions on the advantages, drawbacks and the most fit-for-purpose approach are achieved.     

Proficiency tests using four batches of green alga with controlled levels of cadmium

A set of test materials of the green alga Chlorella vulgaris with different levels of naturally bound Cd and about the same levels of other 13 essential or trace elements (BIOMA 1–4) was produced for proficiency testing of laboratories involved in elemental analysis of food. Criteria of the laboratories’ performance, such as the z-score and combination scores RSZ and SSZ were evaluated and discussed in terms of their suitability for proficiency testing using a set of test materials with the same matrix. The use of the test materials as a set of internal reference materials has also been suggested. Received: 23 May 1997 / Revised: 1 August 1997 / Accepted: 11 August 1997     

Proficiency testing on the analytical control of precious metal production

Our experience of executing proficiency testing programs on the analysis of precious metals and precious metal bearing materials is summarized. Six proficiency testing programs of the ”distributed sample testing” type were carried out using samples essentially distinct from each other: (I) certified samples of fine gold and fine silver, and (II) recoverable raw (scrap and catalyst waste) containing precious metals of unknown contents. The test results were assessed using the London Bullion Market Association (LBMA) rules and by Z-criteria using the ISO/IEC Guide 43 scale. Satisfactory results were presented by all participants for the analysis of certified samples of fine gold and fine silver. The results of catalyst waste analysis agreed rather well, whereas the scrap analysis revealed an essential discrepancy in the data for some samples. The reasons for such discrepancies are discussed. Received: 9 June 2000 Accepted: 21 November 2000     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

The determination of an accepted reference value from proficiency data with stated uncertainties

Proficiency data with stated uncertainties represent a unique opportunity for testing that the reported uncertainties are consistent with the Guide to the expression of uncertainty in measurement (GUM). In most proficiency tests, however, this opportunity is forfeited, because proficiency data are processed without regard to their uncertainties. In this paper we present alternative approaches for determining a reference value as the weighted mean of all mutually consistent results and their stated uncertainties. Using an accepted reference value each reported uncertainty estimate can be expressed as an E _n number, but a value of confirms its validity only if the uncertainty of the reference value is negligible in comparison.Reference values calculated for results from an International Measurement Evaluation Programme (IMEP-9) by “bottom up” as well as “top down” methods were practically identical, although the first strategy yielded the lowest uncertainty. A plot of individual coefficients of variation (CV) versus E _n numbers helps interpretation of the proficiency data, which could be used to validate relative uncertainties down to <1%.     

Capabilities of laboratories to determine melamine in food—results of an international proficiency test

A proficiency test to assess the capabilities of laboratories to determine melamine in a milk powder and a baking mix, representing starch-containing foods like bread and biscuits, was carried out in January 2009. The need for such an interlaboratory comparison arose from a health scare in China about melamine-tainted powdered milk in the second half of 2008. Laboratories in 31 countries, including Australia, China, India, Japan, New Zealand and the USA, and 21 of the 27 Member States of the European Union participated and reported back 114 results for the milk powder and 112 for the baking mix test materials. The reported results were compared to reference values determined by exact-matching double isotope dilution mass spectrometry. The so-determined assigned values were 10.0 ± 0.6 mg/kg melamine in the milk powder and 3.18 ± 0.17 mg/kg melamine in the baking mix. A coverage factor k of 2 was applied to calculate the expanded uncertainties. Three quarters of all reported results for both materials had associated z scores which were satisfactory (z ≤ |2|). Of the reported results, 90% was accompanied by a measurement uncertainty statement, and the majority of the measurement uncertainty ranges were reasonable. A number of laboratories were found to underestimate their measurement uncertainties. Methods that involved the use of stable-isotope-labelled melamine were shown to be clearly advantageous with regard to the accuracy of the results. However, no significant influence by other method parameters could be identified.     

Determination of POPs in environmental matrices – proficiency tests for Polish laboratories

Thirty Polish laboratories participated in proficiency tests (PT) for determination of persistent organic pollutants – POPs (PCBs – polychlorinated biphenyls, PAHs – polyaromatic hydrocarbons and pesticides) in solid environmental matrices (sediments, soil). Participants used their in-house method. The aim of the exercises was to obtain information about the “state of the art” of POPs analysis in environmental samples. The CV (coefficient of variation) of analytical results submitted varied from 45 to 65%, with an outlier rate 2 to 7%. The averages of results for all analytes were in good agreement with reference values. Statistical treatment of results was made. Comparison of the each result with the certified values characterizing the material has been done. Additionally, z-scores were calculated taking into account the uncertainty of a certified value and the acceptable deviation from the certified value in the range of −20% to +10% (in accordance with recommendations in the newest directives of European Union (EU) [1]); For rejecting outliers Hampel test has been applied.     

A new approach in assessing microbiological results in water analysis proficiency testing

Due to the relatively large spread in the results of microbiological proficiency tests, the z-scores are often not able to detect zero or low results as being ‘bad’ results. This paper describes an adapted z-score based on the average or the standard deviation of the 50% ‘highest’ results. The combination of the adapted z-scores of four samples enables a better judgement of the performance of each laboratory. Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board or the Publisher.     

Evaluation of proficiency testing results taking into account their uncertainties

A new composite score for the evaluation of performance of proficiency testing participants is proposed. The score is based on a combination of the z-score, uncertainty of a participant’s measurement result and uncertainty of the proficiency testing scheme’s assigned value. The use of such a composite score will allow evaluation not only of the participant’s ability to determine an analyte in corresponding matrix, but also their understanding of the uncertainty in the obtained analytical result. The score may be helpful for the laboratory’s quality system and for laboratory accreditation according to ISO 17025.     

Economy of proficiency testing: reference versus consensus values

Many proficiency tests are operated with a consensus value derived from the participants’ results. Apart from technical issues, one of the reasons often mentioned is that proficiency tests operated with consensus values would be cheaper than those using reference values obtained from a priori characterisation measurements. The economy of a proficiency test must of course be balanced by the need of the participants, and the quality of the comparison in general. The proficiency tests selected in this study had both a reference value and a consensus value, one of which was used for assessing the performance of the participating laboratories. In this work, both a technical and an economical assessment of how the comparisons were operated is made. From the evaluation, it follows that usually the use of consensus values does not necessarily reduce the costs of a proficiency test. However, frequently it may be observed that the quality of the assessment of the laboratories is better with a reference value. Received: 11 October 2000 Accepted: 3 January 2001     

Evaluation of the use of consensus values in proficiency testing programmes

Proficiency testing (PT) is an essential tool used by laboratory accreditation bodies to assess the competency of laboratories. Because of limited resources of PT providers or for other reasons, the assigned reference value used in the calculation of z-score values has usually been derived from some sort of consensus value obtained by central tendency estimators such as the arithmetic mean or robust mean. However, if the assigned reference value deviates significantly from the ‘true value’ of the analyte in the test material, laboratories’ performance will be evaluated incorrectly. This paper evaluates the use of consensus values in proficiency testing programmes using the Monte Carlo simulation technique. The results indicated that the deviation of the assigned value from the true value could be as large as 40%, depending on the parameters of the proficiency testing programmes under investigation such as sample homogeneity, number of participant laboratories, concentration level, method precision and laboratory bias. To study how these parameters affect the degree of discrepancy between the consensus value and the true value, a fractional factorial design was also applied. The findings indicate that the number of participating laboratories and the distribution of laboratory bias were the prime two factors affecting the deviation of the consensus value from the true value.     

Performance of commercial soil laboratories in a proficiency test program in Brazil

A soil proficiency test (PT) was administered to 50 participant laboratories in which two sets of samples, consisting of 20 yearly PT samples and 5 ‘blind’ samples in clients’ names were analyzed for pH, organic matter, total acidity, extractable calcium, magnesium, potassium and phosphorus by the laboratories. Our objective was to determine whether laboratories take extra care to analyze clients’ samples as they do with regular PT samples. The analytical data were evaluated essentially by the procedure described in the international harmonized protocol for proficiency testing of analytical chemistry laboratories. Performance of participant laboratories was assessed by z-scores and summary z-scores statistics involving sum of squared z-scores interpreted as chi-square ( c_n² ) \left( {\chi_{n}^{2} } \right) distribution for zero-centered z-scores with unit variance. From 8 750 determinations, outliers and stragglers accounted for less than 2% of the entire data. Over 93% of the data were satisfactory, whereas between 2 and 4% were either unsatisfactory or questionable in both the PT and ‘blind’ tests. On the basis of sum of squared z-scores interpreted from c_n² \chi_{n}^{2} distribution table, between 30 and 40% of the laboratories had more than 90% probability of having their measurement data within the robust mean and standard deviation for each soil parameter, while another 30–42% of the laboratories had less than 50% probability of having measurement data within the robust mean and standard deviation. Overall, 21 laboratories (42%) were ranked in Class A either in the PT or ‘blind’ tests out of which 12 of them (57%) retained this ranking in both tests. Fourteen laboratories (28%) were ranked in Class C in either the PT or ‘blind’ tests with only 5 of them (36%) consistently ranking in this class in both tests.     

The proficiency testing programme for trace elements of the CORILA project

Harmonised quality schemes are being implemented in the framework of the thematic area Organization and Dissemination of Data of the Consorzio per la Gestione del Centro di Coordinamento delle Attività di Ricerca Inerenti il Sistema Lagunare di Venezia (Consortium for the Management of the Coordination Centre of Research Activities on the Venice Lagoon System, CORILA). Over the last 2 years, two proficiency tests were completed at the time of writing (while a third one is still in progress) to assess the capabilities of the analytical laboratories involved in the determination of trace elements in environmental inorganic matrices from the Lagoon of Venice.Both interlaboratorial trials were based on the determination of As, Cd, Cr, Cu, Fe, Hg, Pb and Zn in aqueous acidic, matrix-enriched solutions at concentrations within the ranges generally found in marine water and in sediment after chemical digestion, respectively. The proficiency of participants was assessed on the basis of the z score criterion (|z|≤2, acceptable results; 2<|z|<3, questionable results; |z|≥3, unacceptable results). In the two trials, the relevant percentages for the laboratories scoring |z|≤2 and |z|<3 were 61% and 71% and 65% and 77%, respectively. These first two proficiency tests, on the one hand, clearly pointed to the benefits the laboratories had from their participation in these exercises, and on the other hand, provided sound evidence of the need for further trials based on real environmental matrices.     

Is accreditation useful for quality improvement?

 The Laboratory of the Government Chemist (UK) and the Institute for Reference Measurements and Materials (Belgium) evaluated the correlation between accreditation and performance in proficiency tests. It was concluded that accreditation does not have the expected positive effect on the quality of laboratory results. In this journal discussions conducted on this subject during the CITAC workshop at Pittcon in 1998, were published. No satisfactory explanation for this phenomenon was put forward. In this article, it is proposed that the main effect of accreditation is a decrease of intralaboratory spread. The effect on the trueness of laboratory results is not significant due to the lack of certified reference materials and the low frequency of participation in proficiency tests. Proficiency tests cannot be used to find a correlation between accreditation status and quality without changing the set-up of the proficiency test.     

Comparison of “bottom-up” and “top-down” strategies for the estimation of the uncertainty in organic elemental analysis

The estimation of uncertainty in organic elemental analysis for C, H, N and S is reported. Both “bottom up” and “top down” strategies are used for uncertainty calculations. The bottom up approach used the results of C, H, N, and S obtained from the homogeneity study of two pure chemicals (toluene-4-sulfonamide and 4(6)-methyl-2-thiouracil). Two calibration systems, K factor and calibration curve, were applied in this study and no significant differences were obtained. For the “top down” approach, we used the data obtained from a proficiency test on both pure chemicals from among 45 Spanish laboratories. Both approaches are compared and discussed below.     

A proficiency test scheme for the quality control of black tea and development of quality control material for internal quality control

A proficiency test (PT) was organized for quality control analysis of black tea. Test materials for the analyses of total content of powder tea, moisture, total ash, acid-insoluble ash, water-soluble ash, alkalinity of water-soluble ash, water extract, crude fiber and caffeine were sent to the participant laboratories (n:43) in 2009. The assigned value, standard deviation of the parameters and z-scores of the participant laboratories were calculated using the data reported by the participants. The percentage of the reported results for analytes was found to be ranged from 67.5 to 100%. Acceptable z-scores were achieved by 80.5–97.5% of the participant laboratories. More than 15% of unacceptable results were obtained for acid-insoluble ash and caffeine analyses. The quality control material for quality control analysis of black tea was developed from the remaining material.     

Dispersive solid-phase extraction for in-sorbent Fourier-transform infrared detection and identification of nerve agent simulants in analysis for verification of chemical weapon convention

The combination of dispersive solid-phase extraction (DSPE) and Fourier-transform infrared (FTIR) spectroscopy is presented for detection and quantification of markers and simulants of nerve agents. Hydrophilic–lipophilic balance (HLB) sorbent was used for extraction and enrichment of organophosphonates from water. When the extraction efficiency of DSPE was compared with that of conventional solid-phase extraction (SPE), DSPE was more efficient. Extraction conditions such as extraction time, and type and quantity of sorbent material were optimized. In DSPE, extracted analytes are detected and quantified on the sorbent using FTIR as analytical technique. Absorbance in FTIR due to P–O–C stretching was used for detection and quantification. Infrared absorbance of different analytes were compared by determining their molar absorptivities (ε _max). Quantitative analyses were performed employing modified Beer’s law, and relative standard deviations (RSDs) for intraday repeatability and interday reproducibility were found to be in the range 0.30–0.90% and 0.10–0.80% respectively. The limit of detection (LOD) was 5–10 μg mL⁻¹. The applicability of the method was tested with an unknown sample prepared by mimicking the sample obtained in an international official proficiency test.     

原子荧光法测定化妆品中砷的能力验证

在能力验证试验中,采用微波消解-原子荧光光度法测定了化妆品中的砷含量,Z值为-0.81,|Z|2,采用稳健统计"Z比分值"来评价和判定检测结果,结果满意.采用加标回收和精密度试验进行质量控制,并就如何提高试验准确度及需注意的问题进行了讨论.     

Comparability and compatibility of proficiency testing results in schemes with a limited number of participants

Comparability and compatibility of proficiency testing (PT) results are discussed for schemes with a limited number of participants (less than 20–30) based on the use of reference materials (RMs) as test items. Since PT results are a kind of measurement/analysis/test result, their comparability is a property conditioned by traceability to measurement standards applied in the measurement process. At the same time, metrological traceability of the certified value of the RM (sent to PT participants as test items) is also important, since the PT results are compared with the RM certified value. The RM position in the calibration hierarchy of measurement standards sets the degree of comparability for PT results, which can be assessed in the scheme. However, this assessment is influenced by commutability (adequacy or match) of the matrix RM used for PT and routine samples. Compatibility of PT results is a characteristic of the collective (group) performance of the laboratories participating in PT that can be expressed as closeness of the distribution of the PT results to the distribution of the RM data. Achieving quality-of-measurement/analysis/test results in the framework of the concept “tested once, accepted everywhere” requires both comparability and compatibility of the test results.