Evaluation of results of an interlaboratory comparison test on determination of acrylamide in crispbread samples

The European Commission's Directorate General Joint Research Centre has organized several proficiency tests on the determination of acrylamide (AA) in food. This paper presents the results and outcome of a proficiency test that focused on the determination of AA in crispbread samples. One of the goals was the identification of the influence of different parameters such as analyte extraction or instrument calibration on the analytical results. A set of samples, containing 3 different crispbread samples as well as extracts of one crispbread sample and AA standard solutions, was shipped to each participant. A total of 42 European laboratories reported analytical results that were evaluated by applying internationally accepted protocols and procedures. The study found that, for each sample, the results of 4-8 laboratories were outside the range formed by the target value plus or minus the 2-fold of the target standard deviation; thus, they did not perform satisfactorily. In transferring this knowledge to the data of monitoring databases of AA in food, care must be taken that data are quality controlled, as it is likely that some of them may be biased.     

Performance of commercial soil laboratories in a proficiency test program in Brazil

A soil proficiency test (PT) was administered to 50 participant laboratories in which two sets of samples, consisting of 20 yearly PT samples and 5 ‘blind’ samples in clients’ names were analyzed for pH, organic matter, total acidity, extractable calcium, magnesium, potassium and phosphorus by the laboratories. Our objective was to determine whether laboratories take extra care to analyze clients’ samples as they do with regular PT samples. The analytical data were evaluated essentially by the procedure described in the international harmonized protocol for proficiency testing of analytical chemistry laboratories. Performance of participant laboratories was assessed by z-scores and summary z-scores statistics involving sum of squared z-scores interpreted as chi-square ( c_n² ) \left( {\chi_{n}^{2} } \right) distribution for zero-centered z-scores with unit variance. From 8 750 determinations, outliers and stragglers accounted for less than 2% of the entire data. Over 93% of the data were satisfactory, whereas between 2 and 4% were either unsatisfactory or questionable in both the PT and ‘blind’ tests. On the basis of sum of squared z-scores interpreted from c_n² \chi_{n}^{2} distribution table, between 30 and 40% of the laboratories had more than 90% probability of having their measurement data within the robust mean and standard deviation for each soil parameter, while another 30–42% of the laboratories had less than 50% probability of having measurement data within the robust mean and standard deviation. Overall, 21 laboratories (42%) were ranked in Class A either in the PT or ‘blind’ tests out of which 12 of them (57%) retained this ranking in both tests. Fourteen laboratories (28%) were ranked in Class C in either the PT or ‘blind’ tests with only 5 of them (36%) consistently ranking in this class in both tests.     

Performance evaluation of polycyclic aromatic hydrocarbons analysis in sediment from proficiency testing with metrological reference values

This paper describes a metrological approach to evaluate the measurement capability of laboratories participating in two proficiency testing (PT) programmes involving the analysis of five polycyclic aromatic hydrocarbons (PAHs) in sediment samples. Reference values of PAHs in the programmes for performance assessment were obtained from an accurate isotope dilution gas chromatography mass spectrometry (ID-GCMS) method which was thoroughly validated and verified. Isotope dilution mass spectrometry (IDMS) technique usually has a well-defined measurement uncertainty budget and a traceability link to an International System of Units. Provision of the metrological reference values in PT enables the establishment of a technical platform to assess the actual competence of the participating laboratories in sediment PAHs analysis. Results of the PT programmes showed that about 80 % of the laboratories employed gas chromatography in their analyses and the remaining used liquid chromatography. Irrespective of the techniques being used, however, the majority of the participating laboratories were observed to underestimate values in which the mean values of the five reported PAHs were less than those of the ID-GCMS-derived reference values by 13–20 %. Only 41–44 % of the participating laboratories were able to achieve satisfactory z-scores. The present study revealed that the reinforcement of the capability for accurate measurement of PAHs in sediment samples in laboratories worldwide should be addressed.     

A proficiency test scheme for the quality control of black tea and development of quality control material for internal quality control

A proficiency test (PT) was organized for quality control analysis of black tea. Test materials for the analyses of total content of powder tea, moisture, total ash, acid-insoluble ash, water-soluble ash, alkalinity of water-soluble ash, water extract, crude fiber and caffeine were sent to the participant laboratories (n:43) in 2009. The assigned value, standard deviation of the parameters and z-scores of the participant laboratories were calculated using the data reported by the participants. The percentage of the reported results for analytes was found to be ranged from 67.5 to 100%. Acceptable z-scores were achieved by 80.5–97.5% of the participant laboratories. More than 15% of unacceptable results were obtained for acid-insoluble ash and caffeine analyses. The quality control material for quality control analysis of black tea was developed from the remaining material.     

Evaluation of the performance of 57 Japanese participating laboratories by two types of z-scores in proficiency test for the quantification of pesticide residues in brown rice

A proficiency test for the analysis of pesticide residues in brown rice was carried out to support upgrading in analytical skills of participant laboratories. Brown rice containing three target pesticides (etofenprox, fenitrothion, and isoprothiolane) was used as the test samples. The test samples were distributed to the 57 participants and analyzed by appropriate analytical methods chosen by each participant. It was shown that there was no significant difference among the reported values obtained by different types of analytical method. The analytical results obtained by National Metrology Institute of Japan (NMIJ) were 3 % to 10 % greater than those obtained by participants. The results reported by the participant were evaluated by using two types of z-scores, that is, one was the score based on the consensus values calculated from the analytical results of participants, and the other one was the score based on the reference values obtained by NMIJ with high reliability. Acceptable z-scores based on the consensus values and NMIJ reference values were achieved by 87 % to 89 % and 79 % to 94 % of the participants, respectively. Graphical Abstract

Distribution of z and z_NMIJ-scores for isoprothiolane     

Application of ring study: water toxicity determinations by bioluminescence assay with Vibrio fischeri

Acceptance of toxicity bioassays as effective analytical tools in environmental areas needs guarantees of standardization but also validation. Ten European laboratories took part in an inter-laboratory study using different commercial devices based on bioluminescence inhibition of bacteria Vibrio fischeri. Reproducibility and stability by short toxicity endpoints, effective concentration that gives 10%, 50% and 80% of inhibition (EC₁₀, EC₅₀ and EC₈₀) is evaluated. Parametric and non-parametric statistic is applied and performance of participant laboratories is addressed by z-scores calculated by non-parametric statistic. z-Score classification was based on harmonised protocol for proficiency testing of analytical laboratories (satisfactory |z| ≤ 2; questionable 2 < |z| ≤ 3; unsatisfactory |z| > 3). Tested samples were phenol, 3,5-dichlorophenol and influent wastewater. Based on z-score classification, more than 70% of the laboratories showed a satisfactory performance for phenol, 3,5-dichlorophenol and influent wastewater (86%, 90% and 70%, respectively). Reproducibility and stability was observed in toxicant references and in wastewater samples. EC₈₀ determination appears to be more robust that EC₁₀ and EC₅₀. EC determinations can be considered favorable at 5 and 15 min of exposition, in particular for EC₈₀. The use of different commercial devices can not be considered an additional source of variation.     

Development of a proficiency testing scheme for a limited number of participants in the field of natural water analysis

A proficiency testing (PT) scheme was developed for a limited number of analytical laboratories participating in the analysis of natural water in Israel. Three fit-for-intended-use reference materials (RMs) were prepared for a pilot PT from natural water: RM-blank and two in-house reference materials (IHRMs) with seven analytes fortified to achieve different levels of analyte concentrations. The measurands for the PT participants were the traceable spike values certified in the IHRMs, i.e., added mass concentrations of the analytes. The RM-blank and IHRMs were found to be homogeneous and stable over 11 days, the time necessary for the experimental part of the PT. The RMs were distributed to the PT participants as unknown test items similar to routine samples. The test results were evaluated using their deviations from the IHRM-certified values. Eight Israeli laboratories took part in the interlaboratory comparison. Individual laboratory performance and metrological compatibility of the PT results of the participants, as a local group of laboratories, were evaluated for every analyte.     

Evaluation of the use of consensus values in proficiency testing programmes

Proficiency testing (PT) is an essential tool used by laboratory accreditation bodies to assess the competency of laboratories. Because of limited resources of PT providers or for other reasons, the assigned reference value used in the calculation of z-score values has usually been derived from some sort of consensus value obtained by central tendency estimators such as the arithmetic mean or robust mean. However, if the assigned reference value deviates significantly from the ‘true value’ of the analyte in the test material, laboratories’ performance will be evaluated incorrectly. This paper evaluates the use of consensus values in proficiency testing programmes using the Monte Carlo simulation technique. The results indicated that the deviation of the assigned value from the true value could be as large as 40%, depending on the parameters of the proficiency testing programmes under investigation such as sample homogeneity, number of participant laboratories, concentration level, method precision and laboratory bias. To study how these parameters affect the degree of discrepancy between the consensus value and the true value, a fractional factorial design was also applied. The findings indicate that the number of participating laboratories and the distribution of laboratory bias were the prime two factors affecting the deviation of the consensus value from the true value.     

Proficiency test on incurred and spiked pesticide residues in cereals

A proficiency test on incurred and spiked pesticide residues in wheat was organised in 2008. The test material was grown in 2007 and treated in the field with 14 pesticides formulations containing the active substances, alpha-cypermethrin, bifentrin, carbendazim, chlormequat, chlorpyrifos-methyl, difenconazole, epoxiconazole, glyphosate, iprodione, malathion, pirimicarb, prochloraz, spiroxamin and trifloxystrobin. After harvest, the test material was additionally spiked in the laboratory with three pesticides, that where the residues were too low, and axozystrobin. In total, 72 laboratories submitted results and z-scores were calculated for all laboratories and pesticides, except for glyphosate where only five laboratories submitted results and summed weighted z-scores were calculated for the laboratories with a sufficient scope. For several pesticides, the submitted results were strongly depending on the extraction procedure and consequently the assigned values were calculated based on part of the results. Acceptable z-scores were obtained by 56–97% of the participants.     

Aspects of proficiency testing studies of trace elements in environmental samples with a focus on laboratory performance

Four proficiency tests of environmentally significant elements in soil and water are described. The effects of sample matrix, analyte, sample preparation procedure, extraction method efficiency and instrumental technique on laboratory performance were assessed. Participants’ performance was evaluated using z scores and E _n scores. One study included dried sediment and sediment digest as test samples. By testing both dried sediment and sediment digest in the same study, differences in laboratories’ results due to the extraction method or instrumental technique were evaluated. Cadmium and nickel were the elements with the most unsatisfactory z scores for dried sediment compared to sediment digest. The second study had as test samples river water and river water fortified with biosolid. For an evaluation of the efficiency of participants’ extraction method, the results reported for the analysis of river water were compared to the results for analysis of the same river water fortified with biosolid. No consensus could be found between participants’ results for Cr and Ni in this study as the extraction method employed by them produced results that were too variable. The last two studies included saline and fresh water samples. The analysis of fresh and saline water samples with similar analyte concentrations assisted laboratories in developing their methods for trace elements in saline water. For Cr, Pb and Se, the assigned values were reference values traceable to SI, measured using isotope dilution inductively coupled plasma mass spectrometry. Saline water containing low concentrations of analytes presented difficulties to some laboratories. Arsenic and selenium were the analytes most difficult to analyse in all types of environmental samples.     

Bias analysis of proficiency testing programme results

A proficiency testing programme might involve a great number of participating laboratories coming from different countries or regions, and normally they analysed the same test materials using their own routine analytical methods. Hence, the results of a proficiency testing programme may contain valuable information which could serve purposes other than just performance evaluation. This study attempted to extract information from the results of a proficiency testing programme for the purposes of educating the participating laboratories as suggested by ISO/IEC 17043. The “bias analysis” approach introduced in this study was based on the statistical model of measurement and the nature of bias in chemical analysis. With this approach, the participating laboratories could estimate the bias associated with different settings of experimental conditions according to the statistics of subset distribution of the reported results from the participating laboratories. This would be useful for them to review the analytical procedures they used and modify their methods if needed. The approach was applied to the analysis of data obtained from a number of past proficiency testing programmes, and the findings were discussed in this paper.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

Determination of POPs in environmental matrices – proficiency tests for Polish laboratories

Thirty Polish laboratories participated in proficiency tests (PT) for determination of persistent organic pollutants – POPs (PCBs – polychlorinated biphenyls, PAHs – polyaromatic hydrocarbons and pesticides) in solid environmental matrices (sediments, soil). Participants used their in-house method. The aim of the exercises was to obtain information about the “state of the art” of POPs analysis in environmental samples. The CV (coefficient of variation) of analytical results submitted varied from 45 to 65%, with an outlier rate 2 to 7%. The averages of results for all analytes were in good agreement with reference values. Statistical treatment of results was made. Comparison of the each result with the certified values characterizing the material has been done. Additionally, z-scores were calculated taking into account the uncertainty of a certified value and the acceptable deviation from the certified value in the range of −20% to +10% (in accordance with recommendations in the newest directives of European Union (EU) [1]); For rejecting outliers Hampel test has been applied.     

Reference materials to evaluate measurement systems for the nutrient composition of foods: results from USDA’s National Food and Nutrient Analysis Program (NFNAP)

Over a 6.5-year period a total of 2554 values were reported by nine laboratories for 259 certified or reference nutrient concentrations in 26 certified reference materials (CRM) submitted to contract laboratories, blinded, as part of the qualifying process for analytical contracts and in the routine sample stream as part of the National Food and Nutrient Analysis Program. Each value was converted to a Z′-score, reflecting the difference from the assigned value related to the combined expected analytical uncertainty plus the uncertainty in the CRM value. Z′-scores >|3.0| were considered unacceptable. For some nutrients (Na, folate, dietary fiber, pantothenic acid, thiamin, tocopherols, carotenoids, monounsaturated, and polyunsaturated fatty acids), >20% of Z′-scores were >|3.0|. For total fat, vitamin C, and niacin >25% of Z′-scores were >|2.0|. Components for which CRM data were best (more than 90% of Z′-scores <|2.0|) were Mg, P, Mn, Se, and vitamin B12. In some cases deviations from assigned values were not uniform across laboratories and materials. For Na almost all high Z′-scores were for low-Na matrices, suggesting analytical problems related to concentration. Figure Z′-scores for vitamins in certified reference materials Disclaimer: Certain commercial equipment, instruments, or materials are identified in this paper in order to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, or the United States Department of Agriculture, nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose.     

Preparation of in-house reference material of benzylpenicillin in milk and results of a Brazilian proficiency testing scheme

Two milk test materials containing benzylpenicillin and a benzylpenicillin-free material were prepared and used for the operation of a Brazilian proficiency testing (PT) scheme according to the requirements of ISO/IEC 17043. The PT scheme was designed to include laboratories in charge of confirmatory analysis as well as screening analysis. Sets of sample vials, benzylpenicillin-free and spiked items containing 0.2 g of lyophilized milk, were distributed to each participant. Of 20 participants who reported screening data, 18 accomplished satisfying results, providing evidence of the capability of Brazilian laboratories to produce reliable qualitative information. The assigned value (robust average calculated from 6 results) and its uncertainty were (8.28 ± 0.52) μg/L. Participants’ performance was evaluated using z-scores. A small number of participants were able to report quantitative benzylpenicillin results, and consequently care should be taken in the interpretation of the laboratory’s performance, considering the statistical nature of z-scores. Long-term stability testing on remaining PT items showed that a stable benzylpenicillin quality control material was produced and can be used for internal quality control or validation purposes.     

Capabilities of laboratories to determine melamine in food—results of an international proficiency test

A proficiency test to assess the capabilities of laboratories to determine melamine in a milk powder and a baking mix, representing starch-containing foods like bread and biscuits, was carried out in January 2009. The need for such an interlaboratory comparison arose from a health scare in China about melamine-tainted powdered milk in the second half of 2008. Laboratories in 31 countries, including Australia, China, India, Japan, New Zealand and the USA, and 21 of the 27 Member States of the European Union participated and reported back 114 results for the milk powder and 112 for the baking mix test materials. The reported results were compared to reference values determined by exact-matching double isotope dilution mass spectrometry. The so-determined assigned values were 10.0 ± 0.6 mg/kg melamine in the milk powder and 3.18 ± 0.17 mg/kg melamine in the baking mix. A coverage factor k of 2 was applied to calculate the expanded uncertainties. Three quarters of all reported results for both materials had associated z scores which were satisfactory (z ≤ |2|). Of the reported results, 90% was accompanied by a measurement uncertainty statement, and the majority of the measurement uncertainty ranges were reasonable. A number of laboratories were found to underestimate their measurement uncertainties. Methods that involved the use of stable-isotope-labelled melamine were shown to be clearly advantageous with regard to the accuracy of the results. However, no significant influence by other method parameters could be identified.     

A new approach in assessing microbiological results in water analysis proficiency testing

Due to the relatively large spread in the results of microbiological proficiency tests, the z-scores are often not able to detect zero or low results as being ‘bad’ results. This paper describes an adapted z-score based on the average or the standard deviation of the 50% ‘highest’ results. The combination of the adapted z-scores of four samples enables a better judgement of the performance of each laboratory. Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board or the Publisher.     

Development and validation of a liquid chromatography–tandem mass spectrometry method for the determination of febuxostat in human plasma

A rapid and sensitive analytical method based on liquid chromatography coupled to tandem mass spectrometry detection with positive ion electrospray ionization was developed for the determination of febuxostat in human plasma using d₇‐febuxostat as the internal standard (IS). A simple protein precipitation was performed using acetonitrile. The analyte and IS were subjected to chromatographic analysis on a Capcell PAK C₁₈ column (4.6 × 100 mm, 5 µm) using acetonitrile–5 mm ammonium acetate–formic acid (85:15:0.015, v/v/v) as the mobile phase at a flow rate of 0.6 mL/min. An Agilent 6460 electrospray tandem mass spectrometer was operated in the multiple reaction monitoring mode. The precursor‐to‐product ion transitions m/z 317 → m/z 261 (febuxsotat) and m/z 324 → m/z (261 + 262) (d₇‐febuxostat, IS) were used for quantitation. The results were linear over the studied range (10.0–5000 ng/mL), and the total analysis time for each chromatograph was 3 min. The intra‐ and inter‐day precisions were less than 7.9 and 7.2%, respectively, and the accuracy was within ±4.2%. No evidence of analyte instability in human plasma was observed storage at ?20°C for 31 days. This method was successfully applied in the determination of febuxostat concentrations in plasma samples from healthy Chinese volunteers. Copyright © 2012 John Wiley & Sons, Ltd.     

External quality assessment schemes for clinical laboratories

 The clinical routine laboratory generally utilizes cheap, easy and rapid measurement procedures ("methods") in order to meet the requirement for the production of many analytical results (500–3000 per day) of hundreds of different types. The measurement procedures are optimized for the analysis of native patients' samples, but are frequently sensitive to deviation of the composition of the matrix from that in normal fresh samples. The inherent lack of stability of patient samples means that control samples need to be stabilized. The method of stabilization is critical. Furthermore, the method of "spiking" samples with pathological material is a matter of concern. Generally, minimally processed patients' samples should be used in external quality assessment (EQA) schemes. Consensus values are currently the most popular for use as a guide to the best results from participating laboratories in EQA schemes; these often work fairly well. However, the uncertainty and traceability of this type of value is unknown, and in some cases may even be misleading, tending to preserve bad routine methods when these are dominant in the participating laboratories. Reference measurement procedure (RMP) values are recommended to provide scientifically based information, to facilitate the proper choice of methods in the routine laboratories, and to validate the suitability of control materials in EQA schemes. The present paper provides selected examples from a study comparing consensus values with RMP values on lyophilized sera, and also presents results on a fresh frozen thawed serum for the study of commutability. Received: 8 November 1995 Accepted: 8 May 1996     

Interpretation of interlaboratory comparison results to evaluate laboratory proficiency

Guidelines are given for the evaluation of proficiency test (PT) results in order to increase the effectivity of PT participation. For better understanding, some statistical background is given along with some examples to show the effects of the choices made by the PT provider. The calculation method of the assigned value and the selection of the standard deviation both affect the z-score that is used by the participating laboratory to judge the quality of its performance in the PT. Therefore, the participating laboratory is advised to use the PT results with care and, if necessary, to recalculate the z-scores. Finally, advice is given on how not to follow up bad PT results along with some valuable steps that could be part of an effective follow-up procedure.