Acquisition of stability data for pesticides in water sample through proficiency tests

Accreditation and Quality Assurance - In the framework of a proficiency test for multiresidue determination, three water samples, coming from the same batch, were sent to each of the 25...     

Uncertainty estimation in organic elemental analysis using information from proficiency tests

We evaluate the uncertainty in organic elemental analysis of C, H, N, and S. We use data from six proficiency tests (PTs), in which some 35 Spanish laboratories participated. The uncertainty of the technique is estimated from the relative within-laboratory and between-laboratory variances for pure substances and samples with complex matrices (soil, powdered milk, oil, ash, and petroleum coke). We also calculate the relative standard uncertainties for individual laboratories when analysing pure substances using historical data from the participation of each laboratory in different editions of PTs. The uncertainty values obtained for the individual laboratories are comparable with the uncertainty of the technique and correlate with the combined z-scores. The evolution over time of those laboratories participating in common editions of PTs is also evaluated.     

The rational use of proficiency tests and intercomparisons

 Interlaboratory comparisons (in the following abbreviated as intercomparisons) regarding tests, analyses or measurements are among the most worthwhile measures a laboratory can take in order to confirm that its services to clients include the provision of correct results within a stated uncertainty. They give a picture of the whole performance of the laboratory, and they should be used much more than at present. Unfortunately such intercomparisons are, and are made, expensive and prestigious by the formalisms employed in many cases. The connection between intercomparisons and proficiency tests and their use for different purposes is briefly discussed. Some suggestions are made on how to improve the present state of the art, i.e. how to increase the use of intercomparisons, how to perform them efficiently and how to make optimal use of the results. Received: 6 December 1997 · Accepted: 30 January 1998     

Statistical treatment of proficiency testing data

 There are three stages to evaluating a laboratory's results in an interlaboratory proficiency test: establishing the correct result for the test item, determining an evaluation statistic for the particular result, and establishing an acceptable range. There are a wide variety of procedures for accomplishing these three stages and a correspondingly wide variety of statistical techniques in use. Currently in North America the largest number of laboratory proficiency test programs are in the clinical laboratory field, followed by programs for environmental laboratories that test drinking water and waste water. Proficiency testing in both of these fields is under the jurisdiction of the federal government and other regulatory and accreditation agencies. Many of the statistical procedures are specified in the regulations, to assure comparability of different programs and a fair evaluation of performance. In this article statistical procedures recommended in International Organization for Standardization Guide 43, Part 1, are discussed and compared with current practices in North America. Received: 22 April 1998 · Accepted: 12 May 1998     

Reliability of measurement uncertainty estimates for forensic analyses: evidence from recent proficiency tests

The ability of forensic laboratories to reliably estimate the uncertainty of their reported results is assessed by means of data from recent proficiency tests in which participants reported both their results and the associated uncertainties. Topics covered include the determination of blood alcohol, breath alcohol, and controlled substances—services commonly provided by forensic laboratories. It is seen that a statistically significant number of laboratories underestimate their own uncertainties, based on the incidence of the laboratories’ own reported coverage intervals failing to include the participant mean or assigned concentration of the test material.     

Control charts to evaluate long-term performance in proficiency tests

As proficiency tests (PTs) provider, the European Union Reference Laboratory for Chemical Elements in Food of Animal Origin (EURL-CEFAO) yearly organizes exercises on the determination of chemical elements within the area of its mandate. The scheme is addressed to the National Reference Laboratories (NRLs) of the European Union Member States. Each PT is planned giving priority to the matrices and elements for which maximum levels are set in Commission Regulation No. 1881/2006 and following amendments as well as to those mainly considered in National Residues Control Plans. Samples to be distributed are prepared in the Laboratory’s facilities using materials purchased on the market and often adjusting the concentration levels of the analytes around their maximum levels. In the 2006–2011 period, PTs were planned as a long-term programme pursuing the general objective of providing the NRLs with a scheme that can give them the opportunity to check and improve the performance of their analytical methods and to verify the effectiveness of any corrective actions through the repetition of the same matrix. Control charts give participants the possibility of a long-term follow-up of their performance promoting the improvement of Quality Control. To this aim, Shewhart and Cusum control charts for z-scores are prepared according to ISO 13528:2005 and updated after each PT for each laboratory. z-scores are assigned using values of σ _pEURL-CEFAO that correspond to the level of performance appropriate to the demand. Cusum charts are also evaluated by EURL-CEFAO in order to identify problems that may cause a bias in the measurement method.     

Estimates of uncertainty of measurement from proficiency testing data: a case study

The results obtained by a laboratory over a number of proficiency testing/external quality assessment schemes (PT/EQAS) rounds can give information on the uncertainty of its measurements for a given test, provided that conditions such as full coverage of the routine analytical range, traceability, and small uncertainty of the assigned values (compared to the spread of the results) are met and provided that systematic deviations and any other sources of uncertainty are considered. As organisers of the Italian EQAS (ITEQAS) in occupational and environmental laboratory medicine, we tested this hypothesis using as model data from well-performing laboratories taking part in ITEQAS for lead in blood over the last 2 years. We also investigated how different PT/EQAS features (frequency of trials and number of samples) would affect a laboratory estimate of its uncertainty. Such information can be helpful in improving PT/EQAS organisation and define, for a given test: (a) the state of the art of the uncertainty of current measurement procedures, (b) identify needs for improvement of analytical methodologies and (c) set targets for acceptable uncertainty values.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia.Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher.     

Definition of assigned values for proficiency tests in water analysis

The definition of an assigned value is usually achieved by calculating mean values from the data (with different methods) or by designating reference laboratories. Neither method is completely satisfactory. In this paper a new method is presented for the definition of the assigned value for spiked samples with an unknown content of the analyte in the matrix. The method consists of two parts. The first is the estimation of the assigned values from the spiked amounts and the content in the matrix, based on the results of reference laboratories. The other is the designation of these reference laboratories by comparing their results with the assigned values. Because each of these parts requires the other, an iterative procedure is necessary. As an example, the results of a proficiency test for the analysis of copper in wastewater are used to compare the calculated values with those from other methods, e.g., the Huber estimation. Received: 25 September 2000 Accepted: 9 December 2000     

Use of characteristic functions derived from proficiency testing data to evaluate measurement uncertainties

Interlaboratory comparisons show that reproducibility standard deviations are dependent on the concentration of the analyte. Many attempts have been made to model this. In this paper, ??characteristic?? functions are used for modelling the concentration dependence of the reproducibility standard deviations based on data from proficiency tests for water analysis. The characteristics of the resulting functions can be used for the estimation of measurement uncertainties at different concentration levels. These functions are especially useful to determine the concentration levels below which absolute uncertainties tend to be constant and above which the relative uncertainties are more constant. By comparing the characteristic functions of different analytical procedures for the determination of the same analyte, the performance of these procedures under routine application can be compared. Finally, these functions may be used to get an indication on the average quality of analytical result in a specific field to be used by regulators in order to formulate requirements in the legislation that are in accordance with current measurement quality.     

The determination of an accepted reference value from proficiency data with stated uncertainties

Proficiency data with stated uncertainties represent a unique opportunity for testing that the reported uncertainties are consistent with the Guide to the expression of uncertainty in measurement (GUM). In most proficiency tests, however, this opportunity is forfeited, because proficiency data are processed without regard to their uncertainties. In this paper we present alternative approaches for determining a reference value as the weighted mean of all mutually consistent results and their stated uncertainties. Using an accepted reference value each reported uncertainty estimate can be expressed as an E _n number, but a value of confirms its validity only if the uncertainty of the reference value is negligible in comparison.Reference values calculated for results from an International Measurement Evaluation Programme (IMEP-9) by “bottom up” as well as “top down” methods were practically identical, although the first strategy yielded the lowest uncertainty. A plot of individual coefficients of variation (CV) versus E _n numbers helps interpretation of the proficiency data, which could be used to validate relative uncertainties down to <1%.     

Analytical performance is improved by regular participation in proficiency testing: an analysis of data from the Aquacheck proficiency testing scheme

Analysis of historical data from the Aquacheck proficiency testing scheme demonstrates improvement in the performance of participants correlated to the frequency of participation. Analysis of the spread of results returned by all participants, defined by the percentage relative standard deviation (RSD) and the performance (Z) scores awarded to the individual participants, indicates that the overall performance for the group of participants is consistent. However, when the frequency of participation is analysed, it can be clearly seen that those laboratories who participated in the majority of the distributions available reported 10–15% more acceptable (Z-scores between +2 and ?2) results than those labs who took part with a greatly reduced frequency. When ‘more difficult’ analytes are considered, those laboratories who participated in greater than 75% of available rounds reported up to 30% more acceptable results than those laboratories who participated in 25% or fewer, of the available rounds.     

Comparison of different approaches to the statistical evaluation of proficiency tests

The objective of this study was to compare different z-score calculation for the results from proficiency testing programmes. The comparison was carried out on the data from proficiency tests for four different matrices and for all analytes included in the proficiency testing programmes organised by UKZUZ institute from the year 2005 to 2007. It represented 99 samples (three periods per year and eleven samples distributed in each period); the number of determined analytes was 108 in each period. The total number of investigated data sets was 953. Three different approaches to z-score calculation were compared. The results gave reasonably comparable evaluation in most cases but sometimes UKZUZ method proved to be rather strict. The procedure using the robust average and robust standard deviation calculated according to Algorithm A described in ISO 13528 was found to be the most suitable for matrices and level of measurands in UKZUZ proficiency tests.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

Using mixture models for bump-hunting in the results of proficiency tests

The interpretation of the results of proficiency tests by the use of mixture models is described. The data are interpreted as a sample from a mixture of several normal populations. The calculation of the statistics (the means, variances and proportions of each component) is accomplished by means of the ‘EM’ algorithm. The method has several advantages over those previously advanced, principally that the algorithm is fast and easy to execute. Examples from proficiency testing are discussed.     

Evaluation of proficiency tests in microbiological analysis: enumeration of aerobic microorganisms

South Korea’s Ministry of Food and Drug Safety (MFDS) has been developing programs for the inspection and accreditation of food sanitation inspection institutions. Food sanitation inspection institutions such as MFDS regional offices, the Research Institute of Public Health and Environment and authorized private service providers in South Korea must participate in proficiency testing (PT) programs to comply with the Food Sanitation Act and MFDS Notification No. 2012-112. As the PT provider, the MFDS annually plans various microbiological and chemical PT programs for foods, cosmetics, and pharmaceutical products in accordance with ISO/IEC 17043. The aim of this project was to evaluate the performance of microbiological PT programs to ensure the quality of their routine test results. The test materials used were freeze-dried BioBalls from BTF Pty Ltd. Homogeneity and stability were investigated to assess the adequacy of the selected test materials. This project also contains data from inter-laboratory comparisons organized by MFDS in 2011 and 2012. More than 50 laboratories attended the PT program and submitted their results. Laboratory results were rated with z-scores according to the international standard ISO 13528. The results from 2011 and 2012 revealed that all participating laboratories had similar levels of proficiency. Most of the participants received a rating of “Satisfactory.” Moreover, the percentage of participants who received a rating of “Unsatisfactory” decreased from 3.5 % in 2011 to 2.0 % in 2012.     

Laboratory assessment by combined z score values in proficiency tests: experience gained through the European Union proficiency tests for pesticide residues in fruits and vegetables

The obligation for accredited laboratories to participate in proficiency tests under ISO 17025, performing multiresidue methods (MRMs) for pesticide residues, involves the reporting of a large number of individual z scores making the evaluation of the overall performance of the laboratories difficult. It entails, time and again, the need for ways to summarise the laboratory’s overall assessment into a unique combined index. In addition, the need for ways to continually evaluate the performance of the laboratory over the years is equally acknowledged. For these reasons, following 14 years of the European Union Reference Laboratory for Pesticide Residues in Fruits and Vegetables (EUPT-FV), useful formulas have been designed to globally evaluate the assessment of the participating laboratories. The aim is to achieve a formula which is easy to understand, which can be applied and which fits the purposes of long-term evaluation detecting positive and negative trends. Moreover, consideration is needed for a fair compensation of bad results in MRM, taking into account the large number of compounds that are covered. It is therefore important to be aware of the difficulties in getting satisfactory values from a wide range of compounds. This work presents an evaluation of the main well-established combined z score formulas together with those new ones developed here which have been applied to the European proficiency test results (EUPTs) over the years. Previous formulas such as the rescaled sum of z score (RSZ), the sum squared of z score (SSZ) and the relative laboratory performance (RLP) are compared with the newer ones: the sum of weighted z scores (SWZ) and the sum of squared z scores (SZ2). By means of formula comparisons, conclusions on the advantages, drawbacks and the most fit-for-purpose approach are achieved.     

Scoring in genetically modified organism proficiency tests based on log-transformed results

The study considers data from 2 UK-based proficiency schemes and includes data from a total of 29 rounds and 43 test materials over a period of 3 years. The results from the 2 schemes are similar and reinforce each other. The amplification process used in quantitative polymerase chain reaction determinations predicts a mixture of normal, binomial, and lognormal distributions dominated by the latter 2. As predicted, the study results consistently follow a positively skewed distribution. Log-transformation prior to calculating z-scores is effective in establishing near-symmetric distributions that are sufficiently close to normal to justify interpretation on the basis of the normal distribution.     

Determination of POPs in environmental matrices – proficiency tests for Polish laboratories

Thirty Polish laboratories participated in proficiency tests (PT) for determination of persistent organic pollutants – POPs (PCBs – polychlorinated biphenyls, PAHs – polyaromatic hydrocarbons and pesticides) in solid environmental matrices (sediments, soil). Participants used their in-house method. The aim of the exercises was to obtain information about the “state of the art” of POPs analysis in environmental samples. The CV (coefficient of variation) of analytical results submitted varied from 45 to 65%, with an outlier rate 2 to 7%. The averages of results for all analytes were in good agreement with reference values. Statistical treatment of results was made. Comparison of the each result with the certified values characterizing the material has been done. Additionally, z-scores were calculated taking into account the uncertainty of a certified value and the acceptable deviation from the certified value in the range of −20% to +10% (in accordance with recommendations in the newest directives of European Union (EU) [1]); For rejecting outliers Hampel test has been applied.     

Evaluation of the reproducibility standard deviation in the pesticide multi-residue methods on olive oil from past proficiency tests

Accreditation and Quality Assurance - The data reported in the frame of proficiency testing (PT) exercises organized from 2007 to 2016 have been statistically re-evaluated using Algorithm A of the...