The correlation of laboratory performance in proficiency testing with other QA characteristics

Although it seems self-evident that proficiency testing (PT) and accreditation can be expected to improve quality, their relative benefits remain uncertain as does their efficacy. The study reported here examines the following issues: (a) Why do laboratories take part in PT schemes? (b) How does participation in PT fit in with a laboratory's overall quality assurance (QA) system? (c) Is there a link between a laboratory's performance in specific PT and it's QA system? (d) How does PT performance change with time and how do laboratories respond to poor performance? The overall conclusion is that there is no evidence from the present study that laboratories with third-party assessment (accreditation and certification) perform any better in PT than laboratories without. The validity of this conclusion and its significance for the future design and operation of such schemes requires further investigation. In particular, study is required of the degree to which good performance in open PT correlates with blind PT performance, where laboratories are not aware that the samples being analysed are part of a quality assessment exercise.     

Information system and qualifying criteria for proficiency testing schemes

Since October 1998 the European Commission has financed a concerted action on Information System and Qualifying Criteria for Proficiency Testing Schemes within the 4th framework program. As a major result of this project EPTIS, the European Information System on Proficiency Testing Schemes which is available on the Internet since March 2000, is presented in this paper. Today EPTIS contains comprehensive information on approximately 640 proficiency testing schemes from 16 European countries providing information on the state of the art in proficiency testing in Europe. Finally some possible approaches for interlinkages and recognition of proficiency testing schemes are discussed.     

Quality issues in proficiency testing

 Proficiency testing is a means of assessing the ability of laboratories to competently perform specific tests and/or measurements. It supplements a laboratory's own internal quality control procedure by providing an additional external audit of their testing capability and provides laboratories with a sound basis for continuous improvement. It is also a means towards achieving comparability of measurement between laboratories. Participation is one of the few ways in which a laboratory can compare its performance with that of other laboratories. Good performance in proficiency testing schemes provides independent evidence and hence reassurance to the laboratory and its clients that its procedures, test methods and other laboratory operations are under control. For test results to have any credibility, they must be traceable to a standard of measurement, preferably in terms of SI units, and must be accompanied by a statement of uncertainty. Analytical chemists are coming to realise that this is just as true in their field as it is for physical measurements, and applies equally to proficiency testing results and laboratory test reports. Recent approaches toward ensuring the quality and comparability of proficiency testing schemes and the means of evaluating proficiency test results are described. These have led to the drafting of guidelines and subsequently to the development of international requirements for the competence of scheme providers. Received: 2 January 1999 · Accepted: 7 April 1999     

Benefits arising from proficiency testing schemes: The comparison of analytical methods

 The primary objective of proficiency testing (PT) is in the provision of information and support to participating laboratories, to enable them to monitor and improve the quality of their measurements. However, other benefits can be obtained from PT. These include the comparison of data for a given measurement by different methods, the validation of new methods, and the provision of information for laboratories' customers and accreditation bodies. This paper considers the subject of method comparison, and highlights some of the approaches which can be followed, as well as the practical use to which this can be put, to benefit the analytical community more widely. This is illustrated by a case study concerning the measurement of haze in beer. In this study the United Kingdom Institute of Brewing (IoB) conducted a survey of participants in the Brewing Analytes Proficiency Scheme (BAPS). From the survey data taken together with data from the BAPS scheme, the IoB is now in a position to give guidance on the use of particular instruments and procedures, as well as consider changes to the scope of the BAPS scheme to provide greater benefits for participants concerned with measuring haze. Received: 3 March 1998 · Accepted: 9 June 1998     

The proficiency testing programme for trace elements of the CORILA project

Harmonised quality schemes are being implemented in the framework of the thematic area Organization and Dissemination of Data of the Consorzio per la Gestione del Centro di Coordinamento delle Attività di Ricerca Inerenti il Sistema Lagunare di Venezia (Consortium for the Management of the Coordination Centre of Research Activities on the Venice Lagoon System, CORILA). Over the last 2 years, two proficiency tests were completed at the time of writing (while a third one is still in progress) to assess the capabilities of the analytical laboratories involved in the determination of trace elements in environmental inorganic matrices from the Lagoon of Venice.Both interlaboratorial trials were based on the determination of As, Cd, Cr, Cu, Fe, Hg, Pb and Zn in aqueous acidic, matrix-enriched solutions at concentrations within the ranges generally found in marine water and in sediment after chemical digestion, respectively. The proficiency of participants was assessed on the basis of the z score criterion (|z|≤2, acceptable results; 2<|z|<3, questionable results; |z|≥3, unacceptable results). In the two trials, the relevant percentages for the laboratories scoring |z|≤2 and |z|<3 were 61% and 71% and 65% and 77%, respectively. These first two proficiency tests, on the one hand, clearly pointed to the benefits the laboratories had from their participation in these exercises, and on the other hand, provided sound evidence of the need for further trials based on real environmental matrices.     

Environmental analysis in the Czech Republic: organisation of proficiency testing and its use in laboratory assessment

The history, origin, and development of a system for monitoring and assessing water and other environmental laboratories in the Czech Republic is described. The system started in 1991 and has matured to its present complexity with similarities to the accreditation systems found in other countries. Differences from internationally recognized procedures are being corrected step by step. During the first year of its existence ASLAB, as part of its brief, organised proficiency testing (PT) programs for fifty laboratories. Today the total number of regularly participating laboratories exceeds 700 from the Czech Republic, the Slovak Republic, and Germany. This paper describes the ASLAB PT system, discusses some experiences with its use, and describes the use of PT results in assessment of the competence of laboratories. Received: 12 October 2000 Accepted: 7 January 2001     

Experiences from the assessment of proficiency testing providers

Following a pilot project in 2001–2002, UKAS accredited nine PT providers. During the course of the pilot and at subsequent surveillance visits, UKAS has gained experience in assessing organisations against the criteria detailed within ISO/IEC Guide 43-1:1997 and ILAC G13:2000. This article explores some of the key aspects that arose during the pilot and how UKAS has accommodated the addition of the accreditation of PT providers in to its range of services.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

A statistical strategy for discouraging collusion in split-level proficiency testing schemes

A statistical strategy for discouraging collusion in split-level proficiency testing schemes was developed. When the samples are randomly labeled, the difference in the analyte levels in the two samples can be designed to maximize the probability of detecting collusion when it occurs, so that participants who colluded cannot easily receive satisfactory assessments. This favors providing convictive assessments of participants proficiency and giving the participants a level playing field.     

ISO/IEC 17043: the new International Standard for proficiency testing

The revision of ISO/IEC Guide 43 is well underway by CASCO Working Group 28, with the author as Convener. The ballot on the committee draft was successfully completed in June 2008. The Working Group met in September to resolve the CD comments and to recommend that the document should advance as an ISO/IEC DIS. The document is in harmony with ISO/IEC 17025 (2005), ILAC G13 (2007), and the IUPAC Harmonized Protocol (2006) for proficiency testing. Major changes from Guide 43 include considerations for inspection bodies, requirements for metrological traceability of assigned values for calibration schemes, limitations on subcontracting, requirements for reporting the uncertainty of assigned values, and for including method information and technical commentary in scheme reports. Minor changes include requirements for equipment used in the manufacture and testing of proficiency test items, considerations for handling laboratories’ reported uncertainties, and the issuance of certificates of participation. These changes have not generated any substantial objections from the CASCO membership and liaisons.     

A new approach in assessing microbiological results in water analysis proficiency testing

Due to the relatively large spread in the results of microbiological proficiency tests, the z-scores are often not able to detect zero or low results as being ‘bad’ results. This paper describes an adapted z-score based on the average or the standard deviation of the 50% ‘highest’ results. The combination of the adapted z-scores of four samples enables a better judgement of the performance of each laboratory. Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board or the Publisher.     

Inter-comparison exercises as a tool for teaching quality assessment: an experience in Spanish universities

Since the academic year 2001–2002, inter-laboratory trials for students of Analytical Chemistry in Spanish Universities have been organised by the Department of Analytical Chemistry at the University of Barcelona in collaboration with the Complutense University of Madrid, the University of Cordoba and the University of Huelva. The aim of these exercises is to train students in the use of tools for the assessment and improvement of quality in analytical laboratories.Representative samples of environmental and food analysis, agricultural soils and a type of beer were selected. The ethanol content of the beer and the pH, conductivity, and extractable phosphorus and potassium content in the soil were the chosen analytical parameters.Sample preparation, homogeneity and stability studies, as well as the statistical treatment of data from participants, were carried out by the laboratory Mat Control of the Department of Analytical Chemistry of the University of Barcelona.The paper presented heregives the results obtained after two years of experience.Presented at BERM-9—Ninth International Symposium on Biological and Environmental Reference Materials, June 15–19, 2003, Berlin, Germany.     

The role of proficiency testing in the detection and resolution of calibration bias in the LeadCare® blood lead analyzer; limitations of peer-group assessment

Following implementation of the CLIA ‘88 laboratory regulations, the primary role of proficiency testing (external quality assessment, PT) in the U.S. has been widely viewed as one of assuring regulatory compliance. PT can also be an effective tool for detecting widespread analytical problems, subject to limitations based on the method of PT assigned value determination. A recent case study describes the role of two PT programs in detecting and resolving a calibration bias in the LeadCare blood lead analyzer, and illustrates the limitations of peer-group target determination in fulfilling that PT role.     

Hierarchical classification designs for the estimation of different sources of variability in proficiency testing experiments

An approach to the estimation of possible different sources of variation found in proficiency testing experiments is described. Four errors namely, technique, analyst, laboratory and geographical location are considered and calculated by using a rational experimental design based on hierarchical classification. The treatment of the confidence of the design over different experimental arrangements is explored and visualised by calculating a function that depends only on the design and not on the experimental response. An illustrative example based on simulated data is used to show how the theory could be applied in practice.     

The practice of proficiency testing in China

This paper briefly summarises the current situation for proficiency testing (PT) in China, outlines the policy for PT of China’s national accreditation committee for laboratories (CNACL), and exemplifies activities of the CNACL’s metal working group. Received: 9 December 2000 Accepted: 14 December 2000     

Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.     

Nonparametric assessment of comparability of analytical results obtained in proficiency testing based on a metrological approach

A nonparametric sign test is implemented for assessment of comparability of proficiency testing (PT) results when their distribution differs from the normal or other known distribution. It allows testing the null hypothesis about insignificance of the bias of median of results obtained in PT from the traceable certified value of the reference material used in PT as test items, i.e., the hypothesis stating that comparability of the PT results is successful. Probability of type I error of rejecting the null hypothesis when it is true, and probability of type II error of it not rejecting when it is false (the alternative hypothesis about unsuccessful comparability is true) are considered. The test can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results when the comparability criterion developed for a normal results distribution (Accred. Qual. Assur. 10:466–470) is not applicable.     

Statistical treatment of proficiency testing data

 There are three stages to evaluating a laboratory's results in an interlaboratory proficiency test: establishing the correct result for the test item, determining an evaluation statistic for the particular result, and establishing an acceptable range. There are a wide variety of procedures for accomplishing these three stages and a correspondingly wide variety of statistical techniques in use. Currently in North America the largest number of laboratory proficiency test programs are in the clinical laboratory field, followed by programs for environmental laboratories that test drinking water and waste water. Proficiency testing in both of these fields is under the jurisdiction of the federal government and other regulatory and accreditation agencies. Many of the statistical procedures are specified in the regulations, to assure comparability of different programs and a fair evaluation of performance. In this article statistical procedures recommended in International Organization for Standardization Guide 43, Part 1, are discussed and compared with current practices in North America. Received: 22 April 1998 · Accepted: 12 May 1998     

Traceable reference values for routine drinking water proficiency testing: first experiences

Proficiency testing (PT) results have been used to improve traceability in chemical drinking water analysis. With a generalized least-square regression the mass concentrations of As and Sb were calculated in a drinking water that had been used to prepare proficiency testing samples by a spiking procedure. From the mass concentrations in the matrix and the spiked amounts, reference values with an uncertainty budget could be calculated without the need for reference measurements. The degree to which these reference values can be regarded as traceable is discussed. The results showed slight deviations in some samples between reference values and consensus means.     

水产品中呋喃唑酮代谢物AOZ的检测能力验证分析

介绍了全国多个行业51家实验室参加水产品中呋喃唑酮代谢物AOZ的检测能力验证计划的情况。选用活虾药浴暂养后制得并通过均匀性F检验和稳定性t检验的虾肉糜样作为测试样品,采用稳健统计技术对试验结果进行了分析。讨论了水产品中呋喃唑酮代谢物AOZ检测的一些关键点。