Information system and qualifying criteria for proficiency testing schemes

Since October 1998 the European Commission has financed a concerted action on Information System and Qualifying Criteria for Proficiency Testing Schemes within the 4th framework program. As a major result of this project EPTIS, the European Information System on Proficiency Testing Schemes which is available on the Internet since March 2000, is presented in this paper. Today EPTIS contains comprehensive information on approximately 640 proficiency testing schemes from 16 European countries providing information on the state of the art in proficiency testing in Europe. Finally some possible approaches for interlinkages and recognition of proficiency testing schemes are discussed.     

The correlation of laboratory performance in proficiency testing with other QA characteristics

Although it seems self-evident that proficiency testing (PT) and accreditation can be expected to improve quality, their relative benefits remain uncertain as does their efficacy. The study reported here examines the following issues: (a) Why do laboratories take part in PT schemes? (b) How does participation in PT fit in with a laboratory's overall quality assurance (QA) system? (c) Is there a link between a laboratory's performance in specific PT and it's QA system? (d) How does PT performance change with time and how do laboratories respond to poor performance? The overall conclusion is that there is no evidence from the present study that laboratories with third-party assessment (accreditation and certification) perform any better in PT than laboratories without. The validity of this conclusion and its significance for the future design and operation of such schemes requires further investigation. In particular, study is required of the degree to which good performance in open PT correlates with blind PT performance, where laboratories are not aware that the samples being analysed are part of a quality assessment exercise.     

The practice of proficiency testing in China

This paper briefly summarises the current situation for proficiency testing (PT) in China, outlines the policy for PT of China’s national accreditation committee for laboratories (CNACL), and exemplifies activities of the CNACL’s metal working group. Received: 9 December 2000 Accepted: 14 December 2000     

Statistical treatment of proficiency testing data

 There are three stages to evaluating a laboratory's results in an interlaboratory proficiency test: establishing the correct result for the test item, determining an evaluation statistic for the particular result, and establishing an acceptable range. There are a wide variety of procedures for accomplishing these three stages and a correspondingly wide variety of statistical techniques in use. Currently in North America the largest number of laboratory proficiency test programs are in the clinical laboratory field, followed by programs for environmental laboratories that test drinking water and waste water. Proficiency testing in both of these fields is under the jurisdiction of the federal government and other regulatory and accreditation agencies. Many of the statistical procedures are specified in the regulations, to assure comparability of different programs and a fair evaluation of performance. In this article statistical procedures recommended in International Organization for Standardization Guide 43, Part 1, are discussed and compared with current practices in North America. Received: 22 April 1998 · Accepted: 12 May 1998     

Benefits arising from proficiency testing schemes: The comparison of analytical methods

 The primary objective of proficiency testing (PT) is in the provision of information and support to participating laboratories, to enable them to monitor and improve the quality of their measurements. However, other benefits can be obtained from PT. These include the comparison of data for a given measurement by different methods, the validation of new methods, and the provision of information for laboratories' customers and accreditation bodies. This paper considers the subject of method comparison, and highlights some of the approaches which can be followed, as well as the practical use to which this can be put, to benefit the analytical community more widely. This is illustrated by a case study concerning the measurement of haze in beer. In this study the United Kingdom Institute of Brewing (IoB) conducted a survey of participants in the Brewing Analytes Proficiency Scheme (BAPS). From the survey data taken together with data from the BAPS scheme, the IoB is now in a position to give guidance on the use of particular instruments and procedures, as well as consider changes to the scope of the BAPS scheme to provide greater benefits for participants concerned with measuring haze. Received: 3 March 1998 · Accepted: 9 June 1998     

Economy of proficiency testing: reference versus consensus values

Many proficiency tests are operated with a consensus value derived from the participants’ results. Apart from technical issues, one of the reasons often mentioned is that proficiency tests operated with consensus values would be cheaper than those using reference values obtained from a priori characterisation measurements. The economy of a proficiency test must of course be balanced by the need of the participants, and the quality of the comparison in general. The proficiency tests selected in this study had both a reference value and a consensus value, one of which was used for assessing the performance of the participating laboratories. In this work, both a technical and an economical assessment of how the comparisons were operated is made. From the evaluation, it follows that usually the use of consensus values does not necessarily reduce the costs of a proficiency test. However, frequently it may be observed that the quality of the assessment of the laboratories is better with a reference value. Received: 11 October 2000 Accepted: 3 January 2001     

Environmental analysis in the Czech Republic: organisation of proficiency testing and its use in laboratory assessment

The history, origin, and development of a system for monitoring and assessing water and other environmental laboratories in the Czech Republic is described. The system started in 1991 and has matured to its present complexity with similarities to the accreditation systems found in other countries. Differences from internationally recognized procedures are being corrected step by step. During the first year of its existence ASLAB, as part of its brief, organised proficiency testing (PT) programs for fifty laboratories. Today the total number of regularly participating laboratories exceeds 700 from the Czech Republic, the Slovak Republic, and Germany. This paper describes the ASLAB PT system, discusses some experiences with its use, and describes the use of PT results in assessment of the competence of laboratories. Received: 12 October 2000 Accepted: 7 January 2001     

Requirements vs acceptability in proficiency testing schemes and other interlaboratory comparisons

Many laboratories take part in proficiency testing schemes, external quality assessment programmes and other interlaboratory comparisons. These have many similarities but also important differences in their modus operandi and evaluation of performance of participating laboratories. This paper attempts to highlight both the similarities and differences. It also puts particular emphasis on requirements called ”target values for uncertainty” and their meaning. Received: 24 January 2001 Accepted: 25 January 2001     

Evaluation of proficiency testing results taking into account their uncertainties

A new composite score for the evaluation of performance of proficiency testing participants is proposed. The score is based on a combination of the z-score, uncertainty of a participant’s measurement result and uncertainty of the proficiency testing scheme’s assigned value. The use of such a composite score will allow evaluation not only of the participant’s ability to determine an analyte in corresponding matrix, but also their understanding of the uncertainty in the obtained analytical result. The score may be helpful for the laboratory’s quality system and for laboratory accreditation according to ISO 17025.     

An error model for evaluation and interpretation of proficiency testing

Data from proficiency testing can be used to increase our knowledge of the performance of populations of laboratories, individual laboratories and different measurement methods. To support the evaluation and interpretation of results from proficiency testing an error model containing different random and systematic components is presented. From a single round of a proficiency testing scheme the total variation in a population of laboratories can be estimated. With results from several rounds the random variation can be separated into a laboratory and time component and for individual laboratories it is then also possible to evaluate stability and bias in relation to the population mean. By comparing results from laboratories using different methods systematic differences between methods may be indicated. By using results from several rounds a systematic difference can be partitioned into two components: a common systematic difference, possibly depending on the level, and a sample-specific component. It is essential to distinguish between these two components as the former may be eliminated by a correction while the latter must be treated as a random component in the evaluation of uncertainty. Received: 20 November 2000 Accepted: 3 January 2001     

Definition of assigned values for proficiency tests in water analysis

The definition of an assigned value is usually achieved by calculating mean values from the data (with different methods) or by designating reference laboratories. Neither method is completely satisfactory. In this paper a new method is presented for the definition of the assigned value for spiked samples with an unknown content of the analyte in the matrix. The method consists of two parts. The first is the estimation of the assigned values from the spiked amounts and the content in the matrix, based on the results of reference laboratories. The other is the designation of these reference laboratories by comparing their results with the assigned values. Because each of these parts requires the other, an iterative procedure is necessary. As an example, the results of a proficiency test for the analysis of copper in wastewater are used to compare the calculated values with those from other methods, e.g., the Huber estimation. Received: 25 September 2000 Accepted: 9 December 2000     

Microbiological proficiency testing: a personal perspective

The author considers the fundamental differences between analyses in microbiology and those in chemistry and physics, deducing special issues for microbiological proficiency testing. He concludes that the variability and uncertainty implicit in microbiological analysis requires a broader range of proficiency scheme providers providing a broader range of services than in chemistry and physics. Received: 10 November 2000 Accepted: 3 December 2000     

Uncertainty evaluation in proficiency testing: state-of-the-art, challenges, and perspectives

The evaluation of measurement uncertainty, and that of uncertainty statements of participating laboratories will be a challenge to be met in the coming years. The publication of ISO 17025 has led to the situation that testing laboratories should, to a certain extent, meet the same requirements regarding measurement uncertainty and traceability. As a consequence, proficiency test organizers should deal with the issues measurement uncertainty and traceability as well. Two common statistical models used in proficiency testing are revisited to explore the options to include the evaluation of the measurement uncertainty of the PTRV (proficiency test reference value). Furthermore, the use of this PTRV and its uncertainty estimate for assessing the uncertainty statements of the participants for the two models will be discussed. It is concluded that in analogy to Key Comparisons it is feasible to implement proficiency tests in such a way, that the new requirements can be met. Received: 29 September 2000 Accepted: 3 December 2000     

Reliability of food measurements: the application of proficiency testing to GMO analysis

This paper reviews the experience of the Food Analysis Performance Assessment Scheme (FAPAS^®) in operating a proficiency testing scheme for the analysis of genetically modified (GM) food. Initial rounds of proficiency testing have shown a tendency for laboratories to over-estimate GM levels, results obtained by polymerase chain reaction (PCR) and enzyme-linked immunosorbent assay (ELISA) detection methods to be significantly different and that data are skewed and not normally distributed until log-transformed. During the initial rounds, it was found that for analysis and quantification of GM material, it was not possible to assign a target value for standard deviation external to the round data, from which performance could be assessed. However, when working in a log scale, the internally derived, robust standard deviation (^σ) was found to be constant and could be used directly to predict a target value (σ) for performance assessment. Results from the first four rounds have provided valuable information and a general overview of laboratory ability. Choosing a target value for standard deviation which reflects current best practice has enabled laboratory performance to be assessed. Issues surrounding the assessment of performance are discussed which highlight some of the implications raised as a result of this initial assessment, regarding the enforcement of European labelling legislation.     

Practical uses of proficiency testing as valuable tools for validation and performance assessment in environmental analysis

Besides their role as an external quality control tool, PT results or samples could be used as an alternative to fulfil some of the quality assurance requirements such as analytical precision, uncertainty assessment, and internal quality control. This additional use of proficiency testing could help laboratories to reduce the financial impact of their quality assurance process. The purpose of this paper is to highlight some practical uses of PT results or samples in the environmental analytical field, which have been implemented at ISSeP (Institut Scientifique de Service Public), either for method validation or for internal quality control.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

Advances for proficiency testing for genetic laboratory science

Nucleic acid based clinical genetic testing has undergone explosive growth in recent years due in large part to the human genome project. Characterization of the human genome has led to a molecular understanding of the pathogenesis of many human diseases, and ultimately to clinical molecular tests becoming routinely used to diagnose a wide diversity of diseases. This rapid growth in clinical molecular genetic testing coupled with the complexity of the analytical procedures underscores the necessity for proficiency testing (i.e. external quality assessment) to allow laboratories offering such services the ability to evaluate their analytical procedures via inter-laboratory comparisons. The American College of Medical Genetics (ACMG) in partnership with the College of American Pathologists (CAP) have been offering proficiency testing for clinical molecular genetics laboratories since 1995, and presently have more than 230 laboratories from 11 countries enrolled in this program. This paper describes the evolution of this program and several challenges encountered in the delivery of a proficiency testing program for laboratories offering clinical molecular genetic services. Received: 13 April 2002 Accepted: 18 July 2002     

Quality in clinical laboratory measurements

 Clinical chemistry deals with measurements and observations using samples from patients in order to supply clinicians with information to support their decisions in diagnosis and treatment. The discipline utilizes advanced chemical and biochemical methods and also sophisticated instrumentation which allows a high throughput. Clinical chemistry has a long tradition of quality assessment and improvement. The major tools have been proficiency testing or external quality assessment and internal quality control. Nowadays, total quality management has become widely recognized, and accreditation according to international, regional or national schemes has contributed to the design of quality systems and improvement of the reliability of results from clinical laboratories.     

Proficiency testing in cervical cytopathology

Cervical cancer is a leading cause of cancer death in women and may be significantly reduced by the introduction of effective screening and treatment programs. Although proficiency testing programs should, in principle, contribute to the reduction of cervical cancer mortality, the data does not provide convincing evidence for such a result. In this paper we review some results from current and past cytology proficiency testing programs, outlining the challenges to be overcome if these programs are to have an effect on cervical cancer mortality. Received: 23 April 2002 Accepted: 11 July 2002     

The rational use of proficiency tests and intercomparisons

 Interlaboratory comparisons (in the following abbreviated as intercomparisons) regarding tests, analyses or measurements are among the most worthwhile measures a laboratory can take in order to confirm that its services to clients include the provision of correct results within a stated uncertainty. They give a picture of the whole performance of the laboratory, and they should be used much more than at present. Unfortunately such intercomparisons are, and are made, expensive and prestigious by the formalisms employed in many cases. The connection between intercomparisons and proficiency tests and their use for different purposes is briefly discussed. Some suggestions are made on how to improve the present state of the art, i.e. how to increase the use of intercomparisons, how to perform them efficiently and how to make optimal use of the results. Received: 6 December 1997 · Accepted: 30 January 1998