Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the Eurachem Workshop, Rome, 5–7 October 2008 is provided. The eight WG’s covered a range of issues concerned with current practice and future directions; how frequently should laboratories participate in PT/EQA? (WG1); developments in PT/EQA within the EU—what is required in future? (WG2); what issues do developing countries face with regards to PT/EQA? (WG3), what issues are specific to microbiology PT/EQA? (WG4); what new fields are emerging for PT/EQA? (WG5); what will be the impact of the new ISO/IEC 17043 standard? (WG6); do current PT/EQA schemes meet the needs of participants? (WG7); and what are the issues that affect the quality of proficiency test items? (WG8). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Do we need to accredit proficiency testing schemes?

 Proficiency testing (PT) is being increasingly used as an important quality assurance tool for laboratories. The subject of quality of the providers of PT schemes has been discussed increasingly in recent years. Some countries have implemented systems for the accreditation of PT schemes. This paper looks at the background to the accreditation of PT schemes, the likely mechanisms which could be employed for accreditation, and some of the practical aspects.     

Experience with in-house PT-schemes in the chemical industry

There are many different means of demonstrating the quality of performance of an analytical laboratory. Proficiency testing (PT) is just one! As in other analytical fields, interlaboratory comparisons play an important role in the chemical industry. Collaborative trials or method performance studies do have a long tradition in this field. Sometimes they were designed as laboratory performance studies with the clear aim of making analytical results comparable, e.g. petrol, coal, gas, noble metals analyses – not to mention the biggest PT scheme run on a daily world-wide basis – trade itself. All this is an ongoing process, which started long before the idea of assessing and accrediting the performance of analytical laboratories was born. However, when striving for accreditation in 1996, the analytical production laboratories of the Chemicals Business Unit of the Bayer AG in Germany implemented another facet of PT schemes. In-house-PT schemes are performed regularly and turned out to be useful in evaluating, monitoring, and thus improving, the quality of routine analytical work. Received: 5 December 2000 Accepted: 15 January 2001     

Interlinkages and recognition of proficiency-testing schemes

 Because proficiency testing (PT) is increasingly used for the accreditation of testing laboratories and as a tool for backing up existing multilateral recognition arrangements between accreditation bodies, the question of performance and comparability of the proficiency-test providers arises. In this paper different approaches to assess the equivalence of European PT schemes and the competence of their providers are presented. As a first step a workshop is proposed to agree on a pilot study. The final aim is to create confidence in the existing PT schemes in Europe and to use them as common European tools.     

Step-by-step implementation of a quality system in the laboratory

In recent years laboratories have undergone huge transformations due to the technological development of inspection and testing equipment; the introduction of computerised and automated systems; keen competitiveness between companies/laboratories as a result of demand within Europe and on the international market; and greater consumer awareness of the quality of the products available. Laboratory accreditation, though a voluntary process, is formal recognition by an accreditation body of the laboratory's competence to carry out certain tests. This article presents those aspects which should be taken into account in the step-by-step implementation of a quality system and also makes reference to the requirements for the operation of accredited laboratories in accordance with European Standard EN 45001.     

The influence of a quality system according to ISO/IEC 17025 on the performance of Brazilian laboratories in proficiency testing in the environmental area

This paper presents a performance analysis of laboratories in proficiency testing (PT) with the main objective of evaluating whether having accreditation to ISO/IEC 17025 or not having an implemented quality system has an influence on the performance of these laboratories. This research was conducted with data from an accredited PT scheme of physical–chemical water analyses. Nine metals were considered in the analysis, with a total of 896 results. The performance indicator was z-score. The proposed analysis was conducted through statistical methods of analysis of variance, Dunnett’s test, Chi-square and cross-tabulation. It was verified that there is a significant difference between the performance of accredited laboratories, recognised laboratories, and laboratories that do not have an implemented quality system. It was established that an accredited or recognised quality system has a positive influence on the laboratory performance, resulting in a lower z-score (in module) and in a larger percentage of satisfactory results.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the EURACHEM Workshop, Istanbul, 3?C6 October 2011, is provided. The six WGs covered a range of issues concerned with current practice and future directions; implementing the requirements of ISO/IEC 17043 by PT/EQA providers (WG1); accrediting PT/EQA providers to ISO/IEC 17043 (WG2); pre- and post-analytical aspects in PT/EQA (WG3); evaluating participant performance in qualitative PT/EQA schemes (WG4); establishing PT/EQA schemes in developing countries (WG5); and establishing acceptability criteria in microbiology PT/EQA schemes (WG6). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end-users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Accreditation of proficiency-testing schemes in The Netherlands

Proficiency testing by laboratories, national accreditation bodies, and other third parties is becoming more and more considered as a standard and integral part of the quality control system. Therefore, it is of utmost importance that the quality of the provided proficiency-testing (PT) service is outstanding. If PT-schemes are set up in order to help laboratories monitor and improve their quality, PT-schemes need not only be of high quality themselves, but the organizer also needs to be able to demonstrate this. In The Netherlands formal accreditation of the organization of proficiency testing schemes is used as a tool to guarantee high quality schemes and also to enable organizers to demonstrate their competence. Since 1996, the Dutch Council for Accreditation (RvA) has used the ISO-Guide 43–1 to assess PT-organizers in The Netherlands. From January 2000, the ISO-guide 43–1 was replaced by the ILAC G13 document for assessing organizers. Up till now, four institutes have been accredited by the RvA for the organization of PT-schemes.     

Surveys on the accreditation of providers of proficiency testing and external quality assessment schemes

Two surveys among providers of proficiency testing (PT) and external quality assessment (EQA) schemes were carried out during 2004 and 2005. The main objectives were to explore the current status of accreditation/certification and collect the providers’ views. Information based on the response from 160 providers in 32 countries reveals a strong tendency towards accreditation of PT/EQA. It is shown that this type of accreditation is based on several combinations of normative documents, hence illustrating a lack of harmonisation of national accreditation bodies. The surveys also show that schemes are operated under considerably different conditions and that providers’ competence may or may not be underpinned by other certification and/or accreditation. This paper elaborates on a number of issues related to PT/EQA accreditation, including customers’ views, normative documents, providers’ experience from the accreditation process, views expressed by international organisations, and effects of accreditation on participation fees, quality and availability.     

Quality specifications for evaluation and comparison of performance among external quality assessment schemes in occupational and environmental laboratory medicine

Quality specifications (QS) are proposed for lead in blood and for aluminium, copper, selenium and zinc in serum as part of the aim to set standards of performance for laboratories so that results can be demonstrated to be fit for the purpose to which they are applied. The QS were established taking account of the analytical state-of-the-art, physiological variations in the concentrations of the analyte and the clinical purpose for which the assay is to be used. A procedure was devised that uses these QS to give equivalence of assessment among external quality assessment schemes (EQAS), thus avoiding conflicting information which has been demonstrated in the past. Advantages of this procedure are: to provide direct comparison of performance of laboratories taking part in different schemes, to provide equivalence of assessment of laboratory performance necessary to establish mutual recognition agreements, and to demonstrate the fitness for purpose of results from participants.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

External quality assurance programmes in medical laboratories

 Medical laboratories have a long tradition of external quality assessment. Starting from pure quality control of laboratory performances, most schemes have evolved to a powerful tool for improving quality of clinical outcome of results. External quality assurance in medical laboratories not only includes laboratory performance evaluation, but also evaluation of method performance, post-marked vigilance, training and help. In the future, the quality of programmes must further be improved by accreditation of schemes and by using electronic data interchange. Received: 9 December 2000 Accepted: 14 December 2000     

Performance evaluation for proficiency testing with a limited number of participants

Proficiency testing (PT) is an essential tool for laboratories to assess their competency. Also, participation in PT has become one of the mandatory requirements for laboratory to seek accreditation according to ISO/IEC 17025. For this reason, the effectiveness of performance evaluation by PT scheme is of great concern for the participants and for accreditation bodies as well. In practice, owing to unavailability of other appropriate alternatives, PT scheme providers may have to choose using consensus values to evaluate the performance of participants. However, such consensus values approach was not recommended by relevant international guidelines for PT schemes with limited number of participants. With the use of Monte Carlo simulation technique, this study attempted to investigate the effectiveness of using consensus values for performance evaluation in PT schemes with limited number of participants. The simulation process was schemed according to the statistical model provided by ISO 5725-1 for laboratory measurement results, which covered components like method bias, laboratory bias, and measurement precision. The effectiveness of the consensus value approach was expressed as the percentage of participants in a simulation run could get the same evaluation result, either satisfactory or unsatisfactory, against the “true value.” The findings indicated that the number of participants, choice of consensus values, mass fraction of analyte, method bias, laboratory bias, and measurement repeatability of participating laboratories would all affect the effectiveness of the consensus value approach but at different extent. However, under certain circumstances, use of consensus value could still be considered as an acceptable approach for performance evaluation even the number of participants was limited. Some of the findings were further verified using real data from PT schemes where appropriate certified reference materials or reliable reference values were available.     

Implementing quality assurance in an R&D environment at the Belgian Nuclear Research Centre – SCK·CEN

If we agree that quality assurance (QA) in R&;D indeed provides added value, just how do we show the public that such a QA system is operating well? At the Belgian Nuclear Research Centre (SCKÖbullet CEN), where a QA system has been implemented by various laboratories in accordance with the EN 45001 standard within the general framework of ISO 9001 at the institutional level, several laboratories have already been accredited. At the request of one of our customers the Belgian Agency for Radioactive Waste Management (NIRAS/ONDRAF), we implemented a quality system for research projects related to the characterization, treatment and processing of radioactive wastes in view of disposal. We obtained accreditation for this research in 1999. In this paper, we discuss the implementation of our QS and important issues related to the overall management of the QA system, broken down into three parts: organizational, technical and project-specific quality elements such as the QA plan and internal communication.     

Anti-HIV quality assurance programs in Australia and the southeast Asian and Western Pacific regions

Anti-HIV testing is the most regulated area of laboratory medicine in Australia. These regulations have placed the National Sero-logy Reference Laboratory, Australia (NRL) in a unique position to implement a comprehensive quality assurance (QA) program for HIV testing. The elements of our QA program include pre-market evaluation of assays, external quality assessment schemes (EQAS), quality control, specificity monitoring, consultations, and training workshops. The results of the NRL EQAS for Australian laboratories were compared with those of a program developed by the NRL for reference laboratories in the Southeast Asian and Western Pacific (SEAWP) regions. For laboratories authorized to use tests for HIV in Australia, participation in the entire QA program is mandatory, whereas the SEAWP EQAS program is voluntary. While the overall percentage of discrepant results for these programs are similar, the percentage of false negatives, variation in laboratory results, and choice of assay differ. These differences have decreased with time with improvements in assays and laboratory testing practices. The educational component of both EQAS, which comprises workshops, laboratory questionnaires, consultancies, and newsletters, has had a critical impact on the testing practices of laboratories. Received: 30 October 2000 Accepted: 9 December 2000     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working group discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the joint EURACHEM/CITAC/EQALM workshop, Bracknell, UK, 16–18 February 2003 is provided. The nine WGs covered a range of issues concerned with current practice and future directions; PT/EQA as a tool for regulators (WG1); PT/EQA as a tool for accreditation (WG2); evaluation of performance and uncertainty (WG3); frequency of PT/EQA participation (WG4); selection of appropriate PT/EQA schemes (WG5); added value of PT/EQA and cost benefit evaluation (WG6); global harmonisation and rationalisation (WG7); new technical areas and challenges in PT/EQA (WG8); and accreditation of PT/EQA providers (WG9). Participants with different backgrounds were on each WG in order to capture a range of views and experience from different sectors. The discussions reflected on the keynote lectures and built, in many cases, on discussions at previous workshops in 2000 and 2002.     

Environmental analysis in the Czech Republic: organisation of proficiency testing and its use in laboratory assessment

The history, origin, and development of a system for monitoring and assessing water and other environmental laboratories in the Czech Republic is described. The system started in 1991 and has matured to its present complexity with similarities to the accreditation systems found in other countries. Differences from internationally recognized procedures are being corrected step by step. During the first year of its existence ASLAB, as part of its brief, organised proficiency testing (PT) programs for fifty laboratories. Today the total number of regularly participating laboratories exceeds 700 from the Czech Republic, the Slovak Republic, and Germany. This paper describes the ASLAB PT system, discusses some experiences with its use, and describes the use of PT results in assessment of the competence of laboratories. Received: 12 October 2000 Accepted: 7 January 2001     

Twenty-year review of laboratory performance on enteric parasitology external quality assessment surveys 1992–2011, Ontario Canada

Quality Management Program?CLaboratory Services (QMP?CLS) is a mandatory external quality assessment (EQA) and accreditation programme for laboratories in Ontario. This report summarizes performance of Ontario laboratories on QMP?CLS enteric parasitology EQA surveys and compares this performance over time. The results reported by participating laboratories were assessed against the expected results as determined by reference or referee laboratories. The number of laboratories licensed for parasitology decreased dramatically over the 20?years. On average, more than 80?% of participants and often more than 90?% were able to detect and identify the species and stages for the majority of parasites. Many of the challenges contained multiple parasites and about half contained low numbers, which affected performance on individual samples. The level of performance by Ontario laboratories on parasitology EQA surveys was consistently high over the 20?years. The parasites that presented the most difficulty were Enteromonas hominis and Trichomonas hominis.     

The accreditation of reference material producers — EUROLAB's position

A third-party assessment of reference materials (RMs) and their production is not of the highest priority for RM users. But if RM producers do want a third-party involvement this should be realised within the frame of existing conformity assessment schemes. Nevertheless, the right and responsibility of the laboratories to select the most appropriate RMs for their tasks must not be restricted by accreditation bodies or other interested parties.     

Quality assurance in biological monitoring of environmental exposure to pollutants: from reference materials to external quality assessment schemes

The presence of chemicals in the environment is a matter of concern in that it poses potential health risks. At present, exposure to toxic chemicals and their biological and biochemical effects can be better estimated by biological monitoring, through the systematic collection of specimens from potentially exposed humans. Biological monitoring of human exposure to environmental pollutants is hampered by the difficulty to assess data reliability. As a consequence, the validity of biological monitoring should depend on the strict implementation of a quality assurance (QA) program, which includes a series of procedures aiming to ensure that laboratory results meet defined standards of quality and are reliable. For the validation and monitoring of methods’ performance, to ensure the trueness of measurements and to warrant the traceability to international standards, reference materials (RMs) and certified reference materials should be used. Internal quality control and external quality assessment (EQA) are part of overall QA and are carried out to verify that analytical errors are compatible with the specific requirements or needs of the user. In particular EQA schemes (EQAS) allow to test independently the analytical performance of participating laboratories. In the last decades, increasing concern has been raised by urban air pollution; lead and benzene, two gasoline components released by motor vehicle exhausts, are known to be toxic to humans. For biological monitoring of lead exposure of the general population, screening campaigns, utilizing lead in blood as a biomarker, have been carried out since the 1970s. Strict strategies were adopted to ensure data comparability, including the preparation of RMs, the organization of EQAS and the cross-exchange and analysis of blood samples between laboratories. Biological monitoring of benzene exposure could be carried out by means of various biomarkers such as benzene in blood and benzene, trans,trans-muconic acid (t,t-MA) and S-phenylmercapturic acid (S-PMA) in urine. At present, few RMs and EQAS are available for these biomarkers. A pilot EQAS for t,t-MA in urine, adopted to assess the reliability of data regarding benzene exposure, has been organized and carried out between 1996 and 1997 in Italy. From the accrued experience, it clearly emerges the importance of strategies designed to guarantee the quality of biological monitoring data. The use of RMs and the participation in EQAS are highly recommended in order to improve the global performance of methods and laboratories involved in biological monitoring.     

Uses and abuses of PT

Proficiency testing (PT) is a very valuable quality tool for laboratories making all kinds of analytical measurements, in chemistry, microbiology and laboratory medicine. Compared with other quality tools, which are largely concerned with INPUTS (e.g. staff training, calibration, documentation), PT is concerned with OUTPUTS (i.e. the actual results produced by the laboratory). The quality of routine measurements and results, which would normally be reported to customers, both internal and external, should be demonstrated. This is a major reason for many laboratories participating in PT, even if they are not accredited, or use very few, if any, other quality tools. This is important, for it enables the laboratory manager to measure the efficacy of the overall quality system, and to determine if the inputs to this are appropriate. This should be done in a spirit of learning—it was discussed at the previous workshop whether PT schemes are teachers or policemen, and the consensus was that MOST are teachers (or at least try to be). Learning from mistakes, investigating problems, implementing corrective actions and then observing whether thisis successful in the next PT represents what many in our profession would regard as a proper and effective use of PT. Although this may be the intended primary use of PT by laboratories, we must not forget that PT has other valid uses. In particular, the use of PT to demonstrate competence to third parties—accreditation bodies, regulators, and customers—has become more important in the last few years. However, this development, however valuable, can lead to abuse of PT scheme data. This paper will highlight briefly some examples of both proper use and abuse of PT. These are taken from my experience with PT schemes over many years.