Proficiency testing and external quality assurance: crossing borders and disciplines

Proficiency testing and external quality assurance of medical laboratories is now entering its sixth decade. These activities comprise a broad range of applications including: providing participants and public health authorities with estimates of measurement uncertainty and national infrastructure; providing education; provision of a practical basis for accreditation and regulatory compliance. All branches of medical laboratory science have employed external quality assurance as a basis for improvement and comparability. The opportunities and challenges reviewed here include: the proper establishment of multiple target values in comparison to a system of traceability to reference or definitive methods; the problems of matrix effects and commutability of patient and proficiency test samples; generating information on laboratory infrastructure and trends in analytical technique and performance; providing education and setting goals for laboratory improvement; problems of specimen distribution; application of Internet technology; the role of programs in legal mandates and accreditation. Received: 24 April 2002 Accepted: 11 July 2002     

Proficiency testing schemes: solutions for homogeneity control

The organisation of the homogeneity control of samples prepared for proficiency testing schemes (PTS) is a challenge for the PTS organisers and engenders logistical and financial problems. Samples used in PTS need to be homogeneous in order to be sure that if a laboratory has a result different from the other laboratories, its error can be attributed to its analysis method and not to its sample. The present paper describes the solutions found by the Bureau Interprofessionnel dEtudes Analytiques (Bipea) in order to answer the homogeneity control requirement. The methodologies of homogeneity control are adapted to specific characteristics of the products proposed in Bipeas PTS. The first methodology of control, the preparation approach, concerns specific PTS where the samples are formulated and where the assigned values are known and chosen. The second methodology, the target approach, concerns mainly the field of the environment and implies the choice of homogeneity targets. The third methodology, the global approach, concerns the agricultural and agro-food fields and involves comparison of near-IR spectra samples. Each methodology of control has its advantages and disadvantage and is suited to specific products prepared with specific preparation procedures. It has to be stressed that the solutions developed answer quality requirements but are not a substitute for the use of adapted and detailed preparation procedures with validated and regularly checked devices and for the a posteriori control of the results of the laboratories.     

Proficiency testing schemes for therapeutics and toxicology

 This report addresses the proposition: current proficiency testing (PT) schemes for therapeutics and toxicology can easily engender complacency. The proposition was based on the premise that there are substantial between-laboratory differences and objectives in the supply of services for the measurement of drugs, as a result of which it is difficult to design PT schemes which adequately cover the requirements of all laboratories. Thus, acceptable performance in a PT scheme may lead participants to consider that they have tested their procedures more rigorously than is the case. Four areas, validity of the PT scheme, PT material, frequency of testing and acceptance criteria are examined. Received: 15 April 2000 · Accepted: 15 April 2000     

Proficiency testing in cervical cytopathology

Cervical cancer is a leading cause of cancer death in women and may be significantly reduced by the introduction of effective screening and treatment programs. Although proficiency testing programs should, in principle, contribute to the reduction of cervical cancer mortality, the data does not provide convincing evidence for such a result. In this paper we review some results from current and past cytology proficiency testing programs, outlining the challenges to be overcome if these programs are to have an effect on cervical cancer mortality. Received: 23 April 2002 Accepted: 11 July 2002     

Proficiency testing as a tool for implementing internal quality control: the case of ochratoxin A in cocoa powder

This paper describes the interlaboratory study aimed at assessing the performance of 18 laboratories (14 national and 4 European) for Ochratoxin A (OTA) determination in cocoa powder samples. The study was tested at three levels of OTA covering the range in which presumably European regulatory limits could fall in the near future. For the extraction step, almost all laboratories used an aqueous solution of sodium hydrogen carbonate with the exception of one laboratory using dichloromethane consistently with the ELISA procedure adopted in the study. The clean-up step was performed by utilizing the immunoaffinity columns by the two main manufacturers (R-Biopharm Rhone and VICAM) and for the quantitative analysis, HPLC was used by all the participants except one using ELISA. From the output of the study, it can be concluded that at low level (0.19 μg/kg) 10 out of 18 (56%), at medium level (0.45 μg/kg) 11 out of 18 (61%), and at high level (1.45 μg/kg) 12 out of 18 (67%) results fell within the satisfactory ranges. This interlaboratory study provides an estimate of the performance of national and European laboratories involved in OTA determination in cocoa powder samples, which sounds extremely valuable in view of potential future legislation by the European Commission.     

Proficiency testing programmes as a tool in food quality assurance: Overview of Italian experiences

Proficiency testing involves the performance of test procedures on routine samples by a number of laboratories. Interlaboratory proficiency testings provide multiple benefits to participants since they play a key-role in the total quality control of laboratory activities. They serve as a means of self-improving, as a mechanism of continuing education and as a source of information for accreditation agencies. This review highlights basic principles, benefits, criteria and capabilities of a proficiency testing programme for food analysis laboratories as well as their role in the implementation of rapidly developing food control legislation.     

Proficiency testing on the analytical control of precious metal production

Our experience of executing proficiency testing programs on the analysis of precious metals and precious metal bearing materials is summarized. Six proficiency testing programs of the ”distributed sample testing” type were carried out using samples essentially distinct from each other: (I) certified samples of fine gold and fine silver, and (II) recoverable raw (scrap and catalyst waste) containing precious metals of unknown contents. The test results were assessed using the London Bullion Market Association (LBMA) rules and by Z-criteria using the ISO/IEC Guide 43 scale. Satisfactory results were presented by all participants for the analysis of certified samples of fine gold and fine silver. The results of catalyst waste analysis agreed rather well, whereas the scrap analysis revealed an essential discrepancy in the data for some samples. The reasons for such discrepancies are discussed. Received: 9 June 2000 Accepted: 21 November 2000     

Proficiency testing in occupational and environmental health – Current practice and international initiatives for harmonisation

An increasing number of proficiency testing schemes (PT schemes) related to occupational and environmental health have been organised. Most schemes emphasise the importance of the validation of analytical results. With regards to harmonisation of the schemes, there are many differences between the schemes at many levels. These include factors concerning their relationship with legislation, national status, type and quality of proficiency testing material, analytical range and priorities for future development. Since differences between PT schemes have been recognised at the European level by organisers of PT schemes and external quality assessment schemes (EQASs) it seems appropriate to reinforce collaboration between scheme organisers in order to improve the quality of analytical performance in occupational and environmental health.     

Proficiency testing in analytical laboratories: how to make it work

 This paper covers the role of proficiency testing schemes in providing an occasional but objective means of assessing and documenting the reliability of the data produced by a laboratory, and in encouraging the production of data that are "fit-for-purpose". A number of aspects of proficiency testing are examined in order to highlight features critical for their successful implementation. Aspects that are considered are: accreditation, the economics and scope of proficiency testing schemes, methods of scoring, assigned values, the target value of standard deviation σ_p, the homogeneity of the distributed material, proficiency testing in relation to other quality assurance measures and whether proficiency testing is effective. Stress is placed on the importance of any proficiency testing scheme adhering to a protocol that is recognised, preferably internationally. It is also important that the results from the scheme are transparent to both participating laboratory and its "customer". Received: 03 November 1995 Accepted: 20 November 1995     

Proficiency testing as tool for ISO 17025 implementation in National Public Health Laboratory: a mean for improving efficiency

Public Health Laboratories (PHL) as part of Public Health Services are involved in law enforcement and ensuring food and water quality. The laboratories had to go through an organizational and monetary change imposed by the growing and changing needs of the state of Israel. The laboratories had to become more modern and to implement new and more sophisticated testing methods. Another requirement was to perform a steadily increasing number of tests and to be more flexible towards customers’ demands. Yet, the budget was not changed accordingly, as the Public Service to which the laboratories belong to do not respond to changing needs. Management realization was that the accreditation process could be used as a tool to achieve organizational and cultural change. Understanding and transformation were required throughout the organization, including management. Proficiency testing is performed for all testing methods in all areas even for non-accredited tests such as clinical tests. Proficiency testing was used as a tool for organizational culture change. It is a great index with game elements, that gives employees and managers the possibility for comparison, fixing problems and corrective action. The demands of ISO 17025 made it necessary to change peoples’ attitudes and views on both professional and communication levels. Laboratory quality consists now on the four main principles, described in the 5M&E model which is typical to small organizations and which ensures a constantly improving system: ”Policy statement”, ”Machine”, ”Material”, ”Measurement”, ”Method”, ”Manpower”, and ”Environment” targeted to achieve ”Quality upgrade”. Slowly we succeeded in providing better and more reliable services and have increased our income on what would hopefully become our way to financial independence. We hope this process would provide the Ministry of Health with a better chance to public health using the same financial sources. Received: 25 October 2000 Accepted: 12 December 2000     

The increasing demands of quality assurance in chemical testing

The chemical and microbiological testing community is going through a rather difficult period of change. Publication of ISO/IEC 17025 (General requirements for the competence of Calibration and Testing Laboratories, 1999) [1] is placing additional demands on testing laboratories to ensure traceability and estimate uncertainty in their measurements. At the same time, laboratories must remain mindful of the need to provide relevant, timely and economic services to their clients. International Accreditation New Zealand (IANZ) and its accredited laboratories are currently focussing on establishing realistic methods for ensuring traceability to national and international standards and estimating measurement uncertainty. To this end, IANZ recognizes that it has an important role in providing as much advice and assistance as possible, not only to its accredited chemical laboratories but also to all those contemplating accreditation in New Zealand.

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working group discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the joint EURACHEM/CITAC/EQALM workshop, Bracknell, UK, 16–18 February 2003 is provided. The nine WGs covered a range of issues concerned with current practice and future directions; PT/EQA as a tool for regulators (WG1); PT/EQA as a tool for accreditation (WG2); evaluation of performance and uncertainty (WG3); frequency of PT/EQA participation (WG4); selection of appropriate PT/EQA schemes (WG5); added value of PT/EQA and cost benefit evaluation (WG6); global harmonisation and rationalisation (WG7); new technical areas and challenges in PT/EQA (WG8); and accreditation of PT/EQA providers (WG9). Participants with different backgrounds were on each WG in order to capture a range of views and experience from different sectors. The discussions reflected on the keynote lectures and built, in many cases, on discussions at previous workshops in 2000 and 2002.     

Benefits arising from proficiency testing schemes: The comparison of analytical methods

 The primary objective of proficiency testing (PT) is in the provision of information and support to participating laboratories, to enable them to monitor and improve the quality of their measurements. However, other benefits can be obtained from PT. These include the comparison of data for a given measurement by different methods, the validation of new methods, and the provision of information for laboratories' customers and accreditation bodies. This paper considers the subject of method comparison, and highlights some of the approaches which can be followed, as well as the practical use to which this can be put, to benefit the analytical community more widely. This is illustrated by a case study concerning the measurement of haze in beer. In this study the United Kingdom Institute of Brewing (IoB) conducted a survey of participants in the Brewing Analytes Proficiency Scheme (BAPS). From the survey data taken together with data from the BAPS scheme, the IoB is now in a position to give guidance on the use of particular instruments and procedures, as well as consider changes to the scope of the BAPS scheme to provide greater benefits for participants concerned with measuring haze. Received: 3 March 1998 · Accepted: 9 June 1998     

ISO/IEC 17043: the new International Standard for proficiency testing

The revision of ISO/IEC Guide 43 is well underway by CASCO Working Group 28, with the author as Convener. The ballot on the committee draft was successfully completed in June 2008. The Working Group met in September to resolve the CD comments and to recommend that the document should advance as an ISO/IEC DIS. The document is in harmony with ISO/IEC 17025 (2005), ILAC G13 (2007), and the IUPAC Harmonized Protocol (2006) for proficiency testing. Major changes from Guide 43 include considerations for inspection bodies, requirements for metrological traceability of assigned values for calibration schemes, limitations on subcontracting, requirements for reporting the uncertainty of assigned values, and for including method information and technical commentary in scheme reports. Minor changes include requirements for equipment used in the manufacture and testing of proficiency test items, considerations for handling laboratories’ reported uncertainties, and the issuance of certificates of participation. These changes have not generated any substantial objections from the CASCO membership and liaisons.     

The WADA Proficiency Testing Program as an integral part of the fight against doping in sport

This article provides a brief perspective of the World Anti-Doping Agency (WADA) in the context of the fight against doping in sport and of the WADA Proficiency Testing Program as an essential element of the World Anti-Doping Program. The WADA Proficiency Testing Program exists within the framework of the WADA-accredited laboratories and is aimed at improving the effectiveness of the fight against doping by testing the ability of the laboratories to reliably identify and measure the substances and methods defined in the WADA Prohibited List. The paper provides an overview of the WADA Proficiency Testing scheme with respect to organization, test composition and evaluation of anti-doping methods and techniques. Scientific research is mentioned in the context of WADA’s involvement in generating reference materials for improvement of analytical activities of the WADA-accredited laboratories. WADA’s funding of a reference material for 19-norandrosterone in human urine is illustrated as an example.     

The establishment of quality systems in veterinary diagnostic testing laboratories in developing countries: experiences with the FAO/IAEA External Quality Assurance Programme

Quality systems, established to internationally accepted standards, are one mechanism that can assist in evaluations of the sustainability of technology transfer, the proficiency of the user, and the reliability and comparability of data generated, resulting in potential enhancement of laboratory credibility. The means of interpreting existing standards and implementing quality systems in developing country veterinary diagnostic laboratories has become a significant adjunct to the technology transfer element within the Food and Agriculture/ International Atomic Energy Agency, FAO/IAEA programme. The FAO/IAEA External Quality Assurance Programme (EQAP) is given as an example for an initial step towards enhancing the “quality” culture in developing country veterinary laboratories. In 1995 the EQAP began as an effort to assure that test results emanating from laboratories using FAO/IAEA ELISA kits for animal disease diagnosis are valid. For this purpose 15 international external quality-assurance rounds have been performed to date for a variety of animal diseases e.g. Rinderpest, brucellosis, trypanosomosis, and foot-and-mouth disease (FMD). Results indicate that the EQAP is a valuable tool in the assessment of both the results provided by, and use of the ELISA kits provided through, the joint FAO/IAEA programme. Furthermore EQAP can assist laboratory diagnosticians to enhance quality control/quality assurance (QC/QA) procedures for conducting FAO/IAEA ELISAs and to advise on the implementation of similar QC/QA procedures in other laboratory activities. Based on the experiences made during the implementation of the EQAP a proposal for establishing a quality system standard was ratified through the World Organization for Animal Health (OIE) general conference in May 2000. The OIE Standard On Management And Technical Requirements For Laboratories Conducting Tests For Infectious Animal Diseases is based on ISO 17025 and provides a clear formula for establishing quality systems in veterinary diagnostic laboratories world-wide.     

A statistical strategy for discouraging collusion in split-level proficiency testing schemes

A statistical strategy for discouraging collusion in split-level proficiency testing schemes was developed. When the samples are randomly labeled, the difference in the analyte levels in the two samples can be designed to maximize the probability of detecting collusion when it occurs, so that participants who colluded cannot easily receive satisfactory assessments. This favors providing convictive assessments of participants proficiency and giving the participants a level playing field.     

Practical uses of proficiency testing as valuable tools for validation and performance assessment in environmental analysis

Besides their role as an external quality control tool, PT results or samples could be used as an alternative to fulfil some of the quality assurance requirements such as analytical precision, uncertainty assessment, and internal quality control. This additional use of proficiency testing could help laboratories to reduce the financial impact of their quality assurance process. The purpose of this paper is to highlight some practical uses of PT results or samples in the environmental analytical field, which have been implemented at ISSeP (Institut Scientifique de Service Public), either for method validation or for internal quality control.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia