The challenges and benefits of implementing the requirements of ISO/IEC 17043 by PT/EQA providers

ISO/IEC 17043 Conformity assessment??general requirements for proficiency testing is intended to replace previous international guides used to assess the competence of proficiency testing (PT) providers. It expands on the requirements of previous guides and is intended to accommodate PT providers of calibration laboratories and testing laboratories handling both qualitative and quantitative data. QMP-LS is an office-based external quality assessment provider for medical laboratories in Ontario, Canada and operates 46 different PT schemes for approximately 250 diagnostic tests. In 2010, these schemes were accredited to ISO/IEC 17043. Schemes included tests from the following disciplines: chemistry, hematology, microbiology, transfusion medicine, cytology, histology and genetics. Thirty of the schemes were qualitative. The challenges and benefits of implementing ISO/IEC 17043 are discussed, with particular emphasis on clauses addressing the following requirements: statistical design, determination of assigned value, homogeneity and stability testing, packaging, labeling and distribution, performance evaluation and subcontracting services.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the EURACHEM Workshop, Istanbul, 3?C6 October 2011, is provided. The six WGs covered a range of issues concerned with current practice and future directions; implementing the requirements of ISO/IEC 17043 by PT/EQA providers (WG1); accrediting PT/EQA providers to ISO/IEC 17043 (WG2); pre- and post-analytical aspects in PT/EQA (WG3); evaluating participant performance in qualitative PT/EQA schemes (WG4); establishing PT/EQA schemes in developing countries (WG5); and establishing acceptability criteria in microbiology PT/EQA schemes (WG6). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end-users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Evaluation of proficiency tests in microbiological analysis: enumeration of aerobic microorganisms

South Korea’s Ministry of Food and Drug Safety (MFDS) has been developing programs for the inspection and accreditation of food sanitation inspection institutions. Food sanitation inspection institutions such as MFDS regional offices, the Research Institute of Public Health and Environment and authorized private service providers in South Korea must participate in proficiency testing (PT) programs to comply with the Food Sanitation Act and MFDS Notification No. 2012-112. As the PT provider, the MFDS annually plans various microbiological and chemical PT programs for foods, cosmetics, and pharmaceutical products in accordance with ISO/IEC 17043. The aim of this project was to evaluate the performance of microbiological PT programs to ensure the quality of their routine test results. The test materials used were freeze-dried BioBalls from BTF Pty Ltd. Homogeneity and stability were investigated to assess the adequacy of the selected test materials. This project also contains data from inter-laboratory comparisons organized by MFDS in 2011 and 2012. More than 50 laboratories attended the PT program and submitted their results. Laboratory results were rated with z-scores according to the international standard ISO 13528. The results from 2011 and 2012 revealed that all participating laboratories had similar levels of proficiency. Most of the participants received a rating of “Satisfactory.” Moreover, the percentage of participants who received a rating of “Unsatisfactory” decreased from 3.5 % in 2011 to 2.0 % in 2012.     

Review of the new edition of ISO 13528

The international standard ISO 13528 is an important reference document for proficiency testing providers and testing laboratories alike, which provides guidance on the use of statistical methods in proficiency testing. With a view to bringing the document in harmony with the ISO/IEC 17043:2010, the first edition of ISO 13528 was revised and the new edition was published in September 2015. This paper attempted to review and discuss the major changes introduced in the new edition of ISO 13528, particularly the part on statistical design with respect to the objective of proficiency testing scheme and the considerations on assigned value determination and performance scoring. Moreover, procedures provided in the new edition of ISO 13528 for qualitative proficiency testing schemes and various robust statistical methods would also be briefly discussed in the paper.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the Eurachem Workshop, Rome, 5–7 October 2008 is provided. The eight WG’s covered a range of issues concerned with current practice and future directions; how frequently should laboratories participate in PT/EQA? (WG1); developments in PT/EQA within the EU—what is required in future? (WG2); what issues do developing countries face with regards to PT/EQA? (WG3), what issues are specific to microbiology PT/EQA? (WG4); what new fields are emerging for PT/EQA? (WG5); what will be the impact of the new ISO/IEC 17043 standard? (WG6); do current PT/EQA schemes meet the needs of participants? (WG7); and what are the issues that affect the quality of proficiency test items? (WG8). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Development of a standard for the organization of the interlaboratory comparisons in microbiology of food

The ILAC G13 Guide and the ISO IEC Guide 43-1 are the common, general and horizontal bases for accreditation of providers of proficiency testing and interlaboratory comparisons used by several accreditation bodies. Despite their widespread use, these guides omit specific technical requirements, and sometimes even elements of quality management, clearly defined for the organization of proficiency-testing schemes for microbiology of food. The Sub-Committee 9 “Microbiology of food” of ISO TC34 has created a working group (WG4) to establish a standard detailing such specific requirements for microbiology of food. This technical standard will describe those specific requirements of proficiency testing which organizers of proficiency-testing schemes and any subcontractors must satisfy in addition to the requirements of the ILAC G13 Guide and the ISO IEC Guide 43-1 to achieve accreditation or other recognition.     

Identification of possible technical problems in determination of the major inorganic constituents of brown-rice flour by evaluating proficiency test results

To support skill upgrading in analysis of inorganic constituents of environmental and food samples, the National Metrology Institute of Japan (NMIJ) and the National Food Research Institute (NFRI) have organized a proficiency test (PT) of determination of Mn, Fe, Cu, Zn, As, and Cd in brown-rice flour based on the international standard (ISO/IEC 17043:2010). One hundred and thirty-three sets of reports were assessed by use of the E _n -number and z-score approaches in accordance with ISO/IEC 17043 and the international harmonized protocol for PT. The PT results and analytical procedures, reported in detail, were reviewed, and possible technical reasons for questionable or unsatisfactory results are discussed.

The organization of proficiency testing in plant pathology (qualitative methods of analysis) according to the ISO/IEC 17043: example of the French national reference laboratory

The use of methods of analysis capable of producing reliable analytical results is a prerequisite to the effective control of quarantine plant pathogens. Proficiency testing is considered to be one of the most reliable ways to verify and coordinate analytical results. As a French national reference laboratory in plant pathology, the Anses Plant Health Laboratory organizes proficiency tests in order to ensure that officially approved laboratories (certified by government services) are capable of producing reliable analytical results for the detection of plant pathogens. Proficiency tests in plant pathology have a number of notable features including the processing of qualitative results. This paper presents the experience of the Anses Plant Health Laboratory’s Unit for Tropical Pests and Diseases (LSV-RAPT) as an organizer of proficiency tests in plant pathology. The LSV-RAPT has gained recognition for the methodology it has developed in the form of accreditation as a proficiency testing provider according to the ISO/IEC 17043. The methodology can be applied to many other disciplines that use qualitative detection methods.     

Proficiency testing schemes for the assessment of Legionella PCR methodologies

Standard operating procedures used for the detection of bacteria in environmental samples are primarily based on bacterial growth on specific culture media and confirmation by biochemical and/or immunological tests. In the case of Legionella, isolation on BCYE-?? medium is the standard method, although it presents a number of drawbacks, and for this reason, the implementation of molecular methods, mainly those based on PCR, has increased over the last years. Following the ISO/IEC 17025, laboratories need an external evaluation of their work to assure the quality of the results they are producing, and the participation in proficiency testing (PT) schemes is compulsory. For those water-testing laboratories using PCR methods for Legionella, we have developed a PT scheme accredited according to ISO/IEC 17043. The preparation and the statistical analysis of the results are performed following this standard and the ISO 13528. The used samples have a very rapid and easy to use format, consisting of tablets with inactivated freeze-dried Legionella cells or freeze-dried Legionella DNA. In this PT scheme, participants evaluate both Legionella pneumophila and Legionella spp. detection systems and control the whole PCR process from the water sample concentration until the PCR results.     

Metrology in service of society: the role of proficiency testing

A sound system of competent measurements, underpinned by standards and accreditation, is an essential component of the infrastructure of any modern society. Proficiency Testing is an indispensible tool for developing and maintaining this infrastructure. It is used to verify compliance with standards, for ongoing monitoring of laboratory performance, for demonstration of competence, and for evaluating the performance of measurement procedures; it is used to indicate performance problems that might not be detected otherwise; and it is used as a tool for education and self-improvement. The new International Standard, ISO/IEC 17043, contains requirements and recommendations to make optimal use of this important tool.     

ISO/IEC 17043: the new International Standard for proficiency testing

The revision of ISO/IEC Guide 43 is well underway by CASCO Working Group 28, with the author as Convener. The ballot on the committee draft was successfully completed in June 2008. The Working Group met in September to resolve the CD comments and to recommend that the document should advance as an ISO/IEC DIS. The document is in harmony with ISO/IEC 17025 (2005), ILAC G13 (2007), and the IUPAC Harmonized Protocol (2006) for proficiency testing. Major changes from Guide 43 include considerations for inspection bodies, requirements for metrological traceability of assigned values for calibration schemes, limitations on subcontracting, requirements for reporting the uncertainty of assigned values, and for including method information and technical commentary in scheme reports. Minor changes include requirements for equipment used in the manufacture and testing of proficiency test items, considerations for handling laboratories’ reported uncertainties, and the issuance of certificates of participation. These changes have not generated any substantial objections from the CASCO membership and liaisons.     

The third edition of ISO Guide 34: what were the drivers for the revision and what is new?

After the International Laboratory Accreditation Cooperation (ILAC) had taken in 2004, the resolution to conduct accreditation of producers of reference materials according to ISO Guide 34 ‘General requirements for the competence of reference material producers’ in combination with ISO/IEC 17025 ‘General requirements for the competence of testing and calibration laboratories’, ISO/REMCO, the ISO Committee on Reference Materials, decided in 2005 to revise ISO Guide 34 to align it closer with ISO/IEC 17025 and to clarify certain issues for accreditors and producers seeking accreditation without adding new requirements. Moreover, the publication in 2007 of ISO/IEC Guide 99 ‘International vocabulary of metrology—Basic and general concepts and associated terms (VIM)’ triggered additional adaptations of the guide.     

The lead assessor role in the ISO/IEC 17025:2005 accreditation of Brazilian calibration and testing laboratories by the General Coordination of Accreditation (Cgcre)

This work describes the human resources organization of the General Coordination for Accreditation (Cgcre) in Brazil, in which lead and technical assessors, both internal and external to the National Institute of Metrology, Quality and Technology (Inmetro), assess the competence of calibration and testing laboratories to the ISO/IEC 17025:2005 standard. The “Lead Assessor Project” is presented, which consists of an increase in the responsibilities of lead assessors, in such a way that the Division of Laboratories Accreditation (Dicla/Cgcre) can cope with the increasing demand for ISO/IEC 17025:2005 accreditation of Brazilian calibration and testing laboratories.     

Proficiency testing in food microbiology: experience from implementation of ISO/IEC 17043 and ISO/TS 22117

The proficiency testing (PT) scheme ??AQUA?? for food microbiology was organised by the Istituto Zooprofilattico Sperimentale delle Venezie (IZSVe) according to ISO/IEC 17043 and ISO/TS 22117. This paper describes the IZSVe experience on the application of the above-mentioned standards for the PTs, with focus on the Enterobacteriaceae enumeration one. Freeze-dried food matrices contaminated with American Type Culture Collection bacterial strains were used as test samples for each microbiological PT organised by IZSVe. The sample homogeneity and stability were verified prior to distribution to participants and throughout the PT, respectively. The participating laboratories analysed samples using their routine methods, and results were transmitted to IZSVe. Data and methods used by each participating laboratory were analysed in order to evaluate the laboratory performance. With reference to the Enterobacteriaceae PT, the test samples were homogeneous and stable. In addition, most laboratory results were obtained using equivalent test methods. Statistical approaches applied to analyse data generated from all participating laboratories revealed similar outcomes as no significant outlying count and only 5?% of unacceptable results were observed. Finally, the z-score, with the standard deviation that does not vary from round toround, was applied to compare and to evaluate the performance of each laboratory over time highlighting possible persistent trends over several rounds.     

Surveys on the accreditation of providers of proficiency testing and external quality assessment schemes

Two surveys among providers of proficiency testing (PT) and external quality assessment (EQA) schemes were carried out during 2004 and 2005. The main objectives were to explore the current status of accreditation/certification and collect the providers’ views. Information based on the response from 160 providers in 32 countries reveals a strong tendency towards accreditation of PT/EQA. It is shown that this type of accreditation is based on several combinations of normative documents, hence illustrating a lack of harmonisation of national accreditation bodies. The surveys also show that schemes are operated under considerably different conditions and that providers’ competence may or may not be underpinned by other certification and/or accreditation. This paper elaborates on a number of issues related to PT/EQA accreditation, including customers’ views, normative documents, providers’ experience from the accreditation process, views expressed by international organisations, and effects of accreditation on participation fees, quality and availability.     

Experiences from the assessment of proficiency testing providers

Following a pilot project in 2001–2002, UKAS accredited nine PT providers. During the course of the pilot and at subsequent surveillance visits, UKAS has gained experience in assessing organisations against the criteria detailed within ISO/IEC Guide 43-1:1997 and ILAC G13:2000. This article explores some of the key aspects that arose during the pilot and how UKAS has accommodated the addition of the accreditation of PT providers in to its range of services.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

Can single-operator laboratories comply with all of the requirements of ISO/IEC 17025:2005?

ISO/IEC 17025:2005 states that its requirements are “applicable to all laboratories regardless of the number of personnel” and would therefore include single-operator laboratories. However, there are reservations as to whether these laboratories can comply with all of the requirements without jeopardizing independence of judgement and impartiality. Similarly, there are some requirements of ISO/IEC 17025:2005 including staff supervision, internal communication processes and appointment of deputies that are considered unlikely to apply to a single-operator laboratory. The ISO/IEC 17025:2005 is widely used as the international standard of quality assurance by which accreditation bodies assess the competency of testing and calibration laboratories. There does, however, appear to exist, disagreement amongst accreditation experts when considering single-operator laboratories. Some accreditation bodies accredit single-operator laboratories, whilst others require additional human resources prior to granting accreditation. This discrepancy leads to unfair competition amongst laboratories as a single-operator laboratory by definition needs less resources (both human and financial) to achieve and maintain accreditation, compared with a laboratory where additional human resources need to be sought prior to and in order to maintain accreditation. The ISO/IEC 17025:2005 is in the process of being revised, and this is an opportune moment to address the issues aforementioned with the aim of removing ambiguity and enhancing clarity. In addition, the hope is to assist the accreditation bodies themselves to adopt a consensus approach when granting accreditation towards single-operator laboratories.     

Interlinkages and recognition of proficiency-testing schemes

 Because proficiency testing (PT) is increasingly used for the accreditation of testing laboratories and as a tool for backing up existing multilateral recognition arrangements between accreditation bodies, the question of performance and comparability of the proficiency-test providers arises. In this paper different approaches to assess the equivalence of European PT schemes and the competence of their providers are presented. As a first step a workshop is proposed to agree on a pilot study. The final aim is to create confidence in the existing PT schemes in Europe and to use them as common European tools.     

Preparation of in-house reference material of benzylpenicillin in milk and results of a Brazilian proficiency testing scheme

Two milk test materials containing benzylpenicillin and a benzylpenicillin-free material were prepared and used for the operation of a Brazilian proficiency testing (PT) scheme according to the requirements of ISO/IEC 17043. The PT scheme was designed to include laboratories in charge of confirmatory analysis as well as screening analysis. Sets of sample vials, benzylpenicillin-free and spiked items containing 0.2 g of lyophilized milk, were distributed to each participant. Of 20 participants who reported screening data, 18 accomplished satisfying results, providing evidence of the capability of Brazilian laboratories to produce reliable qualitative information. The assigned value (robust average calculated from 6 results) and its uncertainty were (8.28 ± 0.52) μg/L. Participants’ performance was evaluated using z-scores. A small number of participants were able to report quantitative benzylpenicillin results, and consequently care should be taken in the interpretation of the laboratory’s performance, considering the statistical nature of z-scores. Long-term stability testing on remaining PT items showed that a stable benzylpenicillin quality control material was produced and can be used for internal quality control or validation purposes.     

Revision of ISO guide 25

 ISO/IEC guide 25 is the internationally recognised base document for the accreditation of laboratories. Laboratory accreditation is a system of peer assessment and a formal recognition that a laboratory is competent to perform specific tests or types of tests ISO/IEC guide 25 plays a fundamental role in the life of the analytical chemist and is pivotal to the acceptance of the philosophy "once tested everywhere accepted" and to ensuring the mutual acceptance of test data. Within the EU, the attainment of accreditation to ISO/IEC guide 25 has become a way of life and it is now mandatory for laboratories engaged in certain regulatory work areas. Guide 25 is currently under revision and over the past 2 years or so it has been the subject of much debate among the calibration and testing community and it has engendered a considerable amount of written and oral comments. The latest revision entitled "Draft International Standard ISO/IEC DIS 17025: General Requirements for the Competence of Testing and Calibration Laboratories" was circulated to national standard organisations for their "comment and approval" in mid 1998. Voting on this document commenced on 9 July and terminates on 9 December 1998. It is anticipated that a final draft could be circulated in 1999. In accordance with the Vienna agreement this is a parallel ISO/CEN enquiry. This paper will discuss the implications of the technical requirements of the current document for analytical chemistry with particular emphasis on, the strengths, weaknesses and deficits inherent in the draft circulated in July 1998.