Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the EURACHEM Workshop, Istanbul, 3?C6 October 2011, is provided. The six WGs covered a range of issues concerned with current practice and future directions; implementing the requirements of ISO/IEC 17043 by PT/EQA providers (WG1); accrediting PT/EQA providers to ISO/IEC 17043 (WG2); pre- and post-analytical aspects in PT/EQA (WG3); evaluating participant performance in qualitative PT/EQA schemes (WG4); establishing PT/EQA schemes in developing countries (WG5); and establishing acceptability criteria in microbiology PT/EQA schemes (WG6). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end-users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the Eurachem Workshop, Rome, 5–7 October 2008 is provided. The eight WG’s covered a range of issues concerned with current practice and future directions; how frequently should laboratories participate in PT/EQA? (WG1); developments in PT/EQA within the EU—what is required in future? (WG2); what issues do developing countries face with regards to PT/EQA? (WG3), what issues are specific to microbiology PT/EQA? (WG4); what new fields are emerging for PT/EQA? (WG5); what will be the impact of the new ISO/IEC 17043 standard? (WG6); do current PT/EQA schemes meet the needs of participants? (WG7); and what are the issues that affect the quality of proficiency test items? (WG8). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working group discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the joint EURACHEM/CITAC/EQALM workshop, Bracknell, UK, 16–18 February 2003 is provided. The nine WGs covered a range of issues concerned with current practice and future directions; PT/EQA as a tool for regulators (WG1); PT/EQA as a tool for accreditation (WG2); evaluation of performance and uncertainty (WG3); frequency of PT/EQA participation (WG4); selection of appropriate PT/EQA schemes (WG5); added value of PT/EQA and cost benefit evaluation (WG6); global harmonisation and rationalisation (WG7); new technical areas and challenges in PT/EQA (WG8); and accreditation of PT/EQA providers (WG9). Participants with different backgrounds were on each WG in order to capture a range of views and experience from different sectors. The discussions reflected on the keynote lectures and built, in many cases, on discussions at previous workshops in 2000 and 2002.     

Accreditation of PT providers – is it worth the money?

This topic is being presented from two perspectives, namely the views of an accreditation body and those of an accredited provider of external quality assessment schemes. The first perspective, from an accreditation body, is based on the experience of Australia's national laboratory accreditation body (NATA), which has accredited fourteen proficiency testing (PT) providers and is currently processing an application from another. The second perspective is based on the experience of EQUALIS, the national quality assurance organisation for laboratory medicine in Sweden, which was granted accreditation in 2002.Presented at 5^th Eurachem Workshop on Proficiency testing in analytical chemistry, microbiology and laboratory medicine, Portorož (Slovenia) 25–27 Sept 2005     

Do we need to accredit proficiency testing schemes?

 Proficiency testing (PT) is being increasingly used as an important quality assurance tool for laboratories. The subject of quality of the providers of PT schemes has been discussed increasingly in recent years. Some countries have implemented systems for the accreditation of PT schemes. This paper looks at the background to the accreditation of PT schemes, the likely mechanisms which could be employed for accreditation, and some of the practical aspects.     

The correlation of laboratory performance in proficiency testing with other QA characteristics

Although it seems self-evident that proficiency testing (PT) and accreditation can be expected to improve quality, their relative benefits remain uncertain as does their efficacy. The study reported here examines the following issues: (a) Why do laboratories take part in PT schemes? (b) How does participation in PT fit in with a laboratory's overall quality assurance (QA) system? (c) Is there a link between a laboratory's performance in specific PT and it's QA system? (d) How does PT performance change with time and how do laboratories respond to poor performance? The overall conclusion is that there is no evidence from the present study that laboratories with third-party assessment (accreditation and certification) perform any better in PT than laboratories without. The validity of this conclusion and its significance for the future design and operation of such schemes requires further investigation. In particular, study is required of the degree to which good performance in open PT correlates with blind PT performance, where laboratories are not aware that the samples being analysed are part of a quality assessment exercise.     

Interlinkages and recognition of proficiency-testing schemes

 Because proficiency testing (PT) is increasingly used for the accreditation of testing laboratories and as a tool for backing up existing multilateral recognition arrangements between accreditation bodies, the question of performance and comparability of the proficiency-test providers arises. In this paper different approaches to assess the equivalence of European PT schemes and the competence of their providers are presented. As a first step a workshop is proposed to agree on a pilot study. The final aim is to create confidence in the existing PT schemes in Europe and to use them as common European tools.     

Performance evaluation for proficiency testing with a limited number of participants

Proficiency testing (PT) is an essential tool for laboratories to assess their competency. Also, participation in PT has become one of the mandatory requirements for laboratory to seek accreditation according to ISO/IEC 17025. For this reason, the effectiveness of performance evaluation by PT scheme is of great concern for the participants and for accreditation bodies as well. In practice, owing to unavailability of other appropriate alternatives, PT scheme providers may have to choose using consensus values to evaluate the performance of participants. However, such consensus values approach was not recommended by relevant international guidelines for PT schemes with limited number of participants. With the use of Monte Carlo simulation technique, this study attempted to investigate the effectiveness of using consensus values for performance evaluation in PT schemes with limited number of participants. The simulation process was schemed according to the statistical model provided by ISO 5725-1 for laboratory measurement results, which covered components like method bias, laboratory bias, and measurement precision. The effectiveness of the consensus value approach was expressed as the percentage of participants in a simulation run could get the same evaluation result, either satisfactory or unsatisfactory, against the “true value.” The findings indicated that the number of participants, choice of consensus values, mass fraction of analyte, method bias, laboratory bias, and measurement repeatability of participating laboratories would all affect the effectiveness of the consensus value approach but at different extent. However, under certain circumstances, use of consensus value could still be considered as an acceptable approach for performance evaluation even the number of participants was limited. Some of the findings were further verified using real data from PT schemes where appropriate certified reference materials or reliable reference values were available.     

Twenty-year review of laboratory performance on enteric parasitology external quality assessment surveys 1992–2011, Ontario Canada

Quality Management Program?CLaboratory Services (QMP?CLS) is a mandatory external quality assessment (EQA) and accreditation programme for laboratories in Ontario. This report summarizes performance of Ontario laboratories on QMP?CLS enteric parasitology EQA surveys and compares this performance over time. The results reported by participating laboratories were assessed against the expected results as determined by reference or referee laboratories. The number of laboratories licensed for parasitology decreased dramatically over the 20?years. On average, more than 80?% of participants and often more than 90?% were able to detect and identify the species and stages for the majority of parasites. Many of the challenges contained multiple parasites and about half contained low numbers, which affected performance on individual samples. The level of performance by Ontario laboratories on parasitology EQA surveys was consistently high over the 20?years. The parasites that presented the most difficulty were Enteromonas hominis and Trichomonas hominis.     

National External Quality Assessment Schemes for microbiology, parasitology, and virology in Europe

Quality assurance is an important aspect of laboratory management. One of the activities involved is the participation in external quality assessment (EQA) schemes by the clinical laboratory. These EQA schemes should be organised according to well-defined guidelines, such as the ESO/IEC Guide 43. The present work provides an inventory of the EQA schemes organised in Europe. The schemes are related to the fields of bacteriology, parasitology and virology. For each field various surveys have been organised. Data such as the number of participants, frequency of survey and number of samples in a survey are presented. The nature and way of manufacturing the control material is specified.     

Accrediting PT/EQA providers to ISO/IEC 17043

After many years and having several different attempts for the accreditation of proficiency testing provider (PT provider), there is finally one stand-alone standard defining the requirements for the competence of PT providers and therefore an internationally harmonised basis for the accreditation of proficiency test providers. Since February 2010, the ISO/IEC 17043:2010 has replaced ISO/IEC Guide 43:1997 and ILAC G 13:2007. The philosophy of the standard about subcontracting work is different to this of the standards mostly used for accreditation like ISO/IEC 17025:2005 or ISO/IEC 17020:2004, etc. Besides the planning of the proficiency tests (PT), the performance evaluation and the authorisation of the PT reports the ISO/IEC 17043:2010 allows subcontracting for the rest of the work when providing PTs. This is a challenge for the assessors to judge about the competence of a PT provider. In numerous paragraphs, the standard sets very detailed requirements. Nevertheless, there is room for interpretation. For these cases, for example, contracts for subcontractors, procedure for the advisory board, minimum requirements for PT certificates, etc., some proposals are given to enable harmonised approach for the assessment of PT providers.     

Accreditation of calibration laboratories to normative documents: diversity or standardization

Calibration of measuring equipment is conducted by following some normative or applicable documents such as standards, manufacturer manuals and instructions, technical orders issued by defense organizations, or scientific papers. An accreditation body provides its recognition to the calibration laboratories by evaluating their technical competence and their compliance with the quality requirements of ISO/IEC 17025. The accreditation body must have defined criteria in order to evaluate different calibration methods which should ensure that the laboratories are performing the calibration in a technically competent manner when they are fully or even only partially based on the relevant reference documents. A discussion with different points of view about choosing the criteria, as well as the Israel Laboratory Accreditation Authority (ISRAC) policy on this issue, are presented.Presented at the 2nd International Conference on Metrology – Trends and Applications in Calibration and Testing Laboratories, November 4–6, 2003, Eilat, Israel.     

Evaluating participant performance in qualitative PT/EQA schemes

A basic principle that needs to be satisfied before a proficiency testing (PT)/external quality assessment (EQA) scheme is first introduced is that measurement and assessment of performance is possible. For qualitative analyses, data can be presented as ??percentage correct??, and these data can be aggregated to show trends and to facilitate monitoring performance over time. Alternatively data can be analysed to identify significant patterns or changes in practice, or to compare different categories. However, the most useful tool is the application of a numerical score to the results. A number of different scoring strategies can be used, varying in complexity depending on the nature and significance of the type of results generated. Examples from selected microbiological EQA schemes are presented.     

The use of multiple correspondence analysis for the evaluation of inter-laboratory comparisons

Participation in inter-laboratory comparisons (ILC) is one of the recommended means of external quality control according to ISO/IEC 17025:2005. Providers of ILC or proficiency test (PT) schemes collect, besides the measurement results on the test samples, information on the sample treatment and measurement procedure. The objective of this paper is to evaluate in a non-traditional way, using numerical and non-numerical data provided by the participants in IMEP-20 (trace elements in tuna fish), the additional information concerning the applied analytical methods and the accreditation/nomination status. Arsenic was taken as an example. The basic statistical procedure for the evaluation of questionnaire information was the multiple correspondence analysis (MCA). Univariate clustering techniques were applied for the categorization of the numerical data (measurement values). The methodology of the evaluation of supplementary non-numeric information used in this paper might serve (a) to providers of ILC (PT) schemes to modify/improve their questionnaires and (b) to give laboratories better guidance in the methods used for the determination of various analytes in different matrices. This paper is meant serve as a guide for the possible interpretation of the questionnaires accompanying ILC schemas. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel     

Quality management, certification and accreditation in medical laboratories – the view of ECLM

 Increasing demands from health care planners and industrialists conducting clinical trials, as well as general competition, are forcing medical laboratories to seek third-party recognition of their quality management systems. There is a tendency to move from certification of a laboratory director, via certification of the laboratory quality system (ISO 9000 family), to accreditation needing proof of professional and technical competence in laboratory tasks. The requirements of accreditation are presented in several national schemes and in the European Standards series (EN 45 000) and the International Organization for Standardization's guide, ISO/IEC 25, to be amalgamated soon. The latter system provides transnational recognition through participation of the accrediting bodies in the European co-operation for Accreditation. Necessary supplementary guidelines exist for chemical laboratories (Eurachem) and medical laboratories CEAC/ECLM). Traceability and reliability of results are obtained by utilizing a global reference examination system and by participating in transdisciplinary work. The costs of achieving accreditation are considerable and mainly involve the production of quality handbooks and written work procedures by personnel. The rewards are an open system, smoother work, emphasis on prevention of mistakes, and satisfied stakeholders. Received: 5 October 1998 · Accepted: 20 October 1998     

铜冶炼物料中铜的分析质量控制方法综述

目前国内铜精矿供应紧俏,铜冶炼行业的原料复杂,有些偏离了认证和认可标准方法的样品,在检测时,用认证、认可文件不能很好地进行质量管控,需要对分析方法加以改进,从技术上进行管控。结合国内铜冶炼企业的实际情况和多年的实践经验,从样品的加工和保存环节、铜分析的技术环节、人员培训、质量抽查、定期开展质量分析会等方面对铜的分析质量控制进行了总结。在具体的质量控制中,针对铜冶炼行业成分复杂、易变化的物料分析,应根据实际情况进行质量管控,以提高铜分析的准确度。     

Use of participant EQA results to assess sample homogeneity and stability for qualitative analytes

International requirements for PT and EQA state that providers must demonstrate that samples are homogeneous and stable. However, testing for homogeneity and stability can be expensive, use samples that could otherwise serve as quality-control materials, and can also fail to detect significant inhomogeneity and instability. In some situations it may be preferable to use the results from participants to identify problems with sample lots, if historic results follow predictable distributions and a statistical decision rule can be produced. An unusually high proportion of incorrect results may indicate that samples were inhomogeneous or unstable. Conditions under which this can be demonstrated are discussed, and the efficacy of the proposal is demonstrated with two examples. This procedure is especially effective when there are a large number of participants and/or a historic small proportion of incorrect results. Providers who adopt this proposal will need to retain samples for testing and assume the risk of distributing bad samples.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

The challenges and benefits of implementing the requirements of ISO/IEC 17043 by PT/EQA providers

ISO/IEC 17043 Conformity assessment??general requirements for proficiency testing is intended to replace previous international guides used to assess the competence of proficiency testing (PT) providers. It expands on the requirements of previous guides and is intended to accommodate PT providers of calibration laboratories and testing laboratories handling both qualitative and quantitative data. QMP-LS is an office-based external quality assessment provider for medical laboratories in Ontario, Canada and operates 46 different PT schemes for approximately 250 diagnostic tests. In 2010, these schemes were accredited to ISO/IEC 17043. Schemes included tests from the following disciplines: chemistry, hematology, microbiology, transfusion medicine, cytology, histology and genetics. Thirty of the schemes were qualitative. The challenges and benefits of implementing ISO/IEC 17043 are discussed, with particular emphasis on clauses addressing the following requirements: statistical design, determination of assigned value, homogeneity and stability testing, packaging, labeling and distribution, performance evaluation and subcontracting services.     

Uses and abuses of PT

Proficiency testing (PT) is a very valuable quality tool for laboratories making all kinds of analytical measurements, in chemistry, microbiology and laboratory medicine. Compared with other quality tools, which are largely concerned with INPUTS (e.g. staff training, calibration, documentation), PT is concerned with OUTPUTS (i.e. the actual results produced by the laboratory). The quality of routine measurements and results, which would normally be reported to customers, both internal and external, should be demonstrated. This is a major reason for many laboratories participating in PT, even if they are not accredited, or use very few, if any, other quality tools. This is important, for it enables the laboratory manager to measure the efficacy of the overall quality system, and to determine if the inputs to this are appropriate. This should be done in a spirit of learning—it was discussed at the previous workshop whether PT schemes are teachers or policemen, and the consensus was that MOST are teachers (or at least try to be). Learning from mistakes, investigating problems, implementing corrective actions and then observing whether thisis successful in the next PT represents what many in our profession would regard as a proper and effective use of PT. Although this may be the intended primary use of PT by laboratories, we must not forget that PT has other valid uses. In particular, the use of PT to demonstrate competence to third parties—accreditation bodies, regulators, and customers—has become more important in the last few years. However, this development, however valuable, can lead to abuse of PT scheme data. This paper will highlight briefly some examples of both proper use and abuse of PT. These are taken from my experience with PT schemes over many years.     

Measurement traceability and US IVD manufacturers: the impact of metrology

A goal of clinical laboratory science is to produce accurate and comparable patient test results for a specimen independent of analytical methodology. The In Vitro Diagnostics Directive in Europe has provided the impetus for the U.S. in vitro diagnostic (IVD) industry to adapt the concepts of Metrology, the science of measurement, including measurement traceability and measurement uncertainty. The joint committee for traceability in laboratory medicine has provided a valuable database of internationally recognized reference materials and methods and reference laboratories. Much of the responsibility for measurement traceability falls on IVD manufacturers, but all global stakeholders, including the clinical laboratory profession, government Regulatory bodies, metrology institutes, and the providers of EQA/PT surveys, must cooperate to progress toward this goal. The adaptation of the concepts of Metrology to the clinical laboratory is an appropriate and logical development and it will continue in the twenty-first century.