Harmonisation of proficiency testing schemes

The harmonisation of proficiency testing (PT) schemes has been under debate for a long time. There are obvious reasons why harmonisation of the practices in PT would be beneficial. In many areas, there is still a belief that further harmonisation of practices in PT would improve the comparability of measurement data. In particular when two laboratories are to be compared that have not participated in a single PT, problems arise which allegedly can be overcome by further harmonisation of PT schemes. In practice, however, parties involved in PT are not always embracing the idea of harmonisation. With the results of two European projects in mind, a discussion is given on harmonisation aspects, and some considerations are given that may help to decide in practice whether harmonisation is likely to solve particular problems. The first project, the European Proficiency Testing Network (EPTN), is concerned with further harmonisation. The second European project (COEPT) aims at providing a basis to assess equivalence across proficiency tests, and explores the conditions under which such an assessment is feasible.     

Proficiency testing in occupational and environmental health – Current practice and international initiatives for harmonisation

An increasing number of proficiency testing schemes (PT schemes) related to occupational and environmental health have been organised. Most schemes emphasise the importance of the validation of analytical results. With regards to harmonisation of the schemes, there are many differences between the schemes at many levels. These include factors concerning their relationship with legislation, national status, type and quality of proficiency testing material, analytical range and priorities for future development. Since differences between PT schemes have been recognised at the European level by organisers of PT schemes and external quality assessment schemes (EQASs) it seems appropriate to reinforce collaboration between scheme organisers in order to improve the quality of analytical performance in occupational and environmental health.     

Surveys on the accreditation of providers of proficiency testing and external quality assessment schemes

Two surveys among providers of proficiency testing (PT) and external quality assessment (EQA) schemes were carried out during 2004 and 2005. The main objectives were to explore the current status of accreditation/certification and collect the providers’ views. Information based on the response from 160 providers in 32 countries reveals a strong tendency towards accreditation of PT/EQA. It is shown that this type of accreditation is based on several combinations of normative documents, hence illustrating a lack of harmonisation of national accreditation bodies. The surveys also show that schemes are operated under considerably different conditions and that providers’ competence may or may not be underpinned by other certification and/or accreditation. This paper elaborates on a number of issues related to PT/EQA accreditation, including customers’ views, normative documents, providers’ experience from the accreditation process, views expressed by international organisations, and effects of accreditation on participation fees, quality and availability.     

Interlinkages and recognition of proficiency-testing schemes

 Because proficiency testing (PT) is increasingly used for the accreditation of testing laboratories and as a tool for backing up existing multilateral recognition arrangements between accreditation bodies, the question of performance and comparability of the proficiency-test providers arises. In this paper different approaches to assess the equivalence of European PT schemes and the competence of their providers are presented. As a first step a workshop is proposed to agree on a pilot study. The final aim is to create confidence in the existing PT schemes in Europe and to use them as common European tools.     

Comparability of PT schemes – what did we learn from COEPT?

The use of proficiency testing schemes (PTS) by laboratories as an integral part of their quality system has been increasing in recent years. Accreditation bodies, regulators and the laboratories’ customers are increasingly using results from PTS in their relationship with laboratories. There are many PTS available in Europe in analytical chemistry; EPTIS indicates over 400.The comparability of these PTS is now a real issue, as many organisers of PTS move into new markets. The COEPT project has systematically demonstrated (in four technical sectors – water, soil, food and occupational hygiene), that there are many similarities between PTS in each sector. For example, nearly all use the z-score as a performance index. One significant difference between many PTS is the value used for the term s in the z-score equation, and this gives a range of evaluations for the same data point. Despite this, the agreement between PTS in the same sector for the evaluation of data is approximately 85%. COEPT has given us a basis for establishing the comparability of PTS and showing us where further harmonisation could occur.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia.     

The correlation of laboratory performance in proficiency testing with other QA characteristics

Although it seems self-evident that proficiency testing (PT) and accreditation can be expected to improve quality, their relative benefits remain uncertain as does their efficacy. The study reported here examines the following issues: (a) Why do laboratories take part in PT schemes? (b) How does participation in PT fit in with a laboratory's overall quality assurance (QA) system? (c) Is there a link between a laboratory's performance in specific PT and it's QA system? (d) How does PT performance change with time and how do laboratories respond to poor performance? The overall conclusion is that there is no evidence from the present study that laboratories with third-party assessment (accreditation and certification) perform any better in PT than laboratories without. The validity of this conclusion and its significance for the future design and operation of such schemes requires further investigation. In particular, study is required of the degree to which good performance in open PT correlates with blind PT performance, where laboratories are not aware that the samples being analysed are part of a quality assessment exercise.     

Information system and qualifying criteria for proficiency testing schemes

Since October 1998 the European Commission has financed a concerted action on Information System and Qualifying Criteria for Proficiency Testing Schemes within the 4th framework program. As a major result of this project EPTIS, the European Information System on Proficiency Testing Schemes which is available on the Internet since March 2000, is presented in this paper. Today EPTIS contains comprehensive information on approximately 640 proficiency testing schemes from 16 European countries providing information on the state of the art in proficiency testing in Europe. Finally some possible approaches for interlinkages and recognition of proficiency testing schemes are discussed.     

The challenges and benefits of implementing the requirements of ISO/IEC 17043 by PT/EQA providers

ISO/IEC 17043 Conformity assessment??general requirements for proficiency testing is intended to replace previous international guides used to assess the competence of proficiency testing (PT) providers. It expands on the requirements of previous guides and is intended to accommodate PT providers of calibration laboratories and testing laboratories handling both qualitative and quantitative data. QMP-LS is an office-based external quality assessment provider for medical laboratories in Ontario, Canada and operates 46 different PT schemes for approximately 250 diagnostic tests. In 2010, these schemes were accredited to ISO/IEC 17043. Schemes included tests from the following disciplines: chemistry, hematology, microbiology, transfusion medicine, cytology, histology and genetics. Thirty of the schemes were qualitative. The challenges and benefits of implementing ISO/IEC 17043 are discussed, with particular emphasis on clauses addressing the following requirements: statistical design, determination of assigned value, homogeneity and stability testing, packaging, labeling and distribution, performance evaluation and subcontracting services.     

PT-WFD: the network of PT providers to support the implementation of the European water framework directive

This paper introduces a newly formed network of proficiency testing (PT) providers, who are active and experienced in the field of PTs with relevance to the water framework directive (WFD). The objective of this new network is to support the implementation of the WFD in Europe, by providing PT schemes that meet the specific requirements of the WFD and which are run and evaluated in a harmonised way. The main value of creating such a network lies in the potential to foster the harmonised performance evaluation of European laboratories involved in WFD monitoring analyses, thus enhancing the equivalence of WFD monitoring data obtained throughout Europe. In its first year the network organised two joint PT rounds, one for volatile organic substances (VOC) and another for selected pesticides from the list of Priority Substances, with greater than 100 water analysis labs from a number of European countries participating.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working group discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the joint EURACHEM/CITAC/EQALM workshop, Bracknell, UK, 16–18 February 2003 is provided. The nine WGs covered a range of issues concerned with current practice and future directions; PT/EQA as a tool for regulators (WG1); PT/EQA as a tool for accreditation (WG2); evaluation of performance and uncertainty (WG3); frequency of PT/EQA participation (WG4); selection of appropriate PT/EQA schemes (WG5); added value of PT/EQA and cost benefit evaluation (WG6); global harmonisation and rationalisation (WG7); new technical areas and challenges in PT/EQA (WG8); and accreditation of PT/EQA providers (WG9). Participants with different backgrounds were on each WG in order to capture a range of views and experience from different sectors. The discussions reflected on the keynote lectures and built, in many cases, on discussions at previous workshops in 2000 and 2002.     

Causes of error in analytical chemistry: results of a web-based survey of proficiency testing participants

Results of a voluntary-response survey of respondent-identified causes of unacceptable results in nine proficiency testing schemes are reported. The PT schemes were predominantly environment and food analysis schemes.?111 respondents reported 230 identified causes of error. Sample preparation (16?% of causes reported), Equipment failures (13?%), ??Human error?? (13?%) and Calibration (10?%) were the top four general causes of poor analytical results. Among sample preparation errors, sample extraction or recovery problems were the most important causes reported. Most calibration errors were related to errors in calculation and dilution and not in availability or quality of calibration materials. No failures were attributed to failures in commercial software; software-related problems were largely associated with user input errors. Corrective actions were generally specific to the particular problem identified. Review of all reported causes indicated that about 44?% could be attributed to simple operator errors.     

Harmonisation Of Ec Air Quality Directives

Abstract

The Environment Institute of the Joint Research Centre in Ispra devotes an important part of its activities to support the Commission of the European Communities in scientific and technical matters. As an example of these activities, a general overview is given of the harmonisation work lead by its Central Laboratory of Air Pollution for the implementation of EC air quality directives in the European Member States. The different stages of a directive's development are reviewed and illustrated with typical examples of harmonisation work recently undertaken.

Preparatory actions for future regulations are illustrated by the recent intercomparison exercise for VOC measurements, realized in view of the coming directive on photo-oxidants.

Quality assurance programmes are developed that are mainly focused on the evaluation of sampling, calibration and measurement techniques in the Member States. Special attention is also paid to the harmonisation of network design. The results of a recent network design campaign realised in Madrid, using passive sampling techniques in combination with mobile measurements, are presented.

An example of the development of scientific and technical progress is given by the intercomparison of primary NO₂ calibration standards, where the reference method of the directive was checked and compared to new standard methods.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the Eurachem Workshop, Rome, 5–7 October 2008 is provided. The eight WG’s covered a range of issues concerned with current practice and future directions; how frequently should laboratories participate in PT/EQA? (WG1); developments in PT/EQA within the EU—what is required in future? (WG2); what issues do developing countries face with regards to PT/EQA? (WG3), what issues are specific to microbiology PT/EQA? (WG4); what new fields are emerging for PT/EQA? (WG5); what will be the impact of the new ISO/IEC 17043 standard? (WG6); do current PT/EQA schemes meet the needs of participants? (WG7); and what are the issues that affect the quality of proficiency test items? (WG8). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Proficiency testing in analytical chemistry, microbiology, and laboratory medicine – working group discussions on current status, problems, and future directions

Working group (WG) discussions on proficiency testing (PT) held at the joint Eurachem/ EQALM workshop, Borås, Sweden, 24–26 September 2000 are summarized. The discussions focused on aspects of PT and accreditation (WG 1), general aspects of PT in analytical chemistry (WG 2), microbiology (WG 3), and laboratory medicine (WG 4), incorporation of measurement uncertainty into PT schemes (WG 5), international harmonization of PT schemes (WG 6), and the role of PT in the international structure of chemical measurement (WG 7). Current status, problems and future directions are identified. Each WG contained a majority of participants experienced in the subject being covered by that WG, and a few participants with different expertise. This was done to promote cross-fertilization of ideas between sectors, a key objective of the workshop. The WG issues reflected the content of the keynote lectures and some issues were covered from different perspectives by more than one group.     

Estimates of uncertainty of measurement from proficiency testing data: a case study

The results obtained by a laboratory over a number of proficiency testing/external quality assessment schemes (PT/EQAS) rounds can give information on the uncertainty of its measurements for a given test, provided that conditions such as full coverage of the routine analytical range, traceability, and small uncertainty of the assigned values (compared to the spread of the results) are met and provided that systematic deviations and any other sources of uncertainty are considered. As organisers of the Italian EQAS (ITEQAS) in occupational and environmental laboratory medicine, we tested this hypothesis using as model data from well-performing laboratories taking part in ITEQAS for lead in blood over the last 2 years. We also investigated how different PT/EQAS features (frequency of trials and number of samples) would affect a laboratory estimate of its uncertainty. Such information can be helpful in improving PT/EQAS organisation and define, for a given test: (a) the state of the art of the uncertainty of current measurement procedures, (b) identify needs for improvement of analytical methodologies and (c) set targets for acceptable uncertainty values.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia.Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher.     

Proficiency testing schemes for therapeutics and toxicology

 This report addresses the proposition: current proficiency testing (PT) schemes for therapeutics and toxicology can easily engender complacency. The proposition was based on the premise that there are substantial between-laboratory differences and objectives in the supply of services for the measurement of drugs, as a result of which it is difficult to design PT schemes which adequately cover the requirements of all laboratories. Thus, acceptable performance in a PT scheme may lead participants to consider that they have tested their procedures more rigorously than is the case. Four areas, validity of the PT scheme, PT material, frequency of testing and acceptance criteria are examined. Received: 15 April 2000 · Accepted: 15 April 2000     

Challenges in preparing water-matrix reference materials for PAHs and pesticides: examples from SWIFT-WFD proficiency-testing schemes

In accordance with Article 8 of the European Union (EU) Water Framework Directive (WFD), EU Member States are required to establish monitoring programs for the quality of the surface water and the groundwater within each river-basin district. As such data are the basis for regulatory decisions and measures required to achieve WFD environmental objectives, appropriate analytical quality-assurance and quality-control tools have to be implemented by the monitoring laboratories. In this respect, reference materials (RMs) play a key role. Within the framework of the SWIFT-WFD project (Screening methods for Water data InFormation in support of the WFD), several approaches to the preparation of matrix RMs for the analysis of polycyclic aromatic hydrocarbons and pesticides in water have been used in wide-scale proficiency-testing (PT) schemes. We present the different strategies employed in preparing water-matrix RMs for organic analytes. By reviewing the results from the SWIFT-WFD PT schemes, we reflect on the applicability and the suitability of the different approaches.     

A proficiency testing scheme to evaluate the effectiveness of laboratory sample reduction of a soil sample

Accreditation and Quality Assurance - Although proficiency testing (PT) schemes in analytical chemistry are intended to test the entire analytical process, the reality is that some aspects of the...     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the EURACHEM Workshop, Istanbul, 3?C6 October 2011, is provided. The six WGs covered a range of issues concerned with current practice and future directions; implementing the requirements of ISO/IEC 17043 by PT/EQA providers (WG1); accrediting PT/EQA providers to ISO/IEC 17043 (WG2); pre- and post-analytical aspects in PT/EQA (WG3); evaluating participant performance in qualitative PT/EQA schemes (WG4); establishing PT/EQA schemes in developing countries (WG5); and establishing acceptability criteria in microbiology PT/EQA schemes (WG6). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end-users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.