Experiences from the assessment of proficiency testing providers

Following a pilot project in 2001–2002, UKAS accredited nine PT providers. During the course of the pilot and at subsequent surveillance visits, UKAS has gained experience in assessing organisations against the criteria detailed within ISO/IEC Guide 43-1:1997 and ILAC G13:2000. This article explores some of the key aspects that arose during the pilot and how UKAS has accommodated the addition of the accreditation of PT providers in to its range of services.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

Harmonisation of proficiency testing schemes

The harmonisation of proficiency testing (PT) schemes has been under debate for a long time. There are obvious reasons why harmonisation of the practices in PT would be beneficial. In many areas, there is still a belief that further harmonisation of practices in PT would improve the comparability of measurement data. In particular when two laboratories are to be compared that have not participated in a single PT, problems arise which allegedly can be overcome by further harmonisation of PT schemes. In practice, however, parties involved in PT are not always embracing the idea of harmonisation. With the results of two European projects in mind, a discussion is given on harmonisation aspects, and some considerations are given that may help to decide in practice whether harmonisation is likely to solve particular problems. The first project, the European Proficiency Testing Network (EPTN), is concerned with further harmonisation. The second European project (COEPT) aims at providing a basis to assess equivalence across proficiency tests, and explores the conditions under which such an assessment is feasible.

     

Harmonisation of proficiency testing schemes

The harmonisation of proficiency testing (PT) schemes has been under debate for a long time. There are obvious reasons why harmonisation of the practices in PT would be beneficial. In many areas, there is still a belief that further harmonisation of practices in PT would improve the comparability of measurement data. In particular when two laboratories are to be compared that have not participated in a single PT, problems arise which allegedly can be overcome by further harmonisation of PT schemes. In practice, however, parties involved in PT are not always embracing the idea of harmonisation. With the results of two European projects in mind, a discussion is given on harmonisation aspects, and some considerations are given that may help to decide in practice whether harmonisation is likely to solve particular problems. The first project, the European Proficiency Testing Network (EPTN), is concerned with further harmonisation. The second European project (COEPT) aims at providing a basis to assess equivalence across proficiency tests, and explores the conditions under which such an assessment is feasible.     

Microbiology proficiency testing schemes in wine

Accreditation and Quality Assurance - The number of laboratories performing microbiological analyses of wines has gradually increased in recent years over the world. However, the lack of regular...     

Organisation of proficiency schemes by testing and calibration laboratories

Accreditation and Quality Assurance - Each year, an increasing demand for proficiency testing is seen in the world. Participating in this type of activity has been adopted by testing and...     

Harmonization and transferability of performance assessment: experience from four serum aluminum proficiency testing schemes

Proficiency testing schemes monitor laboratory performance and provide a stimulus for improvement in accuracy. Where several schemes operate in the same analytical sector, there are risks that assessments of performance may be in conflict. Performance assessment for the determination of trace elements such as aluminum in serum is particularly important due to the high risk of contamination and therefore erroneous results. The objectives of this work were (1) to compare several mathematical models to establish a predefined standard deviation for proficiency assessment and (2) to evaluate the influence of instrumental methods and proficiency testing scheme on the assessment of performance for serum aluminum measurements. For this purpose, three samples were sent to the participants of four proficiency testing schemes. Assigned values were calculated according to algorithm A according to ISO 13528 and standard deviation for proficiency assessment according to three methods based on individual variability, state of the art or previous proficiency testing results. The method based on individual variability produced a more stringent standard deviation compared to analytical imprecision based on the state of the art. The instrumental methods gave similar results, whereas significant differences were observed between the four proficiency testing schemes indicating that harmonization of the standard deviation for proficiency assessment fails to allow transferability from one proficiency testing scheme to another and that additional factor(s) contribute to variability in performance assessment.     

Information system and qualifying criteria for proficiency testing schemes

Since October 1998 the European Commission has financed a concerted action on Information System and Qualifying Criteria for Proficiency Testing Schemes within the 4th framework program. As a major result of this project EPTIS, the European Information System on Proficiency Testing Schemes which is available on the Internet since March 2000, is presented in this paper. Today EPTIS contains comprehensive information on approximately 640 proficiency testing schemes from 16 European countries providing information on the state of the art in proficiency testing in Europe. Finally some possible approaches for interlinkages and recognition of proficiency testing schemes are discussed.     

Probabilistic comparison and assessment of proficiency testing schemes and laboratories in the somatic cell count of raw milk

The somatic cell count (SCC) of milk is one of the main indicators of the udder health status of lactating mammals and is a hygiene criterion of raw milk used to manufacture dairy products. An increase in SCC is regarded as one of the primary indicators of inflammation of the mammary gland. Therefore, SCC is relevant in food legislation as well as in the payment of ex-farm raw milk and it has a major impact on farm management and breeding programs. Its determination is one of the most frequently performed analytical tests worldwide. Routine measurements of SCC are almost exclusively done using automated fluoro-opto-electronic counting. However, certified reference materials for SCC are lacking, and the microscopic reference method is not reliable because of serious inherent weaknesses. A reference system approach may help to largely overcome these deficiencies and help to assure equivalence in SCC worldwide. The approach is characterised as a positioning system fed by different types of information from various sources. A statistical approach for comparing proficiency tests (PTs) by assessing them using a quality index P _Q and assessing participating laboratories using a quality index P _L, both deriving from probabilities, is proposed. The basic assumption is that PT schemes are conducted according to recognised guidelines in order to compute performance characteristics, such as z-scores, repeatability and reproducibility standard deviations. Standard deviations are compared with the method validation data from the ISO method. Input quantities close to or smaller than the reference data of the method validation or the assigned value of the PT result in values for P _Q and P _L close to the maximum value. Evaluation examples of well-known PTs show the practicability of the proposed approach.     

Requirements vs acceptability in proficiency testing schemes and other interlaboratory comparisons

Many laboratories take part in proficiency testing schemes, external quality assessment programmes and other interlaboratory comparisons. These have many similarities but also important differences in their modus operandi and evaluation of performance of participating laboratories. This paper attempts to highlight both the similarities and differences. It also puts particular emphasis on requirements called ”target values for uncertainty” and their meaning. Received: 24 January 2001 Accepted: 25 January 2001     

Approaches to uncertainty evaluation based on proficiency testing schemes in chemical measurements

Since the advent of the Guide to the Expression of Uncertainty in Measurement (GUM) founding the principles of uncertainty evaluation, numerous projects have been carried out to develop alternative practical methods when it is impossible to model technical or economical aspects of the measurement process. These methods can use all the experimental data available to the laboratories, such as repeatability, reproducibility, quality-control charts, etc. The studies presented in this paper compare the results obtained by the modelling method from GUM with the uncertainties found by applying alternative methods. They show two examples, one in the field of environmental monitoring, the other in the biomedical field, based on the exploitation of PT schemes results. Presented at BERM-11, October 2007, Tsukuba, Japan.     

Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.     

Towards standardization of external quality assessment schemes by using bias values based on biological variation

The precision and trueness of current instruments and methods in clinical laboratories is much better than in the past. However, the z-score and other comparison variables that are currently used in external quality assessment programs are based on relative data. Thus, they may change from program to program and also change over time within the same program and consequently may not be useful or cost-effective. We therefore devised a test-specific decision limit for accepting or rejecting test results based on a combination of the data of within- and between-subject biological variations. We then applied these limits to a group of tests performed in our laboratory and compared our results with those of external quality assessment programs. In addition, we combined external and internal quality control data on the same graph and prepared a two-dimensional graph for different levels of control sera. Inspection of all results of control sera on this new graph was more useful for decision making. We concluded that the z-score is not reliable for comparisons of test results in external quality assessment. As a substitute for this currently accepted practice, we assert that control limits based on biological variation are more reliable, and can be useful in the evaluation of both external and internal quality assessments and their combination on this new graph.     

A statistical strategy for discouraging collusion in split-level proficiency testing schemes

A statistical strategy for discouraging collusion in split-level proficiency testing schemes was developed. When the samples are randomly labeled, the difference in the analyte levels in the two samples can be designed to maximize the probability of detecting collusion when it occurs, so that participants who colluded cannot easily receive satisfactory assessments. This favors providing convictive assessments of participants proficiency and giving the participants a level playing field.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

Benefits arising from proficiency testing schemes: The comparison of analytical methods

 The primary objective of proficiency testing (PT) is in the provision of information and support to participating laboratories, to enable them to monitor and improve the quality of their measurements. However, other benefits can be obtained from PT. These include the comparison of data for a given measurement by different methods, the validation of new methods, and the provision of information for laboratories' customers and accreditation bodies. This paper considers the subject of method comparison, and highlights some of the approaches which can be followed, as well as the practical use to which this can be put, to benefit the analytical community more widely. This is illustrated by a case study concerning the measurement of haze in beer. In this study the United Kingdom Institute of Brewing (IoB) conducted a survey of participants in the Brewing Analytes Proficiency Scheme (BAPS). From the survey data taken together with data from the BAPS scheme, the IoB is now in a position to give guidance on the use of particular instruments and procedures, as well as consider changes to the scope of the BAPS scheme to provide greater benefits for participants concerned with measuring haze. Received: 3 March 1998 · Accepted: 9 June 1998     

Policies, requirements and surveys concerning frequency for participation in proficiency testing schemes

This paper focuses on policies and requirements concerning the frequency for participation in proficiency testing (PT) and provides an overview of the frequency for which PT activities are offered in 25 testing fields. Recent international surveys are summarised.     

Policies,requirements and surveys concerning frequency for participation in proficiency testing schemes

This paper focuses on policies and requirements concerning the frequency for participation in proficiency testing (PT) and provides an overview of the frequency for which PT activities are offered in 25 testing fields. Recent international surveys are summarised.