Quality issues in proficiency testing

 Proficiency testing is a means of assessing the ability of laboratories to competently perform specific tests and/or measurements. It supplements a laboratory's own internal quality control procedure by providing an additional external audit of their testing capability and provides laboratories with a sound basis for continuous improvement. It is also a means towards achieving comparability of measurement between laboratories. Participation is one of the few ways in which a laboratory can compare its performance with that of other laboratories. Good performance in proficiency testing schemes provides independent evidence and hence reassurance to the laboratory and its clients that its procedures, test methods and other laboratory operations are under control. For test results to have any credibility, they must be traceable to a standard of measurement, preferably in terms of SI units, and must be accompanied by a statement of uncertainty. Analytical chemists are coming to realise that this is just as true in their field as it is for physical measurements, and applies equally to proficiency testing results and laboratory test reports. Recent approaches toward ensuring the quality and comparability of proficiency testing schemes and the means of evaluating proficiency test results are described. These have led to the drafting of guidelines and subsequently to the development of international requirements for the competence of scheme providers. Received: 2 January 1999 · Accepted: 7 April 1999     

Benefits arising from proficiency testing schemes: The comparison of analytical methods

 The primary objective of proficiency testing (PT) is in the provision of information and support to participating laboratories, to enable them to monitor and improve the quality of their measurements. However, other benefits can be obtained from PT. These include the comparison of data for a given measurement by different methods, the validation of new methods, and the provision of information for laboratories' customers and accreditation bodies. This paper considers the subject of method comparison, and highlights some of the approaches which can be followed, as well as the practical use to which this can be put, to benefit the analytical community more widely. This is illustrated by a case study concerning the measurement of haze in beer. In this study the United Kingdom Institute of Brewing (IoB) conducted a survey of participants in the Brewing Analytes Proficiency Scheme (BAPS). From the survey data taken together with data from the BAPS scheme, the IoB is now in a position to give guidance on the use of particular instruments and procedures, as well as consider changes to the scope of the BAPS scheme to provide greater benefits for participants concerned with measuring haze. Received: 3 March 1998 · Accepted: 9 June 1998     

Microbiology proficiency testing schemes in wine

Accreditation and Quality Assurance - The number of laboratories performing microbiological analyses of wines has gradually increased in recent years over the world. However, the lack of regular...     

Harmonisation of proficiency testing schemes

The harmonisation of proficiency testing (PT) schemes has been under debate for a long time. There are obvious reasons why harmonisation of the practices in PT would be beneficial. In many areas, there is still a belief that further harmonisation of practices in PT would improve the comparability of measurement data. In particular when two laboratories are to be compared that have not participated in a single PT, problems arise which allegedly can be overcome by further harmonisation of PT schemes. In practice, however, parties involved in PT are not always embracing the idea of harmonisation. With the results of two European projects in mind, a discussion is given on harmonisation aspects, and some considerations are given that may help to decide in practice whether harmonisation is likely to solve particular problems. The first project, the European Proficiency Testing Network (EPTN), is concerned with further harmonisation. The second European project (COEPT) aims at providing a basis to assess equivalence across proficiency tests, and explores the conditions under which such an assessment is feasible.     

Harmonisation of proficiency testing schemes

The harmonisation of proficiency testing (PT) schemes has been under debate for a long time. There are obvious reasons why harmonisation of the practices in PT would be beneficial. In many areas, there is still a belief that further harmonisation of practices in PT would improve the comparability of measurement data. In particular when two laboratories are to be compared that have not participated in a single PT, problems arise which allegedly can be overcome by further harmonisation of PT schemes. In practice, however, parties involved in PT are not always embracing the idea of harmonisation. With the results of two European projects in mind, a discussion is given on harmonisation aspects, and some considerations are given that may help to decide in practice whether harmonisation is likely to solve particular problems. The first project, the European Proficiency Testing Network (EPTN), is concerned with further harmonisation. The second European project (COEPT) aims at providing a basis to assess equivalence across proficiency tests, and explores the conditions under which such an assessment is feasible.

     

A comparison of performance statistics for proficiency testing programmes

Recently, ISO issued a new document on statistical methods for use in proficiency-testing schemes. The document presented a number of new performance statistics that involve the use of measurement uncertainties of the participants’ results and/or the assigned values for the performance evaluation. Making use of the Monte Carlo simulation technique, this paper attempts to compare selected performance statistics of different approaches. In addition, the simulation programme was extended to study the effect of multi-modality in participants’ results on the performance evaluation when different performance statistics were applied.     

The practice of proficiency testing in China

This paper briefly summarises the current situation for proficiency testing (PT) in China, outlines the policy for PT of China’s national accreditation committee for laboratories (CNACL), and exemplifies activities of the CNACL’s metal working group. Received: 9 December 2000 Accepted: 14 December 2000     

The proof of analytical chemical education lies in proficiency testing: A proficiency testing programme in polytechnic academia and vocational colleges in The Netherlands

Participation in proficiency testing schemes (PT-schemes) is an important way to prove and to improve the skills of analytical chemical students. It gives valuable information on the professional capabilities of the students and is thus an important step in the training of laboratory engineers and technicians. In the programme described three types of samples are offered to the students: chemical, environmental and clinical samples. A guiding protocol presents instructions to the participating student groups. Standard operating procedures are available, but the students are also free to select their own method. The results are methodologically and statistically judged by professional PT-organisers and are presented in the customary way: the dataset, the mean (consensus) value of all participants and the Z-scores. The final presentation of the results takes place in an annual symposium where awards are presented to the best performing student groups, and a lecture programme is presented with subjects in the area of quality assurance and quality control. A selection of results from the KILO/PT/1998–1999 and the KILO/PT/1999–2000 programme is presented.     

Statistical treatment of proficiency testing data

 There are three stages to evaluating a laboratory's results in an interlaboratory proficiency test: establishing the correct result for the test item, determining an evaluation statistic for the particular result, and establishing an acceptable range. There are a wide variety of procedures for accomplishing these three stages and a correspondingly wide variety of statistical techniques in use. Currently in North America the largest number of laboratory proficiency test programs are in the clinical laboratory field, followed by programs for environmental laboratories that test drinking water and waste water. Proficiency testing in both of these fields is under the jurisdiction of the federal government and other regulatory and accreditation agencies. Many of the statistical procedures are specified in the regulations, to assure comparability of different programs and a fair evaluation of performance. In this article statistical procedures recommended in International Organization for Standardization Guide 43, Part 1, are discussed and compared with current practices in North America. Received: 22 April 1998 · Accepted: 12 May 1998     

Detection of genetic modification in cured tobacco leaf: proficiency testing using polymerase chain reaction-based methods

The Cooperation Centre for Scientific Research Relative to Tobacco (CORESTA; Paris, France) "Task Force Genetically Modified Tobacco-Detection Methods" investigated the performance of qualitative and quantitative methods based on the polymerase chain reaction (PCR) for the detection and quantitation of genetically modified (GM) tobacco. In the 4 successful rounds of proficiency testing, the cauliflower mosaic virus 35S RNA promoter (CaMV 35S) and the Agrobacterium tumefaciens nopaline synthase terminator (NOS) were selected as target sequences. Blind-coded reference materials containing from 0.1 to 5.0% and from 0.15 to 4% GM tobacco were used in 2 rounds of qualitative and quantitative PCR, respectively. Eighteen laboratories from 10 countries participated in this study. Considering all methods and 2 rounds, the different laboratories were able to detect GM tobacco at the 0.1% level in 46 out of 58 tests in qualitative assays. The results of the proficiency test indicate that both end point screening and real-time quantitative methods are suitable for the detection of genetically modified organisms in tobacco leaf samples having a GM content of 0.1% or higher. The CORESTA proficiency study represents a first step towards the interlaboratory evaluation of accuracy and precision of PCR-based GM tobacco detection, which may lead to the harmonization of analytical procedures and to the enhancement of comparability of testing results produced by different laboratories.     

A statistical strategy for discouraging collusion in split-level proficiency testing schemes

A statistical strategy for discouraging collusion in split-level proficiency testing schemes was developed. When the samples are randomly labeled, the difference in the analyte levels in the two samples can be designed to maximize the probability of detecting collusion when it occurs, so that participants who colluded cannot easily receive satisfactory assessments. This favors providing convictive assessments of participants proficiency and giving the participants a level playing field.     

Establishment of proficiency testing programs in the Philippines

Accreditation and Quality Assurance - In order to improve the competitiveness of Philippine food products in the global market, the Department of Science and Technology—Food and Nutrition...     

The role of proficiency testing in method validation

The general principle of using data collected during proficiency testing (PT) for the additional purpose of assessing method performance is discussed. Particular attention is paid to the similarities and differences in conduct of traditional interlaboratory method performance studies and proficiency testing. Recommendations are made for implementing PT-based method performance studies, including statistical assessment of the data. It is concluded that, properly implemented, PT-based method performance study provides essentially equivalent information to that obtained from traditional collaborative study and may be used in appraising candidate methods with a view to their adoption as standard methods.     

A new approach in assessing microbiological results in water analysis proficiency testing

Due to the relatively large spread in the results of microbiological proficiency tests, the z-scores are often not able to detect zero or low results as being ‘bad’ results. This paper describes an adapted z-score based on the average or the standard deviation of the 50% ‘highest’ results. The combination of the adapted z-scores of four samples enables a better judgement of the performance of each laboratory. Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board or the Publisher.     

Bias analysis of proficiency testing programme results

A proficiency testing programme might involve a great number of participating laboratories coming from different countries or regions, and normally they analysed the same test materials using their own routine analytical methods. Hence, the results of a proficiency testing programme may contain valuable information which could serve purposes other than just performance evaluation. This study attempted to extract information from the results of a proficiency testing programme for the purposes of educating the participating laboratories as suggested by ISO/IEC 17043. The “bias analysis” approach introduced in this study was based on the statistical model of measurement and the nature of bias in chemical analysis. With this approach, the participating laboratories could estimate the bias associated with different settings of experimental conditions according to the statistics of subset distribution of the reported results from the participating laboratories. This would be useful for them to review the analytical procedures they used and modify their methods if needed. The approach was applied to the analysis of data obtained from a number of past proficiency testing programmes, and the findings were discussed in this paper.     

Fast and accurate determination of hemoglobin A1c using an automated sample processor

Summary The ion-exchange Chromatographic determination of hemoglobin A1c has been performed with minicolumns and an automated sample processor (Prep 1) in order to reduce sample preparation time and increase reproducibility. Since the non-availability of accurate standards makes the evaluation of the new method difficult, the different sample fractions were monitored by HPLC with UV detection (reference method). The centrifugal method has been improved until each hemoglobin fraction can be qualitatively and quantitatively recognized by the microprocessor software on the basis of the retention time. The coefficient of variation for Hb1Ac in normal samples (n=8) was 6.9% (mean 2.61%). The new method requires 30 min for 12 simultaneous determinations

---

Schnelle und genaue Bestimmung von Hämoglobin A1c mit Hilfe eines automatischen Processors

Zusammenfassung Die ionenaustausch-chromatographische Bestimmung von Hämoglobin A1c wurde mit Hilfe von Minisäulen und einem automatischenProcessor (Prep 1) durchgeführt, um Arbeitszeit einzusparen und die Reproduzierbarkeit zu erhöhen. Da die Nichtverfügbarkeit genauer Standards eine Auswertung der neuen Methode erschwert, wurden die Probefraktionen mit Hilfe der HPLC mit UV-Detektor überwacht. Das Zentrifugalsystem wurde verbessert, so daß jede Fraktion qualitativ und quantitativ mit der Mikroprocessor-Software aufgrund der Retentionszeit erkannt werden konnte. Der Variationskoeffizient für HbA1c in normalen Proben (n=8) betrug 6,9% (Mittel 2,61%). Das Verfahren benötigt für 12 Simultanbestimmungen 30 min.

     

Analytical performance is improved by regular participation in proficiency testing: an analysis of data from the Aquacheck proficiency testing scheme

Analysis of historical data from the Aquacheck proficiency testing scheme demonstrates improvement in the performance of participants correlated to the frequency of participation. Analysis of the spread of results returned by all participants, defined by the percentage relative standard deviation (RSD) and the performance (Z) scores awarded to the individual participants, indicates that the overall performance for the group of participants is consistent. However, when the frequency of participation is analysed, it can be clearly seen that those laboratories who participated in the majority of the distributions available reported 10–15% more acceptable (Z-scores between +2 and ?2) results than those labs who took part with a greatly reduced frequency. When ‘more difficult’ analytes are considered, those laboratories who participated in greater than 75% of available rounds reported up to 30% more acceptable results than those laboratories who participated in 25% or fewer, of the available rounds.