Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.     

Comparability and compatibility of proficiency testing results in schemes with a limited number of participants

Comparability and compatibility of proficiency testing (PT) results are discussed for schemes with a limited number of participants (less than 20–30) based on the use of reference materials (RMs) as test items. Since PT results are a kind of measurement/analysis/test result, their comparability is a property conditioned by traceability to measurement standards applied in the measurement process. At the same time, metrological traceability of the certified value of the RM (sent to PT participants as test items) is also important, since the PT results are compared with the RM certified value. The RM position in the calibration hierarchy of measurement standards sets the degree of comparability for PT results, which can be assessed in the scheme. However, this assessment is influenced by commutability (adequacy or match) of the matrix RM used for PT and routine samples. Compatibility of PT results is a characteristic of the collective (group) performance of the laboratories participating in PT that can be expressed as closeness of the distribution of the PT results to the distribution of the RM data. Achieving quality-of-measurement/analysis/test results in the framework of the concept “tested once, accepted everywhere” requires both comparability and compatibility of the test results.     

Traceable reference values for routine drinking water proficiency testing: first experiences

Proficiency testing (PT) results have been used to improve traceability in chemical drinking water analysis. With a generalized least-square regression the mass concentrations of As and Sb were calculated in a drinking water that had been used to prepare proficiency testing samples by a spiking procedure. From the mass concentrations in the matrix and the spiked amounts, reference values with an uncertainty budget could be calculated without the need for reference measurements. The degree to which these reference values can be regarded as traceable is discussed. The results showed slight deviations in some samples between reference values and consensus means.     

Performance evaluation for proficiency testing with a limited number of participants

Proficiency testing (PT) is an essential tool for laboratories to assess their competency. Also, participation in PT has become one of the mandatory requirements for laboratory to seek accreditation according to ISO/IEC 17025. For this reason, the effectiveness of performance evaluation by PT scheme is of great concern for the participants and for accreditation bodies as well. In practice, owing to unavailability of other appropriate alternatives, PT scheme providers may have to choose using consensus values to evaluate the performance of participants. However, such consensus values approach was not recommended by relevant international guidelines for PT schemes with limited number of participants. With the use of Monte Carlo simulation technique, this study attempted to investigate the effectiveness of using consensus values for performance evaluation in PT schemes with limited number of participants. The simulation process was schemed according to the statistical model provided by ISO 5725-1 for laboratory measurement results, which covered components like method bias, laboratory bias, and measurement precision. The effectiveness of the consensus value approach was expressed as the percentage of participants in a simulation run could get the same evaluation result, either satisfactory or unsatisfactory, against the “true value.” The findings indicated that the number of participants, choice of consensus values, mass fraction of analyte, method bias, laboratory bias, and measurement repeatability of participating laboratories would all affect the effectiveness of the consensus value approach but at different extent. However, under certain circumstances, use of consensus value could still be considered as an acceptable approach for performance evaluation even the number of participants was limited. Some of the findings were further verified using real data from PT schemes where appropriate certified reference materials or reliable reference values were available.     

Comparability of analytical results obtained in proficiency testing based on a metrological approach

A “yes–no” type of criterion is proposed for the assessment of comparability of proficiency testing (PT) results when the PT scheme is based on a metrological approach, i.e. on the use of a reference material as the test sample, etc. The criterion tests a null hypothesis concerning the insignificance of a bias of the mean of the results from a traceable value certified in the reference material used for the PT. Reliability of such assessment is determined by the probabilities of not rejecting the null hypothesis when it is true, and rejecting it when it is false (the alternative hypothesis is true). It is shown that a number of chemical, metrological and statistical reasons should be taken into account for careful formulation of the hypotheses, enabling the avoidance of an erroneous assessment of the comparability. The criterion can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results.     

Integration of metrological principles in a proficiency-testing scheme in the field of water analysis

Measurement traceability is universally recognised as one of the basic prerequisites for comparability of results obtained in different laboratories and is a basic aspect of metrological sciences such as analytical chemistry. This requirement is underscored by the increasing adoption of standards and measurement quality systems, such as laboratory accreditation against ISO/IEC 17025. Testing laboratories ensure traceability of their measurement results by using appropriate reference standards for calibration of instruments and control of measurement processes. For routine work in the field of water analysis, these standards are usually commercial solutions or in-house solutions prepared from pure products. Therefore, laboratories should demonstrate that their use of reference standards is appropriate and sufficient, which can be done by participation in an appropriate proficiency-testing scheme. The paper reports how measurement traceability of results from field laboratories (nitrite nitrogen, nitrate nitrogen, chloride and sulphate; all in water) can be demonstrated by participation in a proficiency-testing scheme based on reference values.     

ISO/IEC 17043: the new International Standard for proficiency testing

The revision of ISO/IEC Guide 43 is well underway by CASCO Working Group 28, with the author as Convener. The ballot on the committee draft was successfully completed in June 2008. The Working Group met in September to resolve the CD comments and to recommend that the document should advance as an ISO/IEC DIS. The document is in harmony with ISO/IEC 17025 (2005), ILAC G13 (2007), and the IUPAC Harmonized Protocol (2006) for proficiency testing. Major changes from Guide 43 include considerations for inspection bodies, requirements for metrological traceability of assigned values for calibration schemes, limitations on subcontracting, requirements for reporting the uncertainty of assigned values, and for including method information and technical commentary in scheme reports. Minor changes include requirements for equipment used in the manufacture and testing of proficiency test items, considerations for handling laboratories’ reported uncertainties, and the issuance of certificates of participation. These changes have not generated any substantial objections from the CASCO membership and liaisons.     

Uncertainty evaluation in proficiency testing: state-of-the-art, challenges, and perspectives

The evaluation of measurement uncertainty, and that of uncertainty statements of participating laboratories will be a challenge to be met in the coming years. The publication of ISO 17025 has led to the situation that testing laboratories should, to a certain extent, meet the same requirements regarding measurement uncertainty and traceability. As a consequence, proficiency test organizers should deal with the issues measurement uncertainty and traceability as well. Two common statistical models used in proficiency testing are revisited to explore the options to include the evaluation of the measurement uncertainty of the PTRV (proficiency test reference value). Furthermore, the use of this PTRV and its uncertainty estimate for assessing the uncertainty statements of the participants for the two models will be discussed. It is concluded that in analogy to Key Comparisons it is feasible to implement proficiency tests in such a way, that the new requirements can be met. Received: 29 September 2000 Accepted: 3 December 2000     

Development of a proficiency testing scheme for a limited number of participants in the field of natural water analysis

A proficiency testing (PT) scheme was developed for a limited number of analytical laboratories participating in the analysis of natural water in Israel. Three fit-for-intended-use reference materials (RMs) were prepared for a pilot PT from natural water: RM-blank and two in-house reference materials (IHRMs) with seven analytes fortified to achieve different levels of analyte concentrations. The measurands for the PT participants were the traceable spike values certified in the IHRMs, i.e., added mass concentrations of the analytes. The RM-blank and IHRMs were found to be homogeneous and stable over 11 days, the time necessary for the experimental part of the PT. The RMs were distributed to the PT participants as unknown test items similar to routine samples. The test results were evaluated using their deviations from the IHRM-certified values. Eight Israeli laboratories took part in the interlaboratory comparison. Individual laboratory performance and metrological compatibility of the PT results of the participants, as a local group of laboratories, were evaluated for every analyte.     

A statistical strategy for discouraging collusion in split-level proficiency testing schemes

A statistical strategy for discouraging collusion in split-level proficiency testing schemes was developed. When the samples are randomly labeled, the difference in the analyte levels in the two samples can be designed to maximize the probability of detecting collusion when it occurs, so that participants who colluded cannot easily receive satisfactory assessments. This favors providing convictive assessments of participants proficiency and giving the participants a level playing field.     

Information system and qualifying criteria for proficiency testing schemes

Since October 1998 the European Commission has financed a concerted action on Information System and Qualifying Criteria for Proficiency Testing Schemes within the 4th framework program. As a major result of this project EPTIS, the European Information System on Proficiency Testing Schemes which is available on the Internet since March 2000, is presented in this paper. Today EPTIS contains comprehensive information on approximately 640 proficiency testing schemes from 16 European countries providing information on the state of the art in proficiency testing in Europe. Finally some possible approaches for interlinkages and recognition of proficiency testing schemes are discussed.     

Advances for proficiency testing for genetic laboratory science

Nucleic acid based clinical genetic testing has undergone explosive growth in recent years due in large part to the human genome project. Characterization of the human genome has led to a molecular understanding of the pathogenesis of many human diseases, and ultimately to clinical molecular tests becoming routinely used to diagnose a wide diversity of diseases. This rapid growth in clinical molecular genetic testing coupled with the complexity of the analytical procedures underscores the necessity for proficiency testing (i.e. external quality assessment) to allow laboratories offering such services the ability to evaluate their analytical procedures via inter-laboratory comparisons. The American College of Medical Genetics (ACMG) in partnership with the College of American Pathologists (CAP) have been offering proficiency testing for clinical molecular genetics laboratories since 1995, and presently have more than 230 laboratories from 11 countries enrolled in this program. This paper describes the evolution of this program and several challenges encountered in the delivery of a proficiency testing program for laboratories offering clinical molecular genetic services. Received: 13 April 2002 Accepted: 18 July 2002     

Global comparability of analytical results

 Global decisions made on the basis of chemical analytical results need global comparability. The concept of traceability is important and necessary but not sufficient to achieve global comparability, especially for routine analysis in environmental protection, health care and public safety. Sometimes comparability is only determined by the method. In this case validated methods have to be known and recognized internationally. In addition, a minimum level of proficiency of the personnel producing analytical results has to be guaranteed. Finally, a system of measures designed to build up trust is needed for worldwide acceptance of analytical results. The method of self-declaration by the supplier of chemical analytical services has the advantage of presenting specific responsibility and gaining a good reputation. Additional methods should be employed. The evaluation of reference materials by specific quality criteria is mentioned as an example. Received: 1 July 1998 · Accepted: 20 July 1998     

Statistical treatment of proficiency testing data

 There are three stages to evaluating a laboratory's results in an interlaboratory proficiency test: establishing the correct result for the test item, determining an evaluation statistic for the particular result, and establishing an acceptable range. There are a wide variety of procedures for accomplishing these three stages and a correspondingly wide variety of statistical techniques in use. Currently in North America the largest number of laboratory proficiency test programs are in the clinical laboratory field, followed by programs for environmental laboratories that test drinking water and waste water. Proficiency testing in both of these fields is under the jurisdiction of the federal government and other regulatory and accreditation agencies. Many of the statistical procedures are specified in the regulations, to assure comparability of different programs and a fair evaluation of performance. In this article statistical procedures recommended in International Organization for Standardization Guide 43, Part 1, are discussed and compared with current practices in North America. Received: 22 April 1998 · Accepted: 12 May 1998     

Matrix materials for proficiency testing: optimization of a procedure for spiking soil with gamma-emitting radionuclides

The IAEA Reference Materials Group of the Chemistry Unit, Agency’s Laboratories Seibersdorf, has developed and optimized a procedure for spiking some environmental matrices with gamma-emitting radionuclides. This paper describes the spiking procedure, homogeneity testing of the spiked material, and assignment of property values and their associated uncertainties for the radionuclides ⁵⁴Mn, ⁶⁵Zn, ⁶⁰Co, ¹⁰⁹Cd, ¹³⁴Cs, ¹³⁷Cs, ²¹⁰Pb, and ²⁴¹Am. This procedure has already been successfully used in an IAEA proficiency test on the determination of ¹³⁷Cs and ²¹⁰Pb in spiked soil and has been found to be appropriate for production of soil materials for proficiency testing and internal quality control samples. The main advantage of this procedure is a low uncertainty arising from heterogeneity, which was found to be less than 1.2% for all the analytes studied.      

Practical uses of proficiency testing as valuable tools for validation and performance assessment in environmental analysis

Besides their role as an external quality control tool, PT results or samples could be used as an alternative to fulfil some of the quality assurance requirements such as analytical precision, uncertainty assessment, and internal quality control. This additional use of proficiency testing could help laboratories to reduce the financial impact of their quality assurance process. The purpose of this paper is to highlight some practical uses of PT results or samples in the environmental analytical field, which have been implemented at ISSeP (Institut Scientifique de Service Public), either for method validation or for internal quality control.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

Nonparametric assessment of comparability of analytical results obtained in proficiency testing based on a metrological approach

A nonparametric sign test is implemented for assessment of comparability of proficiency testing (PT) results when their distribution differs from the normal or other known distribution. It allows testing the null hypothesis about insignificance of the bias of median of results obtained in PT from the traceable certified value of the reference material used in PT as test items, i.e., the hypothesis stating that comparability of the PT results is successful. Probability of type I error of rejecting the null hypothesis when it is true, and probability of type II error of it not rejecting when it is false (the alternative hypothesis about unsuccessful comparability is true) are considered. The test can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results when the comparability criterion developed for a normal results distribution (Accred. Qual. Assur. 10:466–470) is not applicable.     

The practice of proficiency testing in China

This paper briefly summarises the current situation for proficiency testing (PT) in China, outlines the policy for PT of China’s national accreditation committee for laboratories (CNACL), and exemplifies activities of the CNACL’s metal working group. Received: 9 December 2000 Accepted: 14 December 2000