Comparability of analytical results obtained in proficiency testing based on a metrological approach

A “yes–no” type of criterion is proposed for the assessment of comparability of proficiency testing (PT) results when the PT scheme is based on a metrological approach, i.e. on the use of a reference material as the test sample, etc. The criterion tests a null hypothesis concerning the insignificance of a bias of the mean of the results from a traceable value certified in the reference material used for the PT. Reliability of such assessment is determined by the probabilities of not rejecting the null hypothesis when it is true, and rejecting it when it is false (the alternative hypothesis is true). It is shown that a number of chemical, metrological and statistical reasons should be taken into account for careful formulation of the hypotheses, enabling the avoidance of an erroneous assessment of the comparability. The criterion can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results.     

Local comparability of proficiency testing results: determination of concrete slump and compressive strength values

A proficiency testing (PT) scheme is developed for comparability assessment of results of concrete slump and compressive strength determination. The scheme is based on preparing of a test portion/sample of a concrete in-house reference material (IHRM) at a reference laboratory (RL) in the same conditions for every PT participant. Therefore, in this scheme IHRM instability is not relevant as a source of measurement/test uncertainty, while intra- and between-samples inhomogeneity parameters are evaluated using the results of RL testing of the samples taken at the beginning, the middle and the end of the PT experiment. The IHRM assigned slump and compressive strength values are calculated as averaged RL results. Their uncertainties include the measurement/test uncertainty components and the components arising from the material inhomogeneity. The test results of 25 PT participants were compared with the IHRM assigned values taking into account both the uncertainties of the assigned values and the measurement/test uncertainties of the participants. Since traceability of the IHRM assigned values to the international measurement standards and SI units cannot be stated, local comparability of the results is assessed. It is shown, that comparability of the slump and compressive strength determination results is satisfactory, while uncertainty evaluation for slump results requires additional efforts.     

Global comparability of analytical results

 Global decisions made on the basis of chemical analytical results need global comparability. The concept of traceability is important and necessary but not sufficient to achieve global comparability, especially for routine analysis in environmental protection, health care and public safety. Sometimes comparability is only determined by the method. In this case validated methods have to be known and recognized internationally. In addition, a minimum level of proficiency of the personnel producing analytical results has to be guaranteed. Finally, a system of measures designed to build up trust is needed for worldwide acceptance of analytical results. The method of self-declaration by the supplier of chemical analytical services has the advantage of presenting specific responsibility and gaining a good reputation. Additional methods should be employed. The evaluation of reference materials by specific quality criteria is mentioned as an example. Received: 1 July 1998 · Accepted: 20 July 1998     

Comparability and compatibility of proficiency testing results in schemes with a limited number of participants

Comparability and compatibility of proficiency testing (PT) results are discussed for schemes with a limited number of participants (less than 20–30) based on the use of reference materials (RMs) as test items. Since PT results are a kind of measurement/analysis/test result, their comparability is a property conditioned by traceability to measurement standards applied in the measurement process. At the same time, metrological traceability of the certified value of the RM (sent to PT participants as test items) is also important, since the PT results are compared with the RM certified value. The RM position in the calibration hierarchy of measurement standards sets the degree of comparability for PT results, which can be assessed in the scheme. However, this assessment is influenced by commutability (adequacy or match) of the matrix RM used for PT and routine samples. Compatibility of PT results is a characteristic of the collective (group) performance of the laboratories participating in PT that can be expressed as closeness of the distribution of the PT results to the distribution of the RM data. Achieving quality-of-measurement/analysis/test results in the framework of the concept “tested once, accepted everywhere” requires both comparability and compatibility of the test results.     

A new approach in assessing microbiological results in water analysis proficiency testing

Due to the relatively large spread in the results of microbiological proficiency tests, the z-scores are often not able to detect zero or low results as being ‘bad’ results. This paper describes an adapted z-score based on the average or the standard deviation of the 50% ‘highest’ results. The combination of the adapted z-scores of four samples enables a better judgement of the performance of each laboratory. Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board or the Publisher.     

Proficiency testing scheme for the harmonization and comparability of analytical measurements

According to the experience of the successful implementation of proficiency tests (PT) by using the certified reference value as the assigned value, a new scheme of evaluation is presented by suggesting the use of the uncertainty associated with the certified value. The technical performance of laboratories is evaluated by the parameter quadratic mean error (QME), which is the square root of the sum of the square of the bias and that of the standard deviation of the laboratory. This parameter is considered as the estimate of the measurement uncertainty of the laboratory and is compared to the uncertainty (U) associated with the certified value provided by an NMI. Considering that the calibration and measurement capability, known as the CMC, is recognized among NMIs, the ratio QME/U enables us to compare the PT relative to the CMC of an NMI, and, consequently, to any other comparison results based on the CMC of signatories of the mutual recognition arrangement (MRA) of the International Committee of Weights and Measure (CIPM). Presented at BERM-11, October 2007, Tsukuba, Japan.     

IMEP in support of the comparability of measurement results across the Romanian chemical metrology infrastructure

On the basis of quantitative chemical measurements many important decisions are made in support of legislation or in industrial processes or social aspects. For this reason it is important to improve the quality of chemical measurement results and thus make them comparable and acceptable everywhere. The measurement quality is important to enable an equivalent implementation of the European Union regulations and directives across an enlarged EU. In this context, the European Commission–Joint Research Centre–Institute for Reference Materials and Measurement (EC-JRC-IRMM) set up a programme to improve the scientific basis for metrology in chemistry (MiC) in EU candidate countries in the framework of EU enlargement. Several activities were initiated, such as training, fellowships, sponsoring of seminars, conferences and participation in interlaboratory comparisons. To disseminate measurement traceability, IRMM provides through its International Measurement Evaluation Programme (IMEP) an interlaboratory tool to enable the benchmarking of laboratory performance. IMEP emphasizes the metrological aspects of measurement results, such as traceability and measurement uncertainty. In this way it has become a publicly available European tool for MiC. The Romanian Bureau of Legal Metrology – National Institute of Metrology (BRML-INM) actively supports the participation of Romanian authorized and field laboratories in IMEP interlaboratory comparisons. This paper describes the interest of Romanian laboratories participating in this programme, the analytical and metrological problems that became relevant during these exercises and some actions for improvement. The results from Romanian laboratories participating in IMEP-12 (water), IMEP-16 (wine), IMEP-17 (human serum) and IMEP-20 (tuna fish) are presented. To conclude, the educational and training activities at national level organized jointly by the Romanian National Institute of Metrology (INM) and IRMM are also mentioned.     

Benefits arising from proficiency testing schemes: The comparison of analytical methods

 The primary objective of proficiency testing (PT) is in the provision of information and support to participating laboratories, to enable them to monitor and improve the quality of their measurements. However, other benefits can be obtained from PT. These include the comparison of data for a given measurement by different methods, the validation of new methods, and the provision of information for laboratories' customers and accreditation bodies. This paper considers the subject of method comparison, and highlights some of the approaches which can be followed, as well as the practical use to which this can be put, to benefit the analytical community more widely. This is illustrated by a case study concerning the measurement of haze in beer. In this study the United Kingdom Institute of Brewing (IoB) conducted a survey of participants in the Brewing Analytes Proficiency Scheme (BAPS). From the survey data taken together with data from the BAPS scheme, the IoB is now in a position to give guidance on the use of particular instruments and procedures, as well as consider changes to the scope of the BAPS scheme to provide greater benefits for participants concerned with measuring haze. Received: 3 March 1998 · Accepted: 9 June 1998     

Evaluation of proficiency testing results taking into account their uncertainties

A new composite score for the evaluation of performance of proficiency testing participants is proposed. The score is based on a combination of the z-score, uncertainty of a participant’s measurement result and uncertainty of the proficiency testing scheme’s assigned value. The use of such a composite score will allow evaluation not only of the participant’s ability to determine an analyte in corresponding matrix, but also their understanding of the uncertainty in the obtained analytical result. The score may be helpful for the laboratory’s quality system and for laboratory accreditation according to ISO 17025.     

Practical uses of proficiency testing as valuable tools for validation and performance assessment in environmental analysis

Besides their role as an external quality control tool, PT results or samples could be used as an alternative to fulfil some of the quality assurance requirements such as analytical precision, uncertainty assessment, and internal quality control. This additional use of proficiency testing could help laboratories to reduce the financial impact of their quality assurance process. The purpose of this paper is to highlight some practical uses of PT results or samples in the environmental analytical field, which have been implemented at ISSeP (Institut Scientifique de Service Public), either for method validation or for internal quality control.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

Experiences from the assessment of proficiency testing providers

Following a pilot project in 2001–2002, UKAS accredited nine PT providers. During the course of the pilot and at subsequent surveillance visits, UKAS has gained experience in assessing organisations against the criteria detailed within ISO/IEC Guide 43-1:1997 and ILAC G13:2000. This article explores some of the key aspects that arose during the pilot and how UKAS has accommodated the addition of the accreditation of PT providers in to its range of services.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia     

Approaches to uncertainty evaluation based on proficiency testing schemes in chemical measurements

Since the advent of the Guide to the Expression of Uncertainty in Measurement (GUM) founding the principles of uncertainty evaluation, numerous projects have been carried out to develop alternative practical methods when it is impossible to model technical or economical aspects of the measurement process. These methods can use all the experimental data available to the laboratories, such as repeatability, reproducibility, quality-control charts, etc. The studies presented in this paper compare the results obtained by the modelling method from GUM with the uncertainties found by applying alternative methods. They show two examples, one in the field of environmental monitoring, the other in the biomedical field, based on the exploitation of PT schemes results. Presented at BERM-11, October 2007, Tsukuba, Japan.     

Designs of experiment for proficiency testing with a limited number of participants

Metrological designs of experiment for proficiency testing (PT) with a limited number of participants are discussed. The designs are based on development of in-house reference materials (IHRMs) with traceable assigned values and fit-for-purpose uncertainties, used in a PT scheme as the measurement standards. When adequate certified reference materials (CRMs) are available, a comparative approach for IHRM development, using simultaneous analysis of IHRM and CRM test portions in pairs, is proposed for PT objectives. In the case where adequate CRMs are not available – for example, in the field of analysis of unstable aqueous systems – the function of the measurement standard for PT can be fulfilled by a synthetic IHRM prepared gravimetrically using non-adequate, non-aqueous CRMs or pure substances. The CRM or the pure substance is used as a spike for fortification of a natural water sample, while the natural water sample is used as a working IHRM for the spike determination. In this case the traceability chain is longer, since two IHRMs are added for one quantity determination, but it remains unbroken.Presented at the Second International Conference on Metrology –Trends and Applications in Calibration and Testing Laboratories, 4–6 November 2003, Eilat, Israel     

水产品中呋喃唑酮代谢物AOZ的检测能力验证分析

介绍了全国多个行业51家实验室参加水产品中呋喃唑酮代谢物AOZ的检测能力验证计划的情况。选用活虾药浴暂养后制得并通过均匀性F检验和稳定性t检验的虾肉糜样作为测试样品,采用稳健统计技术对试验结果进行了分析。讨论了水产品中呋喃唑酮代谢物AOZ检测的一些关键点。     

Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.     

The practice of proficiency testing in China

This paper briefly summarises the current situation for proficiency testing (PT) in China, outlines the policy for PT of China’s national accreditation committee for laboratories (CNACL), and exemplifies activities of the CNACL’s metal working group. Received: 9 December 2000 Accepted: 14 December 2000     

The role of proficiency testing in the detection and resolution of calibration bias in the LeadCare® blood lead analyzer; limitations of peer-group assessment

Following implementation of the CLIA ‘88 laboratory regulations, the primary role of proficiency testing (external quality assessment, PT) in the U.S. has been widely viewed as one of assuring regulatory compliance. PT can also be an effective tool for detecting widespread analytical problems, subject to limitations based on the method of PT assigned value determination. A recent case study describes the role of two PT programs in detecting and resolving a calibration bias in the LeadCare blood lead analyzer, and illustrates the limitations of peer-group target determination in fulfilling that PT role.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Environmental analysis in the Czech Republic: organisation of proficiency testing and its use in laboratory assessment

The history, origin, and development of a system for monitoring and assessing water and other environmental laboratories in the Czech Republic is described. The system started in 1991 and has matured to its present complexity with similarities to the accreditation systems found in other countries. Differences from internationally recognized procedures are being corrected step by step. During the first year of its existence ASLAB, as part of its brief, organised proficiency testing (PT) programs for fifty laboratories. Today the total number of regularly participating laboratories exceeds 700 from the Czech Republic, the Slovak Republic, and Germany. This paper describes the ASLAB PT system, discusses some experiences with its use, and describes the use of PT results in assessment of the competence of laboratories. Received: 12 October 2000 Accepted: 7 January 2001     

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.