Proficiency testing in the light of a new rationale in metrology

The novel proposed definition of measurement result in the international metrology vocabulary requires a revision of standards and guidelines for proficiency testing (PT), and a new approach to processing proficiency data is needed to test the ability of laboratories to present not only unbiased quantity values, but reliable estimates of their uncertainty. Hence, an accepted reference value with the smallest possible uncertainty is needed to ascertain the proficiency of laboratories reporting results with lower than average uncertainty. A strategy based on the T-statistic is proposed leading to an accepted reference value that fully reflects the uncertainties reported by participants in a PT scheme and permits calculation of E _n numbers to distinguish whether or not measurement results are consistent with the accepted definition of the measurand. The strategy is applied to PT data from a recent international laboratory intercomparison of uranium isotopic ratios.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Evaluation of proficiency testing results taking into account their uncertainties

A new composite score for the evaluation of performance of proficiency testing participants is proposed. The score is based on a combination of the z-score, uncertainty of a participant’s measurement result and uncertainty of the proficiency testing scheme’s assigned value. The use of such a composite score will allow evaluation not only of the participant’s ability to determine an analyte in corresponding matrix, but also their understanding of the uncertainty in the obtained analytical result. The score may be helpful for the laboratory’s quality system and for laboratory accreditation according to ISO 17025.     

Uncertainty evaluation in proficiency testing: state-of-the-art, challenges, and perspectives

The evaluation of measurement uncertainty, and that of uncertainty statements of participating laboratories will be a challenge to be met in the coming years. The publication of ISO 17025 has led to the situation that testing laboratories should, to a certain extent, meet the same requirements regarding measurement uncertainty and traceability. As a consequence, proficiency test organizers should deal with the issues measurement uncertainty and traceability as well. Two common statistical models used in proficiency testing are revisited to explore the options to include the evaluation of the measurement uncertainty of the PTRV (proficiency test reference value). Furthermore, the use of this PTRV and its uncertainty estimate for assessing the uncertainty statements of the participants for the two models will be discussed. It is concluded that in analogy to Key Comparisons it is feasible to implement proficiency tests in such a way, that the new requirements can be met. Received: 29 September 2000 Accepted: 3 December 2000     

Achieving traceability in chemical measurement—a metrological approach to proficiency testing

ISO/IEC 17025 requires that testing laboratories establish the traceability of their measurements, preferably to the SI units of measurement. The responsibility for establishing traceability lies with each individual laboratory and must be achieved by following a metrological approach.The results of measurements made in such a way are traceable to the standards used in method validation and to the calibration standards used during the measurement process. If these standards are traceable to SI then the measurements will also be traceable to SI.Participation in appropriate proficiency studies (an ISO/IEC 17025 requirement) enables laboratories to demonstrate the comparability of their measurements. If the materials used for the studies have traceable assigned values, then proficiency testing also provides information about measurement accuracy and confirms, or otherwise, that appropriate traceability has been established. This paper will report on a new approach for the establishment of traceable assigned values for chemical testing proficiency studies. The work is conducted at a "fit for purpose" level of measurement uncertainty, with costs contained at a level similar to previous "consensus" based proficiency studies. By establishing traceable assigned values in a cost effective way, NARL aims to demonstrate the added value of the metrological approach to participant laboratories.     

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

The determination of an accepted reference value from proficiency data with stated uncertainties

Proficiency data with stated uncertainties represent a unique opportunity for testing that the reported uncertainties are consistent with the Guide to the expression of uncertainty in measurement (GUM). In most proficiency tests, however, this opportunity is forfeited, because proficiency data are processed without regard to their uncertainties. In this paper we present alternative approaches for determining a reference value as the weighted mean of all mutually consistent results and their stated uncertainties. Using an accepted reference value each reported uncertainty estimate can be expressed as an E _n number, but a value of confirms its validity only if the uncertainty of the reference value is negligible in comparison.Reference values calculated for results from an International Measurement Evaluation Programme (IMEP-9) by “bottom up” as well as “top down” methods were practically identical, although the first strategy yielded the lowest uncertainty. A plot of individual coefficients of variation (CV) versus E _n numbers helps interpretation of the proficiency data, which could be used to validate relative uncertainties down to <1%.     

Proficiency testing in analytical laboratories: how to make it work

 This paper covers the role of proficiency testing schemes in providing an occasional but objective means of assessing and documenting the reliability of the data produced by a laboratory, and in encouraging the production of data that are "fit-for-purpose". A number of aspects of proficiency testing are examined in order to highlight features critical for their successful implementation. Aspects that are considered are: accreditation, the economics and scope of proficiency testing schemes, methods of scoring, assigned values, the target value of standard deviation σ_p, the homogeneity of the distributed material, proficiency testing in relation to other quality assurance measures and whether proficiency testing is effective. Stress is placed on the importance of any proficiency testing scheme adhering to a protocol that is recognised, preferably internationally. It is also important that the results from the scheme are transparent to both participating laboratory and its "customer". Received: 03 November 1995 Accepted: 20 November 1995     

Quality issues in proficiency testing

 Proficiency testing is a means of assessing the ability of laboratories to competently perform specific tests and/or measurements. It supplements a laboratory's own internal quality control procedure by providing an additional external audit of their testing capability and provides laboratories with a sound basis for continuous improvement. It is also a means towards achieving comparability of measurement between laboratories. Participation is one of the few ways in which a laboratory can compare its performance with that of other laboratories. Good performance in proficiency testing schemes provides independent evidence and hence reassurance to the laboratory and its clients that its procedures, test methods and other laboratory operations are under control. For test results to have any credibility, they must be traceable to a standard of measurement, preferably in terms of SI units, and must be accompanied by a statement of uncertainty. Analytical chemists are coming to realise that this is just as true in their field as it is for physical measurements, and applies equally to proficiency testing results and laboratory test reports. Recent approaches toward ensuring the quality and comparability of proficiency testing schemes and the means of evaluating proficiency test results are described. These have led to the drafting of guidelines and subsequently to the development of international requirements for the competence of scheme providers. Received: 2 January 1999 · Accepted: 7 April 1999     

Proficiency testing scheme for the harmonization and comparability of analytical measurements

According to the experience of the successful implementation of proficiency tests (PT) by using the certified reference value as the assigned value, a new scheme of evaluation is presented by suggesting the use of the uncertainty associated with the certified value. The technical performance of laboratories is evaluated by the parameter quadratic mean error (QME), which is the square root of the sum of the square of the bias and that of the standard deviation of the laboratory. This parameter is considered as the estimate of the measurement uncertainty of the laboratory and is compared to the uncertainty (U) associated with the certified value provided by an NMI. Considering that the calibration and measurement capability, known as the CMC, is recognized among NMIs, the ratio QME/U enables us to compare the PT relative to the CMC of an NMI, and, consequently, to any other comparison results based on the CMC of signatories of the mutual recognition arrangement (MRA) of the International Committee of Weights and Measure (CIPM). Presented at BERM-11, October 2007, Tsukuba, Japan.     

Establishment of an assigned value and its uncertainty for tumour markers in proficiency testing in China

Proficiency testing (PT) is an important aspect of clinical laboratory quality control for tumour markers. To evaluate the performance of an individual laboratory, the assigned value and the standard uncertainty of each tumour marker must be determined. However, no data about the standard uncertainty of the assigned value of any analyte in PT is available in China. The aim of this paper is to introduce a new approach for establishment of assigned value and its standard uncertainty for tumour markers. Five sets of control materials at different concentrations were assigned to each laboratory that participated in PT for tumour markers in 2012. We collected the data from each laboratory using the Clinet-EQA reporting system V1.5 and Clinet-EQA evaluation system V1.0 and developed a software program to calculate the assigned value and its uncertainty according to ISO 13528 (Statistical methods for use in proficiency testing by interlaboratory comparisons. ISO 13528. International Standards Organisation, Geneva, 2005). The assigned value and inter-laboratory CV calculated by the new approach was compared with that obtained by the current approach used in PT in our country. No significant difference between the two outcomes was found in our study. Most assigned values had a standard uncertainty that can meet the criterion: $ u_{X} \le \, 0.3\hat{\sigma } $ u X ≤ 0.3 σ ? , which meant the new approach for establishment of the assigned value was acceptable. It is expected that the new approach to determination of the assigned value and its standard uncertainty for PT for tumour markers in China will enable participants to evaluate their measurement uncertainty.     

Assessment of the determination of heavy metals in organic soil improvers by ICP–OES

Directive 86/278/EEC sets maximum levels of heavy metals in sewage sludge used in agriculture to regulate the risk of using these organic soil improvers. The assessment of the compliance of soil improvers with this legislation should be supported on measurements with adequately low uncertainty. This work presents a strategy for assessing the performance of the determination of aqua regia extractable Cu, Zn and Cr in organic soil improvers and urban sewage sludges following EN 13650 standard. The measurement procedure validation involves checking the adequacy of the linear weighted regression model for ICP–OES calibration, the determination of the limit of quantification, the assessment of measurement repeatability, intermediate precision and trueness, and the evaluation of the measurement uncertainty using the differential approach. Routine tests quality control, including the estimated measurement uncertainty, is checked through the analysis of control standards, equivalent to calibrators, and reference materials from proficiency tests. Since the estimated relative expanded uncertainty is smaller than the defined target value (40 %), measurements are fit for assessing compliance of the mass fraction of heavy metals in organic soil improvers with the council directive.     

Proficiency test of pH,conductivity and dissolved oxygen concentration field measurements in river water

In 2013, Proftest SYKE organised the first proficiency test (PT) in Finland for field measurements of temperature, conductivity, dissolved oxygen concentration, oxygen saturation and pH value in river water. The aim was to pilot the organisation of an in situ proficiency test—particularly, how to select the test location—and how to assess the homogeneity and stability of the measurement site and the water to be tested. The focus was also to evaluate the suitability of the common field sensors used for water analysis, as well as the comparability of the results between the instruments under field conditions. The overall application of quality assurance procedures was also surveyed. This paper deals with the results, findings and recommendations for the measurement of pH, conductivity and dissolved oxygen concentration. In total, nine participants with 16 sensors took part in the proficiency test. For the evaluation of the performance of each participant, z scores were calculated allowing 3 % to 8 % deviation from the assigned value. The standard deviation of the participant’s results was lower than organiser expected, and 80 % of the results for pH, 79 % for conductivity and 69 % for dissolved oxygen concentration were regarded as satisfactory. According to the results, the most challenging measurement was for dissolved oxygen with a Clark cell-type measurement principle based on electrochemical reaction. All sensors tested in the PT were less than 5 years, old and they were calibrated according to the manufacturer’s instructions. None of the participants had estimated measurement uncertainty for their sensor measurements. In addition, internal and external quality assurance protocols were usually lacking.     

An uncertainty evaluation for multiple measurements by GUM, III: using a correlation coefficient

After a measurement, a measured value and a measurement uncertainty are produced as a measurement result. By a repeated measurement, another measurement result is produced. Between the individual results of the two measurements, it is shown that there may be a significant correlation. A correlation coefficient can be determined when a GUM-compliant uncertainty budget for a measurement is available. Utilizing the correlations between the N individual results, an equation is derived to combine the N individual uncertainties of N measurements. Using the newly derived equation including the correlation coefficient, three measurement uncertainties of three measurement results are combined as an example. The combined uncertainty is compared with the uncertainty of a measurement which treats the three individual measurements as one process. Papers published in this section do not necessarily reflect the opinion of the editors, the editorial board, or the publisher.     

Operation of a proficiency testing scheme of trace elements in sewage sludge with reference values

A proficiency test (PT) was carried out to investigate the quality of laboratory results measuring trace elements in sewage sludge. The scheme relied on reference values, established through flame atomic absorption spectroscopy and instrumental neutron activation analysis. Within the framework of the project, ”Proficiency testing in Central and Eastern European Countries”, the measurements were done in parallel using induced coupled plasma mass spectrometry. This paper discusses the design of the PTS, the metrological concepts behind it and the way in which it was operated. The discussion includes the estimation of measurement uncertainty of the reference values obtained, homogeneity testing and its uncertainty evaluation, as well as all relevant quality assurance aspects. From the results, it can be concluded that all measurement methods involved agree within their respective uncertainties. Furthermore, it can be concluded that it is feasible to operate a proficiency test of trace elements in sewage sludge using reference values. The agreement between the reference values and the consensus values is satisfactory.     

A new approach for proficiency test exercise data evaluation

In this paper, a new data evaluation method for proficiency test exercises consisting of a combination of a z-test, a zeta test and an uncertainty outlier test is presented. This new method is compared with eight other evaluation methods (both measurement uncertainty using and measurement uncertainty ignoring) in common use and/or recommended by ISO 13528. The data set used to test the evaluation methods is real data and consists of the ⁹⁵Nb results of the National Physical Laboratory Environmental Radioactivity Proficiency Test Exercise 2007. The evaluation of 14 out of 32 results were affected by the choice of method.     

Set-up and evaluation of interlaboratory studies

Interlaboratory comparison by means of method performance precision and bias studies and proficiency testing schemes are described. The set-up of the experiments as well as the evaluation of the data by means of graphical and statistical methods are considered. The use of interlaboratory data for the estimation of measurement uncertainty is also addressed.     

Providing a traceable assigned value in a proficiency testing programme on the determination of benzoic acid in orange juice

A proficiency testing (PT) programme on the determination of benzoic acid in orange juice was organised by the Chemical Metrology Laboratory, Health Sciences Authority, Singapore, for the local food testing laboratories. The PT programme used a metrologically traceable assigned value determined by isotope dilution mass spectrometry to assess the performance of the participating laboratories. This paper discusses the reasons and approach to applying metrological principles to determine the assigned value of a PT programme. The procedure undertaken by the Laboratory to prepare a batch of PT sample, conduct homogeneity and stability testings, as well as value assignment, is presented. The discussions also include the estimation of the measurement uncertainties arising from the characterisation, testing for homogeneity and testing for stability of the PT sample. The Coordinator of the PT programme used the assigned value and its associated measurement uncertainty, together with those of the participating laboratories to assess their performances. Statistical tools applied in the performance evaluations are also presented.