Determination of Assigned Values for Cadmium,Cobalt and Manganese in Drinking Water Proficiency Testing

In proficiency tests the selection of an appropriate assigned value is one of the most critical points. Determination of the assigned value for proficiency test can be done in 2 ways: based on reference value and consensus value. The use of such a consensus value as an assigned value is a commonplace in proficiency testing, because there is seldom at present a cost effective alternative. However, the consensus value has a weakness in that it could be significantly different from the true value due largely inexperienced laboratories. Therefore in this work, the reference values were used as the assigned values for all analytes of interest. The reference values in spiked drinking water samples were produced by the Metrology in Chemistry Laboratory of Research Center for Chemistry - Indonesian Institute of Sciences (RCChem-LIPI) with traceable to SI units through standard solutions from the National Institute Standard and Technology, USA. The reference values of cobalt and manganese were obtained by measurement using two from three comparable methods of Graphite Furnace-Atomic Absorption Spectrometry (GF-AAS), Inductively Coupled Plasma-Optical Emission Spectrometry (ICP-OES) and Inductively Coupled Plasma-Mass Spectrometry (ICP-MS) while that for cadmium was obtained by a primary method of ICP-Isotope Dilution Mass Spectrometry (ICP-IDMS).The obtained reference values were used to evaluate the laboratory performance result in a proficiency testing scheme of cadmium, cobalt and manganese in drinking water which was provided in 2014. All fifty nine laboratories reported cadmium and cobalt at low concentration levels which were about more than 82% showed satisfactory performance results. Similarly, the results of manganese were also comparable at high level concentrations.     

Evaluation of the use of consensus values in proficiency testing programmes

Proficiency testing (PT) is an essential tool used by laboratory accreditation bodies to assess the competency of laboratories. Because of limited resources of PT providers or for other reasons, the assigned reference value used in the calculation of z-score values has usually been derived from some sort of consensus value obtained by central tendency estimators such as the arithmetic mean or robust mean. However, if the assigned reference value deviates significantly from the ‘true value’ of the analyte in the test material, laboratories’ performance will be evaluated incorrectly. This paper evaluates the use of consensus values in proficiency testing programmes using the Monte Carlo simulation technique. The results indicated that the deviation of the assigned value from the true value could be as large as 40%, depending on the parameters of the proficiency testing programmes under investigation such as sample homogeneity, number of participant laboratories, concentration level, method precision and laboratory bias. To study how these parameters affect the degree of discrepancy between the consensus value and the true value, a fractional factorial design was also applied. The findings indicate that the number of participating laboratories and the distribution of laboratory bias were the prime two factors affecting the deviation of the consensus value from the true value.     

A review of proficiency testing scheme in Republic of Korea: in the field of water analysis

For regulatory purposes, it is often required to check the quality of environmental chemical analyses by routine proficiency testing (PT). In this study, the PT scheme of environmental laboratories in Republic of Korea has been reviewed. Especially, real water-based reference materials (RMs) were prepared and distributed for PT schemes. The consensus values from the PT were calculated by robust statistics with its standard uncertainties. The relative bias between the reference values and the consensus values was used as one of the indicators for the determination of the assigned values. The relationship between the performances of the laboratories, expressed by Z scores, and the concentration levels of the RMs were also revealed.     

Operation of a proficiency testing scheme of trace elements in sewage sludge with reference values

A proficiency test (PT) was carried out to investigate the quality of laboratory results measuring trace elements in sewage sludge. The scheme relied on reference values, established through flame atomic absorption spectroscopy and instrumental neutron activation analysis. Within the framework of the project, ”Proficiency testing in Central and Eastern European Countries”, the measurements were done in parallel using induced coupled plasma mass spectrometry. This paper discusses the design of the PTS, the metrological concepts behind it and the way in which it was operated. The discussion includes the estimation of measurement uncertainty of the reference values obtained, homogeneity testing and its uncertainty evaluation, as well as all relevant quality assurance aspects. From the results, it can be concluded that all measurement methods involved agree within their respective uncertainties. Furthermore, it can be concluded that it is feasible to operate a proficiency test of trace elements in sewage sludge using reference values. The agreement between the reference values and the consensus values is satisfactory.     

Proficiency testing in the light of a new rationale in metrology

The novel proposed definition of measurement result in the international metrology vocabulary requires a revision of standards and guidelines for proficiency testing (PT), and a new approach to processing proficiency data is needed to test the ability of laboratories to present not only unbiased quantity values, but reliable estimates of their uncertainty. Hence, an accepted reference value with the smallest possible uncertainty is needed to ascertain the proficiency of laboratories reporting results with lower than average uncertainty. A strategy based on the T-statistic is proposed leading to an accepted reference value that fully reflects the uncertainties reported by participants in a PT scheme and permits calculation of E _n numbers to distinguish whether or not measurement results are consistent with the accepted definition of the measurand. The strategy is applied to PT data from a recent international laboratory intercomparison of uranium isotopic ratios.     

Quality issues in proficiency testing

 Proficiency testing is a means of assessing the ability of laboratories to competently perform specific tests and/or measurements. It supplements a laboratory's own internal quality control procedure by providing an additional external audit of their testing capability and provides laboratories with a sound basis for continuous improvement. It is also a means towards achieving comparability of measurement between laboratories. Participation is one of the few ways in which a laboratory can compare its performance with that of other laboratories. Good performance in proficiency testing schemes provides independent evidence and hence reassurance to the laboratory and its clients that its procedures, test methods and other laboratory operations are under control. For test results to have any credibility, they must be traceable to a standard of measurement, preferably in terms of SI units, and must be accompanied by a statement of uncertainty. Analytical chemists are coming to realise that this is just as true in their field as it is for physical measurements, and applies equally to proficiency testing results and laboratory test reports. Recent approaches toward ensuring the quality and comparability of proficiency testing schemes and the means of evaluating proficiency test results are described. These have led to the drafting of guidelines and subsequently to the development of international requirements for the competence of scheme providers. Received: 2 January 1999 · Accepted: 7 April 1999     

Uncertainty evaluation in proficiency testing: state-of-the-art, challenges, and perspectives

The evaluation of measurement uncertainty, and that of uncertainty statements of participating laboratories will be a challenge to be met in the coming years. The publication of ISO 17025 has led to the situation that testing laboratories should, to a certain extent, meet the same requirements regarding measurement uncertainty and traceability. As a consequence, proficiency test organizers should deal with the issues measurement uncertainty and traceability as well. Two common statistical models used in proficiency testing are revisited to explore the options to include the evaluation of the measurement uncertainty of the PTRV (proficiency test reference value). Furthermore, the use of this PTRV and its uncertainty estimate for assessing the uncertainty statements of the participants for the two models will be discussed. It is concluded that in analogy to Key Comparisons it is feasible to implement proficiency tests in such a way, that the new requirements can be met. Received: 29 September 2000 Accepted: 3 December 2000     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

Is the assessment of interlaboratory comparison results for a small number of tests and limited number of participants reliable and rational?

Tests and/or test items can sometimes be expensive, unique, or only performed in a few laboratories. There can be cases where assigned values are unknown, there is no information, or only poor information on the probability density function attributed to the test result. Sometimes there are neither reference materials nor the ability to establish consensus values due to a lack of experts. It can be impossible to repeat a test on the same item because it is destroyed during the test itself, or the homogeneity of tested items is unknown and no criteria can be established. Specified technical requirements concerning proficiency testing and interlaboratory comparison schemes are generally not applicable in this situation. However, interlaboratory comparison could allow laboratories to have more confidence in their results. The present paper discusses three statistical methods of assessing interlaboratory comparison results obtained in such conditions. Two methods are based on an assigned value determined from participant results through robust analysis. The third is based on the compatibility of results assessed using the ζ parameter. This paper focuses on an interlaboratory comparison for two laboratories, each testing three samples. The use of statistical methods turns out to be high risk, particularly in terms of falsely accepting results. Additionally, is shown that methods dedicated to small samples are also not efficient in detecting discrepancies of test results.     

A convenient and economic approach to achieve SI-traceable reference values to be used in drinking-water interlaboratory comparisons

Metrologically traceable reference values add an essential benefit to interlaboratory comparisons: unlike consensus values, they can be used to establish national and international comparability. Furthermore, the participating laboratories obtain a reliable and unbiased benchmark to check their results for accuracy. Usually, metrologically traceable reference values are obtained by so-called primary methods which demand excessive efforts at great expense. Within the framework of two national drinking-water interlaboratory comparisons (proficiency testing rounds), a new approach to provide metrologically traceable reference values was applied. It is solely based on existing data which were collected during the comparison itself. Lead (Pb) measurements serve as an example to show how metrologically traceable reference values were derived from the lead amount added during sample preparation and the amount of lead already present in the drinking-water matrix used to prepare these samples. Within this approach, the matrix content is calculated in a way similar to a standard addition experiment. An uncertainty budget for the reference value was set up which describes the link to the corresponding SI units. Isotope dilution mass spectrometry (IDMS) as a primary method was used to validate this approach in the case of cadmium, chromium, copper, lead, and nickel.     

The effect of the choice of method for determining assigned value on the assessment of performance in melting point analysis

The determination of melting point is a fundamental test in the Pharmaceutical industry, since it is one of the simplest techniques for the identification of a chemical substance. The melting point provides information on both identity and purity of a chemical substance and for that reason is a key test in the PHARMASSURE proficiency testing (PT) scheme. The PT scheme assesses participant??s determination of melting point, using chemicals of high purity and basing the assigned value on the robust consensus mean (median). In recent rounds, melting point reference standards have been provided as the test material and a reference value used as the assigned value for PT assessment. Comparison of the PT results over a number of rounds, using test materials with a wide range of melting points, shows the overall performance of the participant group is worse in rounds where a reference material and associated reference assigned value are used for performance assessment. When participants were assessed against the reference assigned value, a positive bias was observed in the participant??s results. Detailed information regarding the methodology used demonstrated that the majority of participants use the same analytical method, EU.Ph.2.2.14 (Council of Europe, Strasbourg, 2011), for the determination of melting point although this procedure allows flexibility in key methodological parameters, such as heating ramp rate, which may fail to ensure consistent performance across the group of participant laboratories.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

The BIOREMA project—part 3: International interlaboratory comparison for bio-ethanol test methods

The main objective of the reference materials for biofuel specifications (BIOREMA) project is the development of two test materials (one bio-ethanol material and one biodiesel material) with well-established reference values. Of a series of three papers, this part describes the material preparation, homogeneity study, stability study, and characterisation of the bio-ethanol material. The test material thus obtained was used in an interlaboratory comparison (ILC) to assess current practices and comparability amongst laboratories providing bio-ethanol testing services. Only 13 participants provided data, resulting in a small dataset for evaluation. Further, it appeared that for a number of laboratories, there was not sufficient material for the determination of all requested parameters. In most cases, as far as the data permit, it can be concluded that the consensus values (based on participant’s results) are in good agreement with the reference or the BIOREMA values (obtained by NMIs participating in the project). For three parameters, namely ethanol content, water content, and density, there is good agreement between the reference and consensus values. For these parameters, the reproducibility standard deviation is close to, or even smaller than, the expanded uncertainty associated with the reference value. A number of parameters show very poor reproducibility, for example, pH_e, electrolytic conductivity, and acidity. The same applies to sodium and copper content, which are very low and therefore challenging parameters to measure accurately. The results of the ILC underpin the need for certified reference materials and demonstrate the requirement for more robust quality control to improve the precision and trueness of the results from testing laboratories.     

水中易释放氰化物能力验证样品的研制及应用

根据国家地表水环境质量监测网监测任务要求和环境监测的实际需要,制备了水中易释放氰化物能力验证样品。通过均匀性、稳定性检验以及量值一致性评价,研制的样品均匀性良好,在3℃~6℃冷藏避光保存条件下1年内稳定,样品配制值与多家实验室协作测定结果一致。探讨了样品在能力验证活动中的应用,共有来自全国11个省的32家实验室参加了水中易释放氰化物的能力验证计划,实验室满意率在80%以上,实验室结果出现有问题或不满意主要是由于样品前处理以及检测过程质量控制不当导致的。经检测及实验室反馈的数据验证,该能力验证样品能够应用于能力验证活动。     

Requirements vs acceptability in proficiency testing schemes and other interlaboratory comparisons

Many laboratories take part in proficiency testing schemes, external quality assessment programmes and other interlaboratory comparisons. These have many similarities but also important differences in their modus operandi and evaluation of performance of participating laboratories. This paper attempts to highlight both the similarities and differences. It also puts particular emphasis on requirements called ”target values for uncertainty” and their meaning. Received: 24 January 2001 Accepted: 25 January 2001     

Comparability of analytical results obtained in proficiency testing based on a metrological approach

A “yes–no” type of criterion is proposed for the assessment of comparability of proficiency testing (PT) results when the PT scheme is based on a metrological approach, i.e. on the use of a reference material as the test sample, etc. The criterion tests a null hypothesis concerning the insignificance of a bias of the mean of the results from a traceable value certified in the reference material used for the PT. Reliability of such assessment is determined by the probabilities of not rejecting the null hypothesis when it is true, and rejecting it when it is false (the alternative hypothesis is true). It is shown that a number of chemical, metrological and statistical reasons should be taken into account for careful formulation of the hypotheses, enabling the avoidance of an erroneous assessment of the comparability. The criterion can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results.     

Definition of assigned values for proficiency tests in water analysis

The definition of an assigned value is usually achieved by calculating mean values from the data (with different methods) or by designating reference laboratories. Neither method is completely satisfactory. In this paper a new method is presented for the definition of the assigned value for spiked samples with an unknown content of the analyte in the matrix. The method consists of two parts. The first is the estimation of the assigned values from the spiked amounts and the content in the matrix, based on the results of reference laboratories. The other is the designation of these reference laboratories by comparing their results with the assigned values. Because each of these parts requires the other, an iterative procedure is necessary. As an example, the results of a proficiency test for the analysis of copper in wastewater are used to compare the calculated values with those from other methods, e.g., the Huber estimation. Received: 25 September 2000 Accepted: 9 December 2000     

Reference values versus consensus values: a reaction to the article of Baldan et al.

In a recent article Baldan et al. concluded that reference values can be used in a proficiency test as assigned values and that this does not increase the cost of the proficiency test. They feel that the values found during an homogeneity test could be used as reference values. In this article, it is argued that the conclusions of Baldan et al. have only a very limited validity. Due to the difference in goals, homogeneity data cannot be used for the determination of reference values without a drastic change in set-up. And in practice, in many fields of application, the situation is quite different than Baldan et al. suggest. Thus in many proficiency testing studies it is not technically possible to use reference values and in cases where it is possible, the costs will increase significantly if reference values are used instead of consensus values.     

The IAEA Proficiency Test on Evaluation of Methods for 90Sr Measurement in a Mineral Matrix; Preliminary Evaluation of Sources of Bias and Measurement Uncertainties

The results from numerous intercomparison exercises and proficiency tests indicate that the measurement of ⁹⁰Sr in solid environmental matrices poses a problem to many analysts. The causes of the observed scatter of analytical results are not well understood and therefore difficult to remedy. In order to assess the effect of various analytical operations and measurement routines on the quality of the ⁹⁰Sr data, the IAEA's Analytical Control Services have organised a proficiency test using a mineral sample spiked at three different levels with known amounts of ⁹⁰Sr. This proficiency test generated considerable interest from the radioanalytical community as a total of 192 sets of samples were distributed to 158 radioanalytical laboratories world-wide. The reported data were evaluated with respect to their relative bias against the reference value and with respect to their reported overall uncertainty. The major sources of bias leading to overestimated values are ineffective purification procedures, high background values and a lack of statistical control over background values. The major sources of bias leading to an underestimation are overestimated recovery factors in part due to failure to correct for stable Sr in the sample and possibly failure to correct for quenching in liquid scintillation counting. Preliminary results for a small randomly selected group of laboratories are presented.