Traceable reference values for routine drinking water proficiency testing: first experiences

Proficiency testing (PT) results have been used to improve traceability in chemical drinking water analysis. With a generalized least-square regression the mass concentrations of As and Sb were calculated in a drinking water that had been used to prepare proficiency testing samples by a spiking procedure. From the mass concentrations in the matrix and the spiked amounts, reference values with an uncertainty budget could be calculated without the need for reference measurements. The degree to which these reference values can be regarded as traceable is discussed. The results showed slight deviations in some samples between reference values and consensus means.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Comparability of analytical results obtained in proficiency testing based on a metrological approach

A “yes–no” type of criterion is proposed for the assessment of comparability of proficiency testing (PT) results when the PT scheme is based on a metrological approach, i.e. on the use of a reference material as the test sample, etc. The criterion tests a null hypothesis concerning the insignificance of a bias of the mean of the results from a traceable value certified in the reference material used for the PT. Reliability of such assessment is determined by the probabilities of not rejecting the null hypothesis when it is true, and rejecting it when it is false (the alternative hypothesis is true). It is shown that a number of chemical, metrological and statistical reasons should be taken into account for careful formulation of the hypotheses, enabling the avoidance of an erroneous assessment of the comparability. The criterion can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results.     

Designs of experiment for proficiency testing with a limited number of participants

Metrological designs of experiment for proficiency testing (PT) with a limited number of participants are discussed. The designs are based on development of in-house reference materials (IHRMs) with traceable assigned values and fit-for-purpose uncertainties, used in a PT scheme as the measurement standards. When adequate certified reference materials (CRMs) are available, a comparative approach for IHRM development, using simultaneous analysis of IHRM and CRM test portions in pairs, is proposed for PT objectives. In the case where adequate CRMs are not available – for example, in the field of analysis of unstable aqueous systems – the function of the measurement standard for PT can be fulfilled by a synthetic IHRM prepared gravimetrically using non-adequate, non-aqueous CRMs or pure substances. The CRM or the pure substance is used as a spike for fortification of a natural water sample, while the natural water sample is used as a working IHRM for the spike determination. In this case the traceability chain is longer, since two IHRMs are added for one quantity determination, but it remains unbroken.Presented at the Second International Conference on Metrology –Trends and Applications in Calibration and Testing Laboratories, 4–6 November 2003, Eilat, Israel     

Definition of assigned values for proficiency tests in water analysis

The definition of an assigned value is usually achieved by calculating mean values from the data (with different methods) or by designating reference laboratories. Neither method is completely satisfactory. In this paper a new method is presented for the definition of the assigned value for spiked samples with an unknown content of the analyte in the matrix. The method consists of two parts. The first is the estimation of the assigned values from the spiked amounts and the content in the matrix, based on the results of reference laboratories. The other is the designation of these reference laboratories by comparing their results with the assigned values. Because each of these parts requires the other, an iterative procedure is necessary. As an example, the results of a proficiency test for the analysis of copper in wastewater are used to compare the calculated values with those from other methods, e.g., the Huber estimation. Received: 25 September 2000 Accepted: 9 December 2000     

An interlaboratory study to improve the quality of chemical and biological measurements in waste water

 Analyses of waste water are routinely performed to monitor the level of contamination. To verify the quality of such determinations the National Institute of Chemistry, with the support of the Ministry of Environment and Spatial Planning and the Slovenian Accreditation Agency, organizes interlaboratory comparisons. Over the last 3 years, five interlaboratory trials named "MPP-Waste Water" were organized. Each round attracted around 50 participants, mostly from Slovenia and some from abroad, which enabled the testing of SIST ISO methods or alternative methods. We prepared samples for determination of harmful substances that are important for the characterization of waste water; physico-chemical parameters (pH), global parameters – chemical oxygen demand (COD), biochemical oxygen demand (BOD₅), metals (mercury, cadmium, copper, nickel, lead and chromium (VI)), nutrients (ammonia and total phosphorus), anions (chloride, nitrite, nitrate, sulphate) and toxicity to Daphnia magna. For the analysis of each parameter we prepared two samples at two different concentration levels. The materials used in the proficiency testing were carefully prepared and their homogeneity and stability were verified. The purpose of this scheme was to enable participants to check their day-to-day analytical performance. The results should enable the participants to improve the quality of their analyses.

---

---

Received: 24 October 2002 Accepted: 2 January 2003

---

Acknowledgments We would like to thank the Ministry of Environment and Spatial Planning and the Ministry for Education, Science and Sport for providing financial support. We would like to thank members of the Technical Committee: Mrs. Marjana Kovacˇicˇ, Dr. Katja Otrin-Debevc, Prof. Dr. Marjan Veber and Mrs. Boža Gregorc for their valuable support. Special thanks are due to Dr. Adrian Van der Veen who helped us in running the first PT.

---

Presented at CERMM-3, Central European Reference Materials and Measurements Conference: The function of reference materials in the measurement process, May 30–June 1, 2002, Rogaška Slatina, Slovenia

---

Correspondence to M. Cotman     

Accreditation of proficiency-testing schemes in The Netherlands

Proficiency testing by laboratories, national accreditation bodies, and other third parties is becoming more and more considered as a standard and integral part of the quality control system. Therefore, it is of utmost importance that the quality of the provided proficiency-testing (PT) service is outstanding. If PT-schemes are set up in order to help laboratories monitor and improve their quality, PT-schemes need not only be of high quality themselves, but the organizer also needs to be able to demonstrate this. In The Netherlands formal accreditation of the organization of proficiency testing schemes is used as a tool to guarantee high quality schemes and also to enable organizers to demonstrate their competence. Since 1996, the Dutch Council for Accreditation (RvA) has used the ISO-Guide 43–1 to assess PT-organizers in The Netherlands. From January 2000, the ISO-guide 43–1 was replaced by the ILAC G13 document for assessing organizers. Up till now, four institutes have been accredited by the RvA for the organization of PT-schemes.     

Comparability and compatibility of proficiency testing results in schemes with a limited number of participants

Comparability and compatibility of proficiency testing (PT) results are discussed for schemes with a limited number of participants (less than 20–30) based on the use of reference materials (RMs) as test items. Since PT results are a kind of measurement/analysis/test result, their comparability is a property conditioned by traceability to measurement standards applied in the measurement process. At the same time, metrological traceability of the certified value of the RM (sent to PT participants as test items) is also important, since the PT results are compared with the RM certified value. The RM position in the calibration hierarchy of measurement standards sets the degree of comparability for PT results, which can be assessed in the scheme. However, this assessment is influenced by commutability (adequacy or match) of the matrix RM used for PT and routine samples. Compatibility of PT results is a characteristic of the collective (group) performance of the laboratories participating in PT that can be expressed as closeness of the distribution of the PT results to the distribution of the RM data. Achieving quality-of-measurement/analysis/test results in the framework of the concept “tested once, accepted everywhere” requires both comparability and compatibility of the test results.     

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.     

Global comparability of analytical results

 Global decisions made on the basis of chemical analytical results need global comparability. The concept of traceability is important and necessary but not sufficient to achieve global comparability, especially for routine analysis in environmental protection, health care and public safety. Sometimes comparability is only determined by the method. In this case validated methods have to be known and recognized internationally. In addition, a minimum level of proficiency of the personnel producing analytical results has to be guaranteed. Finally, a system of measures designed to build up trust is needed for worldwide acceptance of analytical results. The method of self-declaration by the supplier of chemical analytical services has the advantage of presenting specific responsibility and gaining a good reputation. Additional methods should be employed. The evaluation of reference materials by specific quality criteria is mentioned as an example. Received: 1 July 1998 · Accepted: 20 July 1998     

Benefits of the implementation of a metrological structure for water analyses

Traceability of measurements still often remains a non-operational notion for end-user chemists. A practical project, sponsored by the French Ministry of Research, has been conducted to evaluate and to demonstrate possible benefits of the implementation of a metrological structure for improvement of the quality of water analyses. LNE (Laboratoire National d'Essais) was in charge of the build up of the traceability chain in a concrete case of determination of some heavy metals (cadmium and lead) in a groundwater. Pure solutions for calibration and a matrix RM have been certified by LNE and then used by 46 labs (mainly French) in a inter-laboratory study. Results have shown a measurable bias in lead analysis in the groundwater for all methods in routine use by laboratories. This project has demonstrated the interest of a metrological approach for method calibration, method validation and estimation of measurement uncertainty.     

Approaching target uncertainty in proficiency testing schemes: experience in the field of water measurement

Good correspondence between evaluations of the proficiency testing data for water measurements according to different scoring schemes was found as a result of implementation of the measurement quality according to the target uncertainty defined in the Drinking Water Directive. The accuracy of the evaluation is determined by the uncertainty of the assigned value, and this uncertainty should be in correspondence with the target uncertainty. Modification of the zeta-score by the thorough application of the target uncertainty is presented. It is convenient for generalised presentation of the proficiency testing data. The allowed bias is included as a linear term on the measured value scale, as it is treated in uncertainty analysis. The importance of implementation of the target measurement uncertainty in other fields of the routine measurements is indicated.     

Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.     

Measurement principles for traceability in chemical analysis

 By the definition of the mole as a base unit for amount-of-substance measures within the International System of Units (SI), chemists can make chemical measurements in full compliance with established metrological principles. Since the mole requires exact knowledge of the chemical entity, which is often neither available nor of practical relevance to the purpose of the measurement, the SI units of mass or length (for volume) are unavoidable in the expression of results of many chemical measurements. Science, technology, and trade depend upon a huge and ever increasing number and variety of chemical determinations to quantify material composition and quality. Thus, international harmonization in the assessments of processes, procedures, and results is highly desirable and clearly cost effective. The authors, with relevant experience and responsibilities in Europe and America, have found some consensus in the interpretation of the metrological principles for chemical measurements, but believe open discussion should precede wide implementation by chemical communities. In fostering this dialogue, this paper shows, for instance, that more precise interpretation of the definitions for "traceability," "calibration," and "validation" is needed for present-day chemical measurements. Problems that face scientists in making measurements do not all vanish just by adherence to the SI. However, such compliance can improve communication among chemists and metrologists. Received: 30 June 1995 Accepted: 30 June 1995     

The impact of metrological traceability on the validity of creatinine measurement as an index of renal function

When the calibration of a routine measurement procedure is traced back to metrological higher order, a significant discrepancy can occur between the analytical conditions of the routine measurement and the analytical conditions that were used in the clinical studies upon which the decision-making criteria are based. This can lead to serious interpretation errors with possible dramatic consequences for patients. The calibration of the creatinine Jaffé method is an excellent example of the importance of medical traceability. The compensated Jaffé method correlated accurately with the reference method and the compensated Jaffé creatinine clearance (CrCl), Cockroft and Gault and MDRD with the ⁵¹Cr EDTA clearance. The Schwartz estimate based upon the compensated Jaffé and enzymatic method overestimated, while uncompensated Jaffé slightly underestimated glomerular filtration rate (GFR). The situation in children is complex since serum creatinine concentrations are much lower in infants, rendering tubular secretion relatively more important. Low-molecular weight proteins have been suggested to replace serum creatinine as a marker for GFR. -trace protein, cystatin C, and ₂-microglobulin showed good correlation with GFR. However, care should be taken in patients presenting with some malignant tumors, since significant increases of cystatin C in patients with metastatic melanoma or colorectal cancer has been reported.Presented at the 9th Conference on Quality in the Spotlight, 18-19 March 2004, Antwerp, Belgium     

Challenges in preparing water-matrix reference materials for PAHs and pesticides: examples from SWIFT-WFD proficiency-testing schemes

In accordance with Article 8 of the European Union (EU) Water Framework Directive (WFD), EU Member States are required to establish monitoring programs for the quality of the surface water and the groundwater within each river-basin district. As such data are the basis for regulatory decisions and measures required to achieve WFD environmental objectives, appropriate analytical quality-assurance and quality-control tools have to be implemented by the monitoring laboratories. In this respect, reference materials (RMs) play a key role. Within the framework of the SWIFT-WFD project (Screening methods for Water data InFormation in support of the WFD), several approaches to the preparation of matrix RMs for the analysis of polycyclic aromatic hydrocarbons and pesticides in water have been used in wide-scale proficiency-testing (PT) schemes. We present the different strategies employed in preparing water-matrix RMs for organic analytes. By reviewing the results from the SWIFT-WFD PT schemes, we reflect on the applicability and the suitability of the different approaches.     

Nonparametric assessment of comparability of analytical results obtained in proficiency testing based on a metrological approach

A nonparametric sign test is implemented for assessment of comparability of proficiency testing (PT) results when their distribution differs from the normal or other known distribution. It allows testing the null hypothesis about insignificance of the bias of median of results obtained in PT from the traceable certified value of the reference material used in PT as test items, i.e., the hypothesis stating that comparability of the PT results is successful. Probability of type I error of rejecting the null hypothesis when it is true, and probability of type II error of it not rejecting when it is false (the alternative hypothesis about unsuccessful comparability is true) are considered. The test can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results when the comparability criterion developed for a normal results distribution (Accred. Qual. Assur. 10:466–470) is not applicable.     

Uncertainty evaluation in proficiency testing: state-of-the-art, challenges, and perspectives

The evaluation of measurement uncertainty, and that of uncertainty statements of participating laboratories will be a challenge to be met in the coming years. The publication of ISO 17025 has led to the situation that testing laboratories should, to a certain extent, meet the same requirements regarding measurement uncertainty and traceability. As a consequence, proficiency test organizers should deal with the issues measurement uncertainty and traceability as well. Two common statistical models used in proficiency testing are revisited to explore the options to include the evaluation of the measurement uncertainty of the PTRV (proficiency test reference value). Furthermore, the use of this PTRV and its uncertainty estimate for assessing the uncertainty statements of the participants for the two models will be discussed. It is concluded that in analogy to Key Comparisons it is feasible to implement proficiency tests in such a way, that the new requirements can be met. Received: 29 September 2000 Accepted: 3 December 2000