Key metrological issues in proficiency testing––response to “Metrological compatibility-a key issue in further accreditation” by K. Heydorn

A discussion of proficiency testing (PT) topics started by Heydorn (Accred Qual Assur 15:643–645, 2010) is continued in the present paper. The role of PT in the accreditation of testing/analytical laboratories, the use of consensus values (average or weighted average, median, observed standard deviation, etc.) and a metrological background of PT schemes are discussed. It is shown that metrological traceability, comparability, and compatibility, as well as commutability of a reference material, are the key issues of any PT scheme that applies certified reference material as test items. Metrological compatibility of PT results in such schemes is a property demonstrating the closeness of the PT results to the certified value in comparison with the measurement uncertainty of their difference. The metrological background is especially important for the selection and use of PT schemes for a limited number of participants (fewer than 30) as detailed in IUPAC/CITAC Guide on the topic published in 2010 in Pure Appl Chem 82(5):1099–1135.     

Nonparametric assessment of comparability of analytical results obtained in proficiency testing based on a metrological approach

A nonparametric sign test is implemented for assessment of comparability of proficiency testing (PT) results when their distribution differs from the normal or other known distribution. It allows testing the null hypothesis about insignificance of the bias of median of results obtained in PT from the traceable certified value of the reference material used in PT as test items, i.e., the hypothesis stating that comparability of the PT results is successful. Probability of type I error of rejecting the null hypothesis when it is true, and probability of type II error of it not rejecting when it is false (the alternative hypothesis about unsuccessful comparability is true) are considered. The test can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results when the comparability criterion developed for a normal results distribution (Accred. Qual. Assur. 10:466–470) is not applicable.     

Local comparability of proficiency testing results: determination of concrete slump and compressive strength values

A proficiency testing (PT) scheme is developed for comparability assessment of results of concrete slump and compressive strength determination. The scheme is based on preparing of a test portion/sample of a concrete in-house reference material (IHRM) at a reference laboratory (RL) in the same conditions for every PT participant. Therefore, in this scheme IHRM instability is not relevant as a source of measurement/test uncertainty, while intra- and between-samples inhomogeneity parameters are evaluated using the results of RL testing of the samples taken at the beginning, the middle and the end of the PT experiment. The IHRM assigned slump and compressive strength values are calculated as averaged RL results. Their uncertainties include the measurement/test uncertainty components and the components arising from the material inhomogeneity. The test results of 25 PT participants were compared with the IHRM assigned values taking into account both the uncertainties of the assigned values and the measurement/test uncertainties of the participants. Since traceability of the IHRM assigned values to the international measurement standards and SI units cannot be stated, local comparability of the results is assessed. It is shown, that comparability of the slump and compressive strength determination results is satisfactory, while uncertainty evaluation for slump results requires additional efforts.     

Proficiency testing of pH and electrolytic conductivity measurements of groundwater: a case study of the difference between consensus and metrologically traceable values

A proficiency testing (PT) scheme was designed for measurements of pH and electrolytic conductivity (EC) of groundwater in Israel. A groundwater sample was used for preparation of test items fit-for-intended-use. The interlaboratory experiment was organized taking into account changes of the test item properties which occurred after sampling due to presence of CO₂ in the groundwater. A total of 34 PT participants reported pH measurements results, while 29 of them reported results of EC measurements. The pH results were normally distributed. However, a significant bias of the pH consensus value of the PT participants from the metrologically traceable certified value (obtained by the National Physical Laboratory of Israel) was detected. This bias was caused by the drift of the measurement response due to water stirring and CO₂ degassing from the sample during the measurement process. The bias of the EC consensus value from the metrologically traceable one was negligible, in spite of an abnormal distribution of the EC results of the PT participants.     

Development of an obidoxime chloride reference material: a metrological approach to the determination of chromatographic purity

A metrological approach to determination of the chromatographic purity of obidoxime chloride and the corresponding obidoxime chloride reference material (RM) with a certified chromatographic purity value have been developed. This value was defined as the ratio of the sum of peak areas of obidoxime chloride isomers to the total peak area of detected substances including impurities (%) under specified HPLC–UV conditions. The RM homogeneity and stability were studied using HPLC with UV detection and evaluated as satisfactory. The certified value calculated from the results of an interlaboratory trial was equal to 99.9% with the expanded uncertainty of 0.6% at the level of confidence 0.95 and the coverage factor 2. The RM certified value, like other results of chromatographic purity determination traceable to the reference measurement procedure, is not traceable directly to the SI mole. However, the results are comparable in metrologically traceable environments, i.e. when relevant measuring laboratory instruments are calibrated with traceability chains to the corresponding SI units. Therefore, the RM can be used as a measurement standard (calibrator) for analytical instruments and as a control sample for quality control of HPLC obidoxime chloride assay results.     

Comparability of analytical results obtained in proficiency testing based on a metrological approach

A “yes–no” type of criterion is proposed for the assessment of comparability of proficiency testing (PT) results when the PT scheme is based on a metrological approach, i.e. on the use of a reference material as the test sample, etc. The criterion tests a null hypothesis concerning the insignificance of a bias of the mean of the results from a traceable value certified in the reference material used for the PT. Reliability of such assessment is determined by the probabilities of not rejecting the null hypothesis when it is true, and rejecting it when it is false (the alternative hypothesis is true). It is shown that a number of chemical, metrological and statistical reasons should be taken into account for careful formulation of the hypotheses, enabling the avoidance of an erroneous assessment of the comparability. The criterion can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results.     

Creatinine determination in urine from the point of view of reference values

In this paper, the creation of a certified reference material for urinary creatinine is described. We used the Jaffe method and HPLC method for establishment of the certified value. Homogeneity tests are also described. We obtained material with sufficient homogeneity, stability, and with certified value (expanded uncertainty, k=2 for CI 95%) (7.77±0.27) mmol·L⁻¹. This material was consequently used for the interlaboratory comparison (EQA Czech Republic for clinical chemistry). Twenty-nine percent of the participants obtained measurement results within the interval of the certified value ± expanded uncertainty, while 85% of the participants obtained values inside the interval of the certified value ± target measurement uncertainty. Direct use of the certified reference materials for method evaluation in EQA programs means a significant advance for monitoring and documentation traceability of results in routine measurements.     

Development of a proficiency testing scheme for a limited number of participants in the field of natural water analysis

A proficiency testing (PT) scheme was developed for a limited number of analytical laboratories participating in the analysis of natural water in Israel. Three fit-for-intended-use reference materials (RMs) were prepared for a pilot PT from natural water: RM-blank and two in-house reference materials (IHRMs) with seven analytes fortified to achieve different levels of analyte concentrations. The measurands for the PT participants were the traceable spike values certified in the IHRMs, i.e., added mass concentrations of the analytes. The RM-blank and IHRMs were found to be homogeneous and stable over 11 days, the time necessary for the experimental part of the PT. The RMs were distributed to the PT participants as unknown test items similar to routine samples. The test results were evaluated using their deviations from the IHRM-certified values. Eight Israeli laboratories took part in the interlaboratory comparison. Individual laboratory performance and metrological compatibility of the PT results of the participants, as a local group of laboratories, were evaluated for every analyte.     

Traceability without uncertainty: current situation in the pharmaceutical industry

The current situation in the pharmaceutical industry is discussed, when the traceability of measurement (analytical) results to certified values of pharmacopoeial reference standards is required, without evaluating their uncertainties. It is shown that the evaluation of measurement uncertainty is necessary for understanding the level of confidence of the analytical results and their comparability, particularly during preparation and characterisation of the reference standards.Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher. Apart from exceptional circumstances, they are not submitted to the usual referee procedure and go essentially unaltered.     

PT/EQA standards and guidelines: some remarks on quality and reliability of test items

Current standards and international guides on PTs/EQAs consider the achievement of the quality and reliability of PT test items as important steps. Similarity to the materials routinely analysed and sufficient homogeneity and stability of the properties of interest are the main quality characteristics required. PT test items are carriers of the property values used as reference to evaluate laboratory performances. Requirements for PT test items are practically the same as those given for reference materials. ISO-REMCO defines reference material as “Material, sufficiently homogeneous and stable with respect to one or more specified properties, which has been established to be, fit for its intended use in a measurement processes”. In the case of PT test items, they should have a degree of homogeneity and stability to be fit for the purpose of identifying deviations among laboratories. This paper reports some remarks on the main characteristics of PT test items, using ISPRA experience as RM producer.     

Designs of experiment for proficiency testing with a limited number of participants

Metrological designs of experiment for proficiency testing (PT) with a limited number of participants are discussed. The designs are based on development of in-house reference materials (IHRMs) with traceable assigned values and fit-for-purpose uncertainties, used in a PT scheme as the measurement standards. When adequate certified reference materials (CRMs) are available, a comparative approach for IHRM development, using simultaneous analysis of IHRM and CRM test portions in pairs, is proposed for PT objectives. In the case where adequate CRMs are not available – for example, in the field of analysis of unstable aqueous systems – the function of the measurement standard for PT can be fulfilled by a synthetic IHRM prepared gravimetrically using non-adequate, non-aqueous CRMs or pure substances. The CRM or the pure substance is used as a spike for fortification of a natural water sample, while the natural water sample is used as a working IHRM for the spike determination. In this case the traceability chain is longer, since two IHRMs are added for one quantity determination, but it remains unbroken.Presented at the Second International Conference on Metrology –Trends and Applications in Calibration and Testing Laboratories, 4–6 November 2003, Eilat, Israel     

Implementation of proficiency testing schemes for a limited number of participants

A metrological background for the selection and use of proficiency testing (PT) schemes for a limited number N of laboratories-participants (less than 20–30) is discussed. The following basic scenarios are taken into account: (1) adequate matrix certified reference materials (CRM) or in-house reference materials (IHRM) with traceable property values are available for PT use as test items; (2) no appropriate matrix CRM is available, but a CRM or IHRM with traceable property values can be applied as a spike or similar; (3) only an IHRM with limited traceability is available. The discussion also considers the effect of a limited population of PT participants N _p on statistical assessment of the PT results for a given sample of N responses from this population. When N _p is finite and the sample fraction N/N _p is not negligible, a correction to the statistical parameters may be necessary. Scores suitable for laboratory performance assessment in such PT schemes are compared. Presented at the 3rd International Conference on Metrology, November 2006, Tel Aviv, Israel.     

The establishment of a quality evaluation system of reference materials

Reference materials have been applied widely to ensure the traceability, comparability and reliability of measurement results. To achieve this purpose, the quality of reference materials (RMs) themselves is surely an important aspect to be pay attention to. A quality evaluation system of RMs has been established through the project “The National Sharing Platform of Reference Materials” in China to give a reliable assessment on the quality of RMs from various sources including the accuracy and comparability of their property values, which is very useful to promote the appropriate selecting and using of RMs in China. Through the application of National Metrology Institute calibration and measurement capabilities on the basis of the international mutual recognition arrangement, it can also provide a powerful supplement to the current activities such as the accreditation of RM producers in the construction of a global harmonized quality control and assurance system of RMs.     

Performance evaluation for proficiency testing with a limited number of participants

Proficiency testing (PT) is an essential tool for laboratories to assess their competency. Also, participation in PT has become one of the mandatory requirements for laboratory to seek accreditation according to ISO/IEC 17025. For this reason, the effectiveness of performance evaluation by PT scheme is of great concern for the participants and for accreditation bodies as well. In practice, owing to unavailability of other appropriate alternatives, PT scheme providers may have to choose using consensus values to evaluate the performance of participants. However, such consensus values approach was not recommended by relevant international guidelines for PT schemes with limited number of participants. With the use of Monte Carlo simulation technique, this study attempted to investigate the effectiveness of using consensus values for performance evaluation in PT schemes with limited number of participants. The simulation process was schemed according to the statistical model provided by ISO 5725-1 for laboratory measurement results, which covered components like method bias, laboratory bias, and measurement precision. The effectiveness of the consensus value approach was expressed as the percentage of participants in a simulation run could get the same evaluation result, either satisfactory or unsatisfactory, against the “true value.” The findings indicated that the number of participants, choice of consensus values, mass fraction of analyte, method bias, laboratory bias, and measurement repeatability of participating laboratories would all affect the effectiveness of the consensus value approach but at different extent. However, under certain circumstances, use of consensus value could still be considered as an acceptable approach for performance evaluation even the number of participants was limited. Some of the findings were further verified using real data from PT schemes where appropriate certified reference materials or reliable reference values were available.     

A framework for the application and materials in analytical chemistry

 Working group 5 of EuraChem Nederland has developed a framework for the implementation of reference materials in analytical chemistry. In this discussed paper, the framework is proposed as a tool for the development of standard operation procedures (SOPs) in laboratories. The implementation of (certified) reference materials in these SOPs is of major importance in establishing comparability and traceability in measurement results, which in turn play a crucial role in measurement in support of trade, environmental issues, and characterisation of materials. Recent developments in the field of uncertainty analysis require the application of reference materials. It is recognised that the calculation of the combined measurement uncertainty becomes almost impossible without the use of certified reference materials with a stated uncertainty. Received: 1 December 1995 Accepted: 20 December 1995     

Determination of the purity of acidimetric standards by constant-current coulometry, and the intercomparison between CRMs

The accuracy and uncertainty of the coulometric measurement results of reference materials for acidimetric titration were examined in this study. The results for amidosulfuric acid and potassium hydrogen phthalate are presented. The uncertainty was investigated by examining the dependency on the sample size and on the electrolysis current. Changes in the titration parameters did not result in any significant effects on the titration results. Acidimetric standards with the certified value linked to the SI were developed. In addition, the intercomparison of acidimetric standards was carried out by gravimetric titration, and the relationship between our coulometric results was determined. Furthermore, due to recent internationalization, not only the traceability to the SI but also the relationship and consistency of their analytical data have gained increasing importance. Our results were validated using certified reference materials (CRMs) obtained from different National Metrology Institutes (NMIs), and their relationships are presented. Presented at -- “BERM-10” -- April 2006, Charleston, SC, USA.     

Traceability of (values carried by) reference materials

 Traceability is a property of the result of a measurement. Since values carried by (reference) materials must also have been obtained, of necessity, by measurement, the definition of traceability also applies to reference materials. It is extremely helpful to give the traceability (of the origin) of a reference material a separate name, i.e. 'trackability'. An analysis of the function of values carried by reference materials, shows that they can fulfill different functions, depending on the intended use. One of the functions located outside the traceability chain – and hence not very relevant for establishing traceability – is evaluating the approximate size of the uncertainty of the measurement of an unknown sample by performing a similar measurement on a reference material, used as a 'simulated sample'. Another function is located inside the traceability chain, where the reference material is used as an added 'internal standard'. Then, the value carried by the reference material is essential for establishing the traceability of the measured value of an unknown sample. In the latter application, the reference material acts as an 'amount standard' (the certified value for amount is used). Received: 11 November 1999 / Accepted: 24 February 2000     

Traceable property values of in-house reference materials

The traceability of in-house reference materials (IHRM) is discussed. It is shown that a systematic error in results of a measured value, specific to a measurement method or to a laboratory developing an IHRM, can be overcome if a comparative approach to IHRM characterization is used. A traceability chain of the value carried by the IHRM to the value carried by the reference material with higher metrological status and sufficiently similar matrix (for example, a certified reference material – CRM according to ISO Guide 30) is helpful in such a case. The chain is realized when the IHRM samples are analysed simultaneously with the CRM samples under the same conditions. This and other traceability chains necessary for the IHRM development are examined as the measurement information sources.     

Integration of metrological principles in a proficiency-testing scheme in the field of water analysis

Measurement traceability is universally recognised as one of the basic prerequisites for comparability of results obtained in different laboratories and is a basic aspect of metrological sciences such as analytical chemistry. This requirement is underscored by the increasing adoption of standards and measurement quality systems, such as laboratory accreditation against ISO/IEC 17025. Testing laboratories ensure traceability of their measurement results by using appropriate reference standards for calibration of instruments and control of measurement processes. For routine work in the field of water analysis, these standards are usually commercial solutions or in-house solutions prepared from pure products. Therefore, laboratories should demonstrate that their use of reference standards is appropriate and sufficient, which can be done by participation in an appropriate proficiency-testing scheme. The paper reports how measurement traceability of results from field laboratories (nitrite nitrogen, nitrate nitrogen, chloride and sulphate; all in water) can be demonstrated by participation in a proficiency-testing scheme based on reference values.