A new approach for proficiency test exercise data evaluation

In this paper, a new data evaluation method for proficiency test exercises consisting of a combination of a z-test, a zeta test and an uncertainty outlier test is presented. This new method is compared with eight other evaluation methods (both measurement uncertainty using and measurement uncertainty ignoring) in common use and/or recommended by ISO 13528. The data set used to test the evaluation methods is real data and consists of the ⁹⁵Nb results of the National Physical Laboratory Environmental Radioactivity Proficiency Test Exercise 2007. The evaluation of 14 out of 32 results were affected by the choice of method.     

EQRAIN: uranium and plutonium interlaboratory exercises from 1997 to 2016—comparison to ITVs-2010

Since 1987, the CEA’s Committee for the establishment of analysis methods (CETAMA) has regularly implemented interlaboratory comparisons, entitled “evaluation of the quality results of analysis in the nuclear industry” (EQRAIN). Notably, the EQRAIN U and EQRAIN Pu interlaboratory comparisons assess proficiency in measuring a mass content of uranium or plutonium in reference solutions. This paper presents the results of measurement uncertainty assessments from EQRAIN U and EQRAIN Pu comparisons over 20 years of exercises (1997–2016). The mathematical approach developed in this work allowed to estimate the impact of short-term systematic and random errors to the overall uncertainty of each analytical method used in the interlaboratory comparison program. This statistical analysis shows a good consistency between measurement uncertainty values from EQRAINs and the measurement uncertainty target values established by the International Atomic Energy Agency for nuclear material balances (ITVs-2010).

     

Proficiency testing in the light of a new rationale in metrology

The novel proposed definition of measurement result in the international metrology vocabulary requires a revision of standards and guidelines for proficiency testing (PT), and a new approach to processing proficiency data is needed to test the ability of laboratories to present not only unbiased quantity values, but reliable estimates of their uncertainty. Hence, an accepted reference value with the smallest possible uncertainty is needed to ascertain the proficiency of laboratories reporting results with lower than average uncertainty. A strategy based on the T-statistic is proposed leading to an accepted reference value that fully reflects the uncertainties reported by participants in a PT scheme and permits calculation of E _n numbers to distinguish whether or not measurement results are consistent with the accepted definition of the measurand. The strategy is applied to PT data from a recent international laboratory intercomparison of uranium isotopic ratios.     

Proficiency testing scheme for the harmonization and comparability of analytical measurements

According to the experience of the successful implementation of proficiency tests (PT) by using the certified reference value as the assigned value, a new scheme of evaluation is presented by suggesting the use of the uncertainty associated with the certified value. The technical performance of laboratories is evaluated by the parameter quadratic mean error (QME), which is the square root of the sum of the square of the bias and that of the standard deviation of the laboratory. This parameter is considered as the estimate of the measurement uncertainty of the laboratory and is compared to the uncertainty (U) associated with the certified value provided by an NMI. Considering that the calibration and measurement capability, known as the CMC, is recognized among NMIs, the ratio QME/U enables us to compare the PT relative to the CMC of an NMI, and, consequently, to any other comparison results based on the CMC of signatories of the mutual recognition arrangement (MRA) of the International Committee of Weights and Measure (CIPM). Presented at BERM-11, October 2007, Tsukuba, Japan.     

Quality issues in proficiency testing

 Proficiency testing is a means of assessing the ability of laboratories to competently perform specific tests and/or measurements. It supplements a laboratory's own internal quality control procedure by providing an additional external audit of their testing capability and provides laboratories with a sound basis for continuous improvement. It is also a means towards achieving comparability of measurement between laboratories. Participation is one of the few ways in which a laboratory can compare its performance with that of other laboratories. Good performance in proficiency testing schemes provides independent evidence and hence reassurance to the laboratory and its clients that its procedures, test methods and other laboratory operations are under control. For test results to have any credibility, they must be traceable to a standard of measurement, preferably in terms of SI units, and must be accompanied by a statement of uncertainty. Analytical chemists are coming to realise that this is just as true in their field as it is for physical measurements, and applies equally to proficiency testing results and laboratory test reports. Recent approaches toward ensuring the quality and comparability of proficiency testing schemes and the means of evaluating proficiency test results are described. These have led to the drafting of guidelines and subsequently to the development of international requirements for the competence of scheme providers. Received: 2 January 1999 · Accepted: 7 April 1999     

Use of uncertainty estimates as reported by participants in proficiency testing for the evaluation of their results: pros and cons

The aim of this paper is to study the measurement uncertainties reported in proficiency tests (PTs) using examples from PTs in the environmental sector and to compare the obtained measurement uncertainty estimates using different approaches. In addition, the paper focusses on the differences between the z-score and the zeta score. Since the year 2000, the Finnish Environment Institute has asked participants to report analytical methods as well as measurement uncertainties in connection with PT results. The measurement uncertainties of the assigned value have also been evaluated. On the basis of the results, the measurement uncertainties reported by the participants varied greatly. Participants often reported underestimated measurement uncertainties, but overestimated uncertainties were also reported. At the moment, it seems as if performance assessment should be based on the z-score because of a number of significant over- and underestimated measurement uncertainties. The zeta score should be used for information and educational purposes mainly.     

Proficiency testing for measurement of radon (222Rn) in drinking water

The Finnish Environment Institute in collaboration with the Radiation and nuclear safety authority (STUK) carried out the proficiency test for measurement of radon (²²²Rn) in water. Samples were taken from two drilled wells in November 2007. STUK has supplied the regional laboratories with RADEK MKGB-01 equipment based on gamma spectrometry. Two samples for this PT were taken from two drilled wells. Ground water moves irregularly in the process and cracks of the bedrock which is why each participant received an individual sample. Each participant’s sample was measured also by STUK using liquid scintillation counter (LSC) as the reference method in this proficiency test. In estimating laboratory performance the results that deviated less than ±10% from the value measured by STUK using LSC were regarded satisfactory. In total, 73% of the results in the analysis of the sample R1 and 82% in the analysis of the sample R2 deviated less than 10% from the values measured by STUK. The results reported by the participants were generally smaller than the results measured by STUK. The deviation between each participant’s result and the result measured by STUK with the LSC was ?7.4% (Sample R1) and ?6.2% (Sample R2). Due to the lack of certified reference materials and a reliable proficiency testing data, it is impossible to check the traceability of radon measurements by using the reference method (LSC) at this moment.     

Validation according to European Commission Decision 2002/657/EC of a confirmatory method for aflatoxin M1 in milk based on immunoaffinity columns and high performance liquid chromatography with fluorescence detection

A high performance liquid chromatographic method with fluorimetric detection for the determination of aflatoxin M₁ (AFM₁) in milk has been optimized and validated according to Commission Decision 2002/657/EC by using the conventional validation approach. The procedure for determining selectivity, recovery, precision, decision limit (CC_α), detection capability (CC_β) and ruggedness of the method has been reported. The results of the validation process demonstrate the agreement of the method with the provisions of Commission Regulation 401/2006/EC. The mean recovery calculated at three levels of fortification (0.5, 1.0, and 1.5-fold the MRL) was 91% and the maximum relative standard deviation value for the within-laboratory reproducibility was 15%. Limit of detection (LOD) and limit of quantitation (LOQ) values were 0.006 μg kg⁻¹ and 0.015 μg kg⁻¹ while the CC_α and CC_β values were 0.058 μg kg⁻¹ and 0.065 μg kg⁻¹, respectively. The relative expanded measurement uncertainty of the method was 7%. The method was not affected by slight variations of some critical factors (ruggedness minor changes) as pre-treatment and clean-up of milk samples, thermal treatment and different storage conditions, as well as by major changes valued in terms of milk produced by different species (buffalo, goat and sheep). The method allowed accurate confirmation analyses of milk samples, resulted positive by the screening method. In fact, the Z-score values attained in a proficiency test round were well below the reference value of 1, proving the excellent laboratory performances.     

Establishment of an assigned value and its uncertainty for tumour markers in proficiency testing in China

Proficiency testing (PT) is an important aspect of clinical laboratory quality control for tumour markers. To evaluate the performance of an individual laboratory, the assigned value and the standard uncertainty of each tumour marker must be determined. However, no data about the standard uncertainty of the assigned value of any analyte in PT is available in China. The aim of this paper is to introduce a new approach for establishment of assigned value and its standard uncertainty for tumour markers. Five sets of control materials at different concentrations were assigned to each laboratory that participated in PT for tumour markers in 2012. We collected the data from each laboratory using the Clinet-EQA reporting system V1.5 and Clinet-EQA evaluation system V1.0 and developed a software program to calculate the assigned value and its uncertainty according to ISO 13528 (Statistical methods for use in proficiency testing by interlaboratory comparisons. ISO 13528. International Standards Organisation, Geneva, 2005). The assigned value and inter-laboratory CV calculated by the new approach was compared with that obtained by the current approach used in PT in our country. No significant difference between the two outcomes was found in our study. Most assigned values had a standard uncertainty that can meet the criterion: $ u_{X} \le \, 0.3\hat{\sigma } $ u X ≤ 0.3 σ ? , which meant the new approach for establishment of the assigned value was acceptable. It is expected that the new approach to determination of the assigned value and its standard uncertainty for PT for tumour markers in China will enable participants to evaluate their measurement uncertainty.     

Performance of HPGe gamma spectrometry system for the measurement of low level radioactivity

Comprehensive quality assurance/quality control procedure is very much necessary to obtain accurate and precise analytical measurement result. This paper discusses the quality control aspects of the High-Purity Germanium (HPGe) based gamma spectrometry system, which has been used for the measurement of low-level radioactivity in environmental samples. The gamma spectrometry system consisting of coaxial n-type HPGe detector having 50% relative efficiency with respect to 7.62 cm x 7.62 cm NaI (Tl), Nuclear Instrumentation Module (NIM) based pulse processing electronic accessories and 8 k MCA. To reduce the background contribution, 7.5 cm thick lead has been placed surrounding the detector. The minimum detectable activities (MDA) with 95% confidence level (for 300 g soil sample and 100,000 s counting time) for important radionuclides such as ²³⁸U, ²²⁶Ra, ²³²Th, ⁴⁰K, and ¹³⁷Cs are 10.4, 4.3, 4.1, 16.9 and 0.1 Bq kg^?1, respectively. The Quality control (X bar R) charts were plotted using ¹³⁷Cs and ⁴⁰K background counts observed periodically, which showed that the fluctuation is well within the confidence limit and confirms the stability of the system. The laboratory has been participating in the proficiency tests (PTs) of the International Atomic Energy Agency (IAEA). In recently concluded PTs, the samples include soil, spiked standard solution, spinach, phosphogypsum and spiked air filter were analysed for the natural, fission and activation products radionuclides. The performance evaluation of the IAEA PTs showed that the laboratory results were in good agreement with the target value, which confirms the reliability and traceability of the gamma spectrometric measurement result of the laboratory.     

The IAEA Proficiency Test on Evaluation of Methods for 90Sr Measurement in a Mineral Matrix; Preliminary Evaluation of Sources of Bias and Measurement Uncertainties

The results from numerous intercomparison exercises and proficiency tests indicate that the measurement of ⁹⁰Sr in solid environmental matrices poses a problem to many analysts. The causes of the observed scatter of analytical results are not well understood and therefore difficult to remedy. In order to assess the effect of various analytical operations and measurement routines on the quality of the ⁹⁰Sr data, the IAEA's Analytical Control Services have organised a proficiency test using a mineral sample spiked at three different levels with known amounts of ⁹⁰Sr. This proficiency test generated considerable interest from the radioanalytical community as a total of 192 sets of samples were distributed to 158 radioanalytical laboratories world-wide. The reported data were evaluated with respect to their relative bias against the reference value and with respect to their reported overall uncertainty. The major sources of bias leading to overestimated values are ineffective purification procedures, high background values and a lack of statistical control over background values. The major sources of bias leading to an underestimation are overestimated recovery factors in part due to failure to correct for stable Sr in the sample and possibly failure to correct for quenching in liquid scintillation counting. Preliminary results for a small randomly selected group of laboratories are presented.     

Measurement uncertainty of food carotenoid determination

For consistent interpretation of an analytical method result it is necessary to evaluate the confidence that can be placed in it, in the form of a measurement uncertainty estimate. The Guide to the expression of Uncertainty in Measurement issued by ISO establishes rules for evaluating and expressing uncertainty. Carotenoid determination in food is a complex analytical process involving several mass transfer steps (extraction, evaporation, saponification, etc.), making difficult the application of these guidelines. The ISO guide was interpreted for analytical chemistry by EURACHEM, which includes the possibility of using intra- and interlaboratory information. Measurement uncertainty was estimated based on laboratory validation data, including precision and method performance studies, and also, based on laboratory participation in proficiency tests. These methods of uncertainty estimation were applied to analytical results of different food matrices of fruits and vegetables. Measurement uncertainty of food carotenoid determination was 10–30% of the composition value in the great majority of cases. Higher values were found for measurements near instrumental quantification limits (e.g. 75% for β-cryptoxanthin, and 99% for lutein, in pear) or when sample chromatograms presented interferences with the analyte peak (e.g. 44% for α-carotene in orange). Lower relative expanded measurement uncertainty values (3–13%) were obtained for food matrices/analytes not requiring the saponification step. Based on these results, the saponification step should be avoided if food carotenoids are not present in the ester form. Food carotenoid content should be expressed taking into account the measurement uncertainty; therefore the maximum number of significant figures of a result should be 2.     

Reliability of food measurements: the application of proficiency testing to GMO analysis

This paper reviews the experience of the Food Analysis Performance Assessment Scheme (FAPAS^®) in operating a proficiency testing scheme for the analysis of genetically modified (GM) food. Initial rounds of proficiency testing have shown a tendency for laboratories to over-estimate GM levels, results obtained by polymerase chain reaction (PCR) and enzyme-linked immunosorbent assay (ELISA) detection methods to be significantly different and that data are skewed and not normally distributed until log-transformed. During the initial rounds, it was found that for analysis and quantification of GM material, it was not possible to assign a target value for standard deviation external to the round data, from which performance could be assessed. However, when working in a log scale, the internally derived, robust standard deviation (^σ) was found to be constant and could be used directly to predict a target value (σ) for performance assessment. Results from the first four rounds have provided valuable information and a general overview of laboratory ability. Choosing a target value for standard deviation which reflects current best practice has enabled laboratory performance to be assessed. Issues surrounding the assessment of performance are discussed which highlight some of the implications raised as a result of this initial assessment, regarding the enforcement of European labelling legislation.     

Uncertainty estimation of anions and cations measured by ion chromatography in fine urban ambient particles (PM2.5)

The present work presents a measurement uncertainty evaluation according to Guide to the Expression of Uncertainty in Measurement (GUM) of the concentration of the cations K⁺ and Li⁺ and anions NO₃⁻² and SO₄⁻² in fine airborne particulate matter, refers to particles less than 2.5 μm in diameter (PM_2.5), as measured by ion chromatography (US-EPA 300 method). The GUM method is not typically used to report uncertainty. In general, the analytical results only report the measurement’s standard deviation under repetition as an uncertainty; thus, not all sources of uncertainty are considered. In this work, the major sources of uncertainty regarding the measurements were identified as contributions to linear least square regression lines, repeatability, precision, and trueness. The expanded uncertainty was approximately 20% for anions and cations. The largest contribution to uncertainty was found to be repeatability.     

Evaluation of proficiency testing results taking into account their uncertainties

A new composite score for the evaluation of performance of proficiency testing participants is proposed. The score is based on a combination of the z-score, uncertainty of a participant’s measurement result and uncertainty of the proficiency testing scheme’s assigned value. The use of such a composite score will allow evaluation not only of the participant’s ability to determine an analyte in corresponding matrix, but also their understanding of the uncertainty in the obtained analytical result. The score may be helpful for the laboratory’s quality system and for laboratory accreditation according to ISO 17025.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

Achieving traceability in chemical measurement—a metrological approach to proficiency testing

ISO/IEC 17025 requires that testing laboratories establish the traceability of their measurements, preferably to the SI units of measurement. The responsibility for establishing traceability lies with each individual laboratory and must be achieved by following a metrological approach.The results of measurements made in such a way are traceable to the standards used in method validation and to the calibration standards used during the measurement process. If these standards are traceable to SI then the measurements will also be traceable to SI.Participation in appropriate proficiency studies (an ISO/IEC 17025 requirement) enables laboratories to demonstrate the comparability of their measurements. If the materials used for the studies have traceable assigned values, then proficiency testing also provides information about measurement accuracy and confirms, or otherwise, that appropriate traceability has been established. This paper will report on a new approach for the establishment of traceable assigned values for chemical testing proficiency studies. The work is conducted at a "fit for purpose" level of measurement uncertainty, with costs contained at a level similar to previous "consensus" based proficiency studies. By establishing traceable assigned values in a cost effective way, NARL aims to demonstrate the added value of the metrological approach to participant laboratories.     

Determination of 226Ra in solid samples of few milligrams after mineralisation and measurement by solid scintillation

A protocol is proposed for the measurement of ²²⁶Ra on 150 mg of solid sample without radiochemistry. To evaluate the performance of this method, standard samples were used and the results were in good agreement with reference values. The detection limit obtained is about 130 Bq kg⁻¹ (dry weight) without mineralised solution concentration by evaporation. A concentration of the solution by 4 and/or an increase of sample mass by 4 in the case of microwave digestion system, allows achieving a detection limit of 30 Bq kg⁻¹ (dry weight) and thus measuring ²²⁶Ra in most soils. This method could also be used for NORM sites on soil and sediment samples.

     

Uncertainty in aflatoxin B<Subscript>1</Subscript> analysis using information from proficiency tests

We describe how to calculate uncertainty in the determination of aflatoxin B₁ in nuts and maize using an HPLC method with prederivatisation with trifluoroacetic acid and fluorescence detection. The uncertainty is easily calculated using the information obtained from the participation in several proficiency tests. All the sources of uncertainty are grouped in two terms: (1) matrix variability, intermediate precision and sample heterogeneity, and (2) method concordance. This methodology has the advantage that no extra work needs to be done because all the information to calculate uncertainty comes from proficiency tests. The uncertainty values were calculated using samples whose concentration ranged between 2 and 20 g L^–1. The relative standard uncertainty computed for aflatoxin B₁ was 16.3%.