The BIOREMA project—Part 2: International interlaboratory comparison for biodiesel test methods

The results of an interlaboratory comparison, using various measurement methods to carry out biodiesel testing, are presented and the findings are discussed. The interlaboratory comparison was organised within the framework of an EU-funded project called BIOREMA. A general overview of the project and results of an interlaboratory comparison on bioethanol are published as Part 1 and 2 of this series of papers. In the study presented here, reference values, provided by national metrology institutes and expert laboratories, were used for evaluating the results. Consensus values, derived from the results of all participants, were used to assess any bias between the results from the national metrology institutes and testing laboratories. The emphasis in this interlaboratory comparison was not the performance rating of the individual laboratories, but recognising and interpreting differences caused by the measurement methods applied. For most biodiesel parameters, a good agreement of measurement results was found among different methods, and between the consensus and reference values. The study material was a rapeseed oil fatty acid methyl ester, for which it was demonstrated that it is feasible to prepare and characterise reference materials for biodiesel with well-established reference values for many parameters.     

Are physico-chemical soil characteristics key factors to select the polycyclic aromatic hydrocarbons extraction procedure?

The aim of this work was to assess the impact of soil characteristics and constituents in the total extraction of hydrophobic organic pollutants, such as polycyclic aromatic hydrocarbons (PAH), in real polluted soil samples from different sources. Soil samples were obtained from a wood creosote treatment plant, in the vicinity of a metallurgy industry and coal thermal power stations. Soils showed a wide diversity of textures, organic matter (OM) and CaCO₃ content, pH and electrical conductivity to assure representativeness of multiple situations. Two extraction procedures with soft (solid–liquid extraction, SLE) and intense (pressurised liquid extraction, PLE) extraction power were used to determine the total concentration of PAH in soils. Results obtained showed that soil properties affect the effectiveness of the extraction procedures tested. The validation of PAH extraction procedure with a reference soil did not confirm that the procedure was adequate for all kinds of soil. Results showed that OM content and clay were the main soil characteristics that should be taken into account to select the most adequate PAH extraction procedure for any given polluted soil.     

Comparison of different statistical methods for evaluation of proficiency test data

In analytical chemistry, proficiency testing usually consists in tests that laboratories conduct under routine conditions and report the result to the PT provider who then converts the result to a score which helps the participant to assess the accuracy of the result. The aim of this work is to show PT providers, accreditations bodies, and participating laboratories that different scoring results can be achieved depending on the evaluation system selected. The influence of different evaluation techniques on the results of an interlaboratory comparison for determination of gold in precious metals alloys was investigated. Results from 19 participating laboratories were evaluated by means of the three procedures: (1) classical statistical approach—outliers detection; (2) robust methods—(2A) robust procedure and (2B) ISO 13528; and (3) fitness for purpose. Evaluation of the same PT data revealed very interesting issues depending on the different scoring systems that were used and the robustness of the statistical methods used for detecting outliers. As a general rule, laboratories with scoring Z > 2 offered clearly poorer performance in robust approaches than classical ones. In order to support this first evidence, we evaluated a second data set with results from 24 laboratories (mercury from soil samples) by means of the four mentioned approaches. Selection and comparison of different scoring systems must be done very carefully, because sometimes they are not the best approach for studying the data population or the more appropriate one for evaluating the distribution of the data. Finally it should be taken into account that sometimes the robust scoring systems are not always suitable for evaluating the results of some PT schemes.     

Development of a proficiency testing scheme for a limited number of participants in the field of natural water analysis

A proficiency testing (PT) scheme was developed for a limited number of analytical laboratories participating in the analysis of natural water in Israel. Three fit-for-intended-use reference materials (RMs) were prepared for a pilot PT from natural water: RM-blank and two in-house reference materials (IHRMs) with seven analytes fortified to achieve different levels of analyte concentrations. The measurands for the PT participants were the traceable spike values certified in the IHRMs, i.e., added mass concentrations of the analytes. The RM-blank and IHRMs were found to be homogeneous and stable over 11 days, the time necessary for the experimental part of the PT. The RMs were distributed to the PT participants as unknown test items similar to routine samples. The test results were evaluated using their deviations from the IHRM-certified values. Eight Israeli laboratories took part in the interlaboratory comparison. Individual laboratory performance and metrological compatibility of the PT results of the participants, as a local group of laboratories, were evaluated for every analyte.     

Capability of laboratories to determine core nutrients in foods: results of an international proficiency test

A proficiency test (PT) to assess the capabilities of laboratories to determine nutrients in a biscuit sample was carried out in September 2009. The need for such interlaboratory comparison arose from an increasing nutrition labeling requirements of different countries around the world. Forty-eight laboratories worldwide participated in this PT program for the determination of total lipids, saturated fat, trans-fat, protein, sugars, total dietary fiber (TDF), ash and sodium in food. This program was organized under the auspices of the Asia-Pacific Laboratory Accreditation Cooperation (APLAC). APLAC considers PT programs as one of the objective means of assessing the performance of the accredited laboratories. The program is one of the APLAC PT series whose primary purposes are to establish mutual agreement on the equivalence of the operation of APLAC member laboratories and to provide an opportunity to identify testing deficiency so that corrective actions can be taken, if necessary. The results of this program indicated that the consensus mean values estimated by robust statistics were in good agreement with the corresponding values obtained in the homogeneity tests except for trans-fat and TDF. The relative standard deviations of participant results for trans-fat and TDF were also found to be higher (19 and 24%, respectively) than the other test parameters. The relative standard deviations for other test parameters were less than 10%. One-half of the laboratories obtained satisfactory z-scores (| z | < 3) (\left| z \right| < 3) for all their reported results. Although participants were instructed to provide the measurement uncertainties of their reported results, only 27.5% of them had done so. Albeit many of the measurement uncertainties appear to be reasonable, some of them were found to vary widely, indicating that some laboratories still have difficulties in estimating measurement uncertainties. Although the overall measurement capability can be considered to be satisfactory, there is still room for improvement of analytical procedures.     

PT/EQA standards and guidelines: some remarks on quality and reliability of test items

Current standards and international guides on PTs/EQAs consider the achievement of the quality and reliability of PT test items as important steps. Similarity to the materials routinely analysed and sufficient homogeneity and stability of the properties of interest are the main quality characteristics required. PT test items are carriers of the property values used as reference to evaluate laboratory performances. Requirements for PT test items are practically the same as those given for reference materials. ISO-REMCO defines reference material as “Material, sufficiently homogeneous and stable with respect to one or more specified properties, which has been established to be, fit for its intended use in a measurement processes”. In the case of PT test items, they should have a degree of homogeneity and stability to be fit for the purpose of identifying deviations among laboratories. This paper reports some remarks on the main characteristics of PT test items, using ISPRA experience as RM producer.     

Use of an excess variance approach for the certification of reference materials by interlaboratory comparison

In the nuclear field, the accuracy and comparability of analytical results are crucial to insure correct accountancy, good process control and safe operational conditions. All of these require reliable measurements based on reference materials whose certified values must be obtained by robust metrological approaches according to the requirements of ISO guides 34 and 35. The data processing of the characterization step is one of the key steps of a reference material production process. Among several methods, the use of interlaboratory comparison results for reference material certification is very common. The DerSimonian and Laird excess variance approach, described and implemented in this paper, is a simple and efficient method for the data processing of interlaboratory comparison results for reference material certification. By taking into account not only the laboratory uncertainties but also the spread of the individual results into the calculation of the weighted mean, this approach minimizes the risk to get biased certified values in the case where one or several laboratories either underestimate their measurement uncertainties or do not identify all measurement biases. This statistical method has been applied to a new CETAMA plutonium reference material certified by interlaboratory comparison and has been compared to the classical weighted mean approach described in ISO Guide 35. This paper shows the benefits of using an “excess variance” approach for the certification of reference material by interlaboratory comparison.     

Preparation of candidate certified reference materials for the quality control of EDTA-and acetic acid-extractable trace metal determinations in sewage sludge-amended soil and terra rossa soil

The determination of extractable trace metal contents in soil using single extraction procedures is currently performed in many laboratories to assess the bioavailable metal fraction (and related phytotoxic effects) and the accessability to the environment (e.g. contamination of ground waters). Owing to the need for validation of the extraction schemes used and of the analytical techniques, the EC Measurements and Testing Programme (formerly BCR) has organized a project for improving the quality of determination of extractable trace metal contents in soil, the first step being an interlaboratory study to adopt common extraction procedures and the second being a certification campaign to certify two soils for their extractable trace element contents following these procedures. This paper gives a brief overview of the project organisation and describes the preparation, homogeneity and stability studies of two soil candidate reference materials (sewage sludge-amended and terra rossa soils).     

Determination of perfluorinated alkyl acid concentrations in human serum and milk standard reference materials

Standard Reference Materials (SRMs) are certified reference materials produced by the National Institute of Standards and Technology that are homogeneous materials well characterized with values for specified properties, such as environmental contaminant concentrations. They can be used to validate measurement methods and are critical in improving data quality. Disagreements in perfluorinated alkyl acid (PFAA) concentrations measured in environmental matrices during past interlaboratory comparisons emphasized the need for SRMs with values assigned for PFAAs. We performed a new interlaboratory comparison among six laboratories and provided, for the first time, value assignment of PFAAs in SRMs. Concentrations for perfluorooctane sulfonate (PFOS), perfluorooctanoate (PFOA), and other PFAAs in two human serum and two human milk SRMs are reported. PFAA concentration measurements agreed for serum SRM 1957 using different analytical methods in six laboratories and for milk SRM 1954 in three laboratories. The interlaboratory relative standard deviation for PFOS in SRM 1957 was 7%, which is an improvement over past interlaboratory studies. Matrix interferences are discussed, as well as temporal trends and the percentage of branched vs. linear isomers. The concentrations in these SRMs are similar to the present-day average concentrations measured in human serum and milk, resulting in representative and useful control materials for PFAA human monitoring studies.     

A practical procedure for assigning a reference value and uncertainty in the frame of an interlaboratory comparison

A method is suggested for the calculation of a reference value and its uncertainty to be used in the frame of an interlaboratory comparison (ILC). It is assumed that the reference value of the measurand is determined independently from the ILC round. It is derived from a limited set of measurement results obtained from one or several expert laboratories. The procedure involves three stages: (1) check of the experimental data and possible corrections; (2) check of the consistency of data, and possibly increase of the uncertainties in order to attain internal consistency; (3) choice between fully, partially or un-weighted mean.     

One way of disseminating reference values with demonstrated traceability and demonstrated uncertainty to field laboratories: IMEP

An operational interlaboratory comparison programme is described which disseminates SI-traceable reference values to laboratories worldwide. These reference values have an uncertainty and traceability that is demonstrated at the highest metrological level. Participating laboratories can use these values to establish the degree of equivalence of their measurement results and can use this to support their measurement capability claims, e.g. towards third parties. The programme has been run by the Institute for Reference Materials and Measurements (IRMM) since 1988, in the first phase as an awareness programme. Currently, IRMM is focusing its efforts on educational aspects of metrology via a collaboration with the European Co-operation for Accreditation, national metrological institutes (NMIs) and interested academic networks. The viewgraphs used are presented in the “Electronic Supplementary Material” of this ACQUAL issue.     

Single and Sequential Extraction in Sediments and Soils

Abstract

The determination of extractable trace metal contents in soils and sediments is currently performed in many laboratories to assess the bioavailable metal fraction (and related potential phyto-toxic effects) and the importance and possibility of mobilization of trace metals from polluted soil, sludge and sediment upon landfill application. Single and sequential extraction schemes are used for the assessment of the different “forms” of trace metals (e.g. “mobile/bioavailable”, “carbonate-bound” etc.). The lack of uniformity in the different extraction procedures used throughout the world does not allow the results to be compared or the procedures to be validated which has led to many criticisms in the past few years. Moreover, the lack of suitable reference materials for this type of operationally defined determinations did not enable the quality of the measurements to be controlled. Owing to the need for establishing common schemes for single and sequential extractions as well as for the improvement of the quality of extractable trace metal determinations in soil and sediments, the Community Bureau of Reference (BCR) has organised a project which results along with the state of the art of extractable trace metal determinations, use and applicability of extraction schemes and analytical limitations were decided to be thoroughly discussed in a workshop. This paper presents its main conclusions.     

Interlaboratory Study to Improve the Quality Control of Methylmercury Determination in Sediment

The results are presented of an interlaboratory study on methylmercury (MeHg) in sediment carried out by a group of European laboratories within the framework of a project managed by the EC Standards, Measurements and Testing Programme (formerly BCR). The aim of this exercise was to evaluate the performance of current methods used for MeHg determination in sediment in order to improve the state-of-the-art prior to the certification of a candidate reference material. The paper describes the organization of the interlaboratory study, the preparation of the sediment material used, the techniques evaluated and the results obtained by the participating laboratories. The outcome of the collaborative project showed that certification could be contemplated, providing that certain analytical techniques were optimized, especially with regard to extraction methods.     

Proficiency testing of pH and electrolytic conductivity measurements of groundwater: a case study of the difference between consensus and metrologically traceable values

A proficiency testing (PT) scheme was designed for measurements of pH and electrolytic conductivity (EC) of groundwater in Israel. A groundwater sample was used for preparation of test items fit-for-intended-use. The interlaboratory experiment was organized taking into account changes of the test item properties which occurred after sampling due to presence of CO₂ in the groundwater. A total of 34 PT participants reported pH measurements results, while 29 of them reported results of EC measurements. The pH results were normally distributed. However, a significant bias of the pH consensus value of the PT participants from the metrologically traceable certified value (obtained by the National Physical Laboratory of Israel) was detected. This bias was caused by the drift of the measurement response due to water stirring and CO₂ degassing from the sample during the measurement process. The bias of the EC consensus value from the metrologically traceable one was negligible, in spite of an abnormal distribution of the EC results of the PT participants.     

Performance evaluation of polycyclic aromatic hydrocarbons analysis in sediment from proficiency testing with metrological reference values

This paper describes a metrological approach to evaluate the measurement capability of laboratories participating in two proficiency testing (PT) programmes involving the analysis of five polycyclic aromatic hydrocarbons (PAHs) in sediment samples. Reference values of PAHs in the programmes for performance assessment were obtained from an accurate isotope dilution gas chromatography mass spectrometry (ID-GCMS) method which was thoroughly validated and verified. Isotope dilution mass spectrometry (IDMS) technique usually has a well-defined measurement uncertainty budget and a traceability link to an International System of Units. Provision of the metrological reference values in PT enables the establishment of a technical platform to assess the actual competence of the participating laboratories in sediment PAHs analysis. Results of the PT programmes showed that about 80 % of the laboratories employed gas chromatography in their analyses and the remaining used liquid chromatography. Irrespective of the techniques being used, however, the majority of the participating laboratories were observed to underestimate values in which the mean values of the five reported PAHs were less than those of the ID-GCMS-derived reference values by 13–20 %. Only 41–44 % of the participating laboratories were able to achieve satisfactory z-scores. The present study revealed that the reinforcement of the capability for accurate measurement of PAHs in sediment samples in laboratories worldwide should be addressed.     

Quality assurance and quality control in forest soil analyses: a comparison between European soil laboratories

In transnational monitoring programmes, a balance between international reference methods, which improve spatial comparability, and national analysis methods that favour temporal comparability, by their use and testing over many years, needs to be sought. Prior to the next Pan-European Forest Soil Survey, a third interlaboratory comparison of soil analysis methods was organised. All participating laboratories were requested to use the same reference methods. Fifty-two soil laboratories from 27 European countries analysed a total of 48 soil parameters on three soil samples which were typical for European forest soils. The results of the statistical analysis showed a high interlaboratory and intralaboratory variability, especially for the acid oxalate extractions, particle size distribution, exchangeable elements and total carbonates. The intercomparability of the test results did not improve compared to the previous ring test. As the exercise aimed primarily at comparing the performance of the laboratories, it was not powerful enough to find cause–effect relationships between the meta information provided by the laboratories and the variability of the test results.

     

Quality assurance and quality control in forest soil analyses: a comparison between European soil laboratories

In transnational monitoring programmes, a balance between international reference methods, which improve spatial comparability, and national analysis methods that favour temporal comparability, by their use and testing over many years, needs to be sought. Prior to the next Pan-European Forest Soil Survey, a third interlaboratory comparison of soil analysis methods was organised. All participating laboratories were requested to use the same reference methods. Fifty-two soil laboratories from 27 European countries analysed a total of 48 soil parameters on three soil samples which were typical for European forest soils. The results of the statistical analysis showed a high interlaboratory and intralaboratory variability, especially for the acid oxalate extractions, particle size distribution, exchangeable elements and total carbonates. The intercomparability of the test results did not improve compared to the previous ring test. As the exercise aimed primarily at comparing the performance of the laboratories, it was not powerful enough to find cause–effect relationships between the meta information provided by the laboratories and the variability of the test results.     

Experimental evaluation of different precision criteria applicable to microbiological counting methods

The dispersion of microbiological counting measurements, when repeating the analysis on the same material both within a laboratory (repeatability) and between laboratories (reproducibility) can be characterized by the organization of interlaboratory studies, where several sets of identical test materials are sent to several laboratories. Using the example of data generated by an interlaboratory study on enumeration of Listeria monocytogenes in foods by the standardized reference method (colony-count technique), 2 types of robust estimators of reproducibility standard deviations, based on the median, were examined, in comparison with the classical estimators, based on the mean. Experimental evaluation indicated that the 3 approaches gave consistent results for most of the combinations. The usual log10 transformation of the enumeration results was also questioned before these calculations were conducted.     

Providing a traceable assigned value in a proficiency testing programme on the determination of benzoic acid in orange juice

A proficiency testing (PT) programme on the determination of benzoic acid in orange juice was organised by the Chemical Metrology Laboratory, Health Sciences Authority, Singapore, for the local food testing laboratories. The PT programme used a metrologically traceable assigned value determined by isotope dilution mass spectrometry to assess the performance of the participating laboratories. This paper discusses the reasons and approach to applying metrological principles to determine the assigned value of a PT programme. The procedure undertaken by the Laboratory to prepare a batch of PT sample, conduct homogeneity and stability testings, as well as value assignment, is presented. The discussions also include the estimation of the measurement uncertainties arising from the characterisation, testing for homogeneity and testing for stability of the PT sample. The Coordinator of the PT programme used the assigned value and its associated measurement uncertainty, together with those of the participating laboratories to assess their performances. Statistical tools applied in the performance evaluations are also presented.     

Laboratory intercomparison study on the analysis of short-chain chlorinated paraffins in an extract of industrial soil

Short-chain chlorinated paraffins (SCCPs) comprise a class of organic pollutants used in many industrial applications and released into the environment. The analytical determination of SCCPs is very challenging. Although there is at present no fully validated measurement procedure that might be applied in routine monitoring, the European Union Water Framework Directive (WFD) has required regular monitoring of this class of compounds at river-basin scale since 2007.To assess the status quo of the analysis of SCCPs in relation to the requirements of the WFD, we organized an interlaboratory comparison on the quantification of SCCPs in an extract of an industrial soil. Six laboratories participated in the exercise using three different techniques [i.e. gas chromatography (GC) coupled to mass spectrometry (MS) in electron-capture negative ionization mode, GC with atomic emission detection, and carbon-skeleton GC-MS]. The results reported were in the range 8.5–3200 mg/L. This confirms that reliable quantification of SCCPs is still very difficult to achieve and that the comparability of SCCP data reported to the European Commission is at least questionable.