Use of characteristic functions derived from proficiency testing data to evaluate measurement uncertainties

Interlaboratory comparisons show that reproducibility standard deviations are dependent on the concentration of the analyte. Many attempts have been made to model this. In this paper, ??characteristic?? functions are used for modelling the concentration dependence of the reproducibility standard deviations based on data from proficiency tests for water analysis. The characteristics of the resulting functions can be used for the estimation of measurement uncertainties at different concentration levels. These functions are especially useful to determine the concentration levels below which absolute uncertainties tend to be constant and above which the relative uncertainties are more constant. By comparing the characteristic functions of different analytical procedures for the determination of the same analyte, the performance of these procedures under routine application can be compared. Finally, these functions may be used to get an indication on the average quality of analytical result in a specific field to be used by regulators in order to formulate requirements in the legislation that are in accordance with current measurement quality.     

Evaluation of the analytical method performance for incurred samples

A methodology for the evaluation of the performance of an analytical method for incurred samples is presented. Since this methodology is based on intra-laboratory information, it is suitable for analytical fields that lack reference materials with incurred analytes and it can be used to evaluate the analytical steps prior to the analytical portion, which are usually excluded in proficiency tests or at the certification of reference materials. This methodology can be based on tests performed on routine samples allowing the collection of information on the more relevant combinations analyte/matrix; therefore, this approach is particularly useful for analytical fields that involve a high number of analyte/matrix combinations, which are difficult to cover even considering the frequent participation in expensive proficiency tests.This approach is based on the development of a model of the performance of the analytical method based on the differential approach for the quantification of measurement uncertainty and on the comparison of recovery associated with each one of the analytical steps whose performance can vary with the analyte origin, for spiked and incurred samples.This approach was applied to the determination of pesticide residues in apples. For the analytes covered, no evidence was found that the studied sample processing and extraction steps performance for this matrix varies with the analyte origins.     

Establishing measurement traceability in clinical chemistry

Measurement traceability is probably the most important tool for achievement of comparability in clinical chemistry. As stipulated by the In Vitro Diagnostica Directive of the European Union and several ISO standards, values assigned to calibrators and control materials must be traceable to reference materials and/or reference procedures of higher order. In the German proficiency testing system, statutory use of reference measurement procedures for several measurands has been in force since 1988. As a result, reference procedures are now regularly applied for the setting up of target values in the control samples of internal and external quality assessment and for assigning values to the manufacturers calibrator and control materials. Noticeably, the comparability of results obtained by different diagnostic tests has greatly improved for the measurement of many metabolites and substrates, e.g. creatinine, cholesterol, uric acid, total glycerol and urea. For many measurands in laboratory medicine the implementation of the concept of traceability proves to be much more difficult; this mainly concerns the measurement of proteins, in particular enzymes, proteo-hormones, tumour markers and cardiac markers. For such measurands the analyte must first of all be distinctively defined before a reference system can be established which comprises reference procedures, reference materials and networks of reference laboratories.Presented at the CCQM Workshop on Traceability 16–17 April 2002 at the BIPM, Sèvres, France     

From GUM to alternative methods for measurement uncertainty evaluation

Since the advent of the Guide to the expression of Uncertainty in Measurement (GUM) in 1995 laying the principles of uncertainty evaluation numerous projects have been carried out to develop alternative practical methods that are easier to implement namely when it is impossible to model the measurement process for technical or economical aspects. In this paper, the author presents the recent evolution of measurement uncertainty evaluation methods. The evaluation of measurement uncertainty can be presented according to two axes based on intralaboratory and interlaboratory approaches. The intralaboratory approach includes “the modelling approach” (application of the procedure described in section 8 of the GUM, known as GUM uncertainty framework) and “the single laboratory validation approach”. The interlaboratory approaches are based on collaborative studies and they are respectively named “interlaboratory validation approach” and “proficiency testing approach”.     

Comparability of measurement results for pesticide residues in foodstuffs: an open issue?

The dispersion of results from proficiency tests for the analysis of pesticide residues in foodstuffs suggests that improvements in the compatibility of measurement results are needed. Currently observed divergences can make the evaluation conclusion on foodstuffs compliance with certain legislation dependent on the consulted laboratory. This work discusses the origin and impact of this lack of compatibility, following the metrological concepts presented at the latest version of the “International Vocabulary of Metrology” (VIM3), thus allowing for a clear diagnostic of the problem. The reporting of results from different measurement methods uncorrected for the observed analyte recovery makes them traceable to different “operationally defined measurement procedures” (VIM3) and, therefore, not comparable. When results from different measurement methods are reported corrected for analyte recovery, R, and R is different for spiked and incurred residues, measurement results may be not compatible if this effect is not considered on the uncertainty budget. This discussion is illustrated with metrological models for any possible combination of “measurement performance” and “maximum residue level”. These models are complemented with experimental data of the analysis of pesticide residues in a sample of ginseng powder from a proficiency test. The adopted experimental design allowed the identification of additional threats to metrological compatibility in this field. Solutions to the faced problem are discussed for practicability and impact on regulatory issues. The use of a universal “reference measurement procedure” proves to be the most feasible way of ensuring comparability of measurements in this field.     

Factor transformation to produce statistics describing the uncertainty of analytical data.

Analytical uncertainty produced by random error has a positively skewed distribution and accuracy and precision have non-linear scales. Compared with conventional statistics, factor transformation of the data allows more appropriate interpretation of results and facilitates graphical inspection of data. Statistics are compared for practical examples of performance in proficiency tests and of repeatability and reproducibility in collaborative studies. Factor transformation is shown to be applicable to wide ranges of analyte concentration and measurement precision.     

The application of data from proficiency testing to laboratory accreditation according to ISO 17025

Current methods of testing laboratories for their proficiency in reporting correct measurement results are liable to substantial errors of the second kind. This means that laboratories with deflated uncertainties are accepted as proficient, even though their reported measurement results pave the way for erroneous conclusions. Only by using E _n numbers based on an accepted reference value with the lowest possible uncertainty can the risk of recognising incorrect measurement results be kept at an acceptable level. Based on an actual set of proficiency test (PT)-data for the concentration of Pb in water, this paper compares PT results obtained by methods using E _n numbers with methods based on z-scores. Kaj Heydorn is a technical assessor at The Danish Accreditation and Metrology Fund (DANAK).     

An error model for evaluation and interpretation of proficiency testing

Data from proficiency testing can be used to increase our knowledge of the performance of populations of laboratories, individual laboratories and different measurement methods. To support the evaluation and interpretation of results from proficiency testing an error model containing different random and systematic components is presented. From a single round of a proficiency testing scheme the total variation in a population of laboratories can be estimated. With results from several rounds the random variation can be separated into a laboratory and time component and for individual laboratories it is then also possible to evaluate stability and bias in relation to the population mean. By comparing results from laboratories using different methods systematic differences between methods may be indicated. By using results from several rounds a systematic difference can be partitioned into two components: a common systematic difference, possibly depending on the level, and a sample-specific component. It is essential to distinguish between these two components as the former may be eliminated by a correction while the latter must be treated as a random component in the evaluation of uncertainty. Received: 20 November 2000 Accepted: 3 January 2001     

Quality issues in proficiency testing

 Proficiency testing is a means of assessing the ability of laboratories to competently perform specific tests and/or measurements. It supplements a laboratory's own internal quality control procedure by providing an additional external audit of their testing capability and provides laboratories with a sound basis for continuous improvement. It is also a means towards achieving comparability of measurement between laboratories. Participation is one of the few ways in which a laboratory can compare its performance with that of other laboratories. Good performance in proficiency testing schemes provides independent evidence and hence reassurance to the laboratory and its clients that its procedures, test methods and other laboratory operations are under control. For test results to have any credibility, they must be traceable to a standard of measurement, preferably in terms of SI units, and must be accompanied by a statement of uncertainty. Analytical chemists are coming to realise that this is just as true in their field as it is for physical measurements, and applies equally to proficiency testing results and laboratory test reports. Recent approaches toward ensuring the quality and comparability of proficiency testing schemes and the means of evaluating proficiency test results are described. These have led to the drafting of guidelines and subsequently to the development of international requirements for the competence of scheme providers. Received: 2 January 1999 · Accepted: 7 April 1999     

Evaluation of proficiency testing results taking into account their uncertainties

A new composite score for the evaluation of performance of proficiency testing participants is proposed. The score is based on a combination of the z-score, uncertainty of a participant’s measurement result and uncertainty of the proficiency testing scheme’s assigned value. The use of such a composite score will allow evaluation not only of the participant’s ability to determine an analyte in corresponding matrix, but also their understanding of the uncertainty in the obtained analytical result. The score may be helpful for the laboratory’s quality system and for laboratory accreditation according to ISO 17025.     

Effect of the sample matrix on measurement uncertainty in X-ray fluorescence analysis

The estimation of measurement uncertainty, with reference to univariate calibration functions, is discussed in detail in the Eurachem Guide “Quantifying Uncertainty in Analytical Measurement”. The adoption of these recommendations to quantitative X-ray fluorescence analysis (XRF) involves basic problems which are above all due to the strong influence of the sample matrix on the analytical response. In XRF-analysis, the proposed recommendations are consequently applicable only to the matrix corrected response. The application is also restricted with regard to both the matrices and analyte concentrations.In this context the present studies are aimed at the problems to predict measurement uncertainty also with reference to more variable sample compositions. The corresponding investigations are focused on the use of the intensity of the Compton scattered tube line as an internal standard to assess the effect of the individual sample matrix on the analytical response relatively to a reference matrix. Based on this concept the estimation of the measurement uncertainty of an analyte presented in an unknown specimen can be predicted in consideration of the data obtained under defined matrix conditions.     

Estimates of uncertainty of measurement from proficiency testing data: a case study

The results obtained by a laboratory over a number of proficiency testing/external quality assessment schemes (PT/EQAS) rounds can give information on the uncertainty of its measurements for a given test, provided that conditions such as full coverage of the routine analytical range, traceability, and small uncertainty of the assigned values (compared to the spread of the results) are met and provided that systematic deviations and any other sources of uncertainty are considered. As organisers of the Italian EQAS (ITEQAS) in occupational and environmental laboratory medicine, we tested this hypothesis using as model data from well-performing laboratories taking part in ITEQAS for lead in blood over the last 2 years. We also investigated how different PT/EQAS features (frequency of trials and number of samples) would affect a laboratory estimate of its uncertainty. Such information can be helpful in improving PT/EQAS organisation and define, for a given test: (a) the state of the art of the uncertainty of current measurement procedures, (b) identify needs for improvement of analytical methodologies and (c) set targets for acceptable uncertainty values.Presented at the Eurachem PT Workshop September 2005, Portorož, Slovenia.Papers published in this section do not necessarily reflect the opinion of the Editors, the Editorial Board and the Publisher.     

Estimation of uncertainty in routine pH measurement

A procedure for estimation of measurement uncertainty of routine pH measurement (pH meter with two-point calibration, with or without automatic temperature compensation, combination glass electrode) based on the ISO method is presented. It is based on a mathematical model of pH measurement that involves nine input parameters. Altogether 14 components of uncertainty are identified and quantified. No single uncertainty estimate can be ascribed to a pH measurement procedure: the uncertainty of pH strongly depends on changes in experimental details and on the pH value itself. The uncertainty is the lowest near the isopotential point and in the center of the calibration line and can increase by a factor of 2 (depending on the details of the measurement procedure) when moving from around pH 7 to around pH 2 or 11. Therefore it is necessary to estimate the uncertainty separately for each measurement. For routine pH measurement the uncertainty cannot be significantly reduced by using more accurate standard solutions than ±0.02 pH units – the uncertainty improvement is small. A major problem in estimating the uncertainty of pH is the residual junction potential, which is almost impossible to take rigorously into account in the framework of a routine pH measurement.¹ Received: 11 August 2001 Accepted: 22 February 2002     

A statistical strategy for discouraging collusion in split-level proficiency testing schemes

A statistical strategy for discouraging collusion in split-level proficiency testing schemes was developed. When the samples are randomly labeled, the difference in the analyte levels in the two samples can be designed to maximize the probability of detecting collusion when it occurs, so that participants who colluded cannot easily receive satisfactory assessments. This favors providing convictive assessments of participants proficiency and giving the participants a level playing field.     

Estimation of measurement uncertainty in food microbiology: The ISO approach

Given the current interest in measurement uncertainty (MU) in food microbiology, in particular for laboratory accreditation purposes, and the need to have harmonized reference documents specifically in that area at the international level, ISO is conducting works to meet this need. An ISO Technical Specification (ISO/TS 19036) is being prepared on MU estimation for quantitative determinations. A global approach has been chosen, based on the reproducibility standard deviation of the final result of the measurement process. Three possibilities are envisaged for the estimation of the reproducibility standard deviation, in a decreasing order of preference: The intra-laboratory standard deviation, the inter-laboratory standard deviation derived from method validation, and the inter-laboratory standard deviation derived from proficiency testing.The uncertainty of qualitative determinations is still under investigation, and will be covered by a separate ISO publication. Presented at AOAC Europe/Eurachem Symposium March 2005, Brussels, Belgium     

Proficiency testing for quality assurance of allergens methods

Proficiency testing is an external quality control check, whereby the quality of an analytical result is checked against criteria that are set independently of the laboratory carrying out the analysis. Participants in a proficiency test are encouraged to use the method of their choice to determine the analyte in question. The collated results submitted by the participants are used to derive the best estimate of the ‘true’ level, or assigned value, of the analyte, as a consensus value of the whole data set. Generally, the data submitted will be normally distributed and from a single population, but if a data set is found to be multimodal, then the selection of one of the modes as the assigned value is possible where there is supporting data, typically methodology information. Unless there are independent grounds for preferring one mode over another, it is not possible to set an assigned value or calculate z-scores. However, the analysis of allergens has presented proficiency testers with a new challenge, since it has become apparent that quantitative results may be dependent on the brand of enzyme-linked immunosorbent assay kit used, the specific analyte targeted (e.g. total content or allergen protein content) and the limit of detection achievable. FAPAS® has run more than 40 proficiency tests for allergen analysis over the past 7 years, during which time methods have been developed and improved and the requirements for determination of food ingredient allergens has increased. Two case studies are presented which highlight some of the issues around the use of allergen measurement methods.

Open image in new window

Figure A selection of food items which might cause allergenic or intolerance reactions

     

Uncertainty evaluation in proficiency testing: state-of-the-art, challenges, and perspectives

The evaluation of measurement uncertainty, and that of uncertainty statements of participating laboratories will be a challenge to be met in the coming years. The publication of ISO 17025 has led to the situation that testing laboratories should, to a certain extent, meet the same requirements regarding measurement uncertainty and traceability. As a consequence, proficiency test organizers should deal with the issues measurement uncertainty and traceability as well. Two common statistical models used in proficiency testing are revisited to explore the options to include the evaluation of the measurement uncertainty of the PTRV (proficiency test reference value). Furthermore, the use of this PTRV and its uncertainty estimate for assessing the uncertainty statements of the participants for the two models will be discussed. It is concluded that in analogy to Key Comparisons it is feasible to implement proficiency tests in such a way, that the new requirements can be met. Received: 29 September 2000 Accepted: 3 December 2000     

Affinity partitioning for membrane purification exploiting the biotin-NeutrAvidin interaction. Model study of mixed liposomes and membranes

The linear-solvent strength (LSS) model of gradient elution has been applied to estimate parameters of lipophilicity and acidity of a series of drugs and model chemicals. Apparent pKa values and log kw values for individual analytes were determined in 2-3 gradient runs. The first experiment (or first two experiments) uses a wide-range organic modifier gradient with pH chosen for suppressed ionization of the analyte. The result of this experiment allows an estimate of contents of organic modifier of the mobile phase (%B) providing the required retention coefficient, k, for the non-ionized analyte. The following experiment is carried out with the latter %B and a pH-gradient of the aqueous component of the eluent that is sufficient to overlap the possible pKa-value of the analyte. The initial pH of the buffer used to make the mobile phase is selected to insure that the analyte is in non-ionized form. The resulting retention time allows an estimate of PKa in a solvent of the selected %B. At the same time, estimates of log kw can also be obtained. The log kw parameter obtained from gradient HPLC by the approach proposed correlated well with the corresponding value obtained by standard procedure of extrapolation of retention data determined in a series of isocratic measurements. Correlation between log kw and the reference parameter of lipophilicity, log P, was very good for a series of test analytes and satisfactory for a structurally diverse series of drugs. The approach supported with specific detection procedures can be recommended for fast screening of lipophilicity of individual components of complex mixtures like those produced by combinatorial chemistry. The values of pKa obtained in a study were found to correlate with the literature pKa data determined in water for a set of aniline derivatives studied. In case of a series of drugs the correlation was less than moderate if the general procedure of pKa determination was applied.