Harmonization and transferability of performance assessment: experience from four serum aluminum proficiency testing schemes

Proficiency testing schemes monitor laboratory performance and provide a stimulus for improvement in accuracy. Where several schemes operate in the same analytical sector, there are risks that assessments of performance may be in conflict. Performance assessment for the determination of trace elements such as aluminum in serum is particularly important due to the high risk of contamination and therefore erroneous results. The objectives of this work were (1) to compare several mathematical models to establish a predefined standard deviation for proficiency assessment and (2) to evaluate the influence of instrumental methods and proficiency testing scheme on the assessment of performance for serum aluminum measurements. For this purpose, three samples were sent to the participants of four proficiency testing schemes. Assigned values were calculated according to algorithm A according to ISO 13528 and standard deviation for proficiency assessment according to three methods based on individual variability, state of the art or previous proficiency testing results. The method based on individual variability produced a more stringent standard deviation compared to analytical imprecision based on the state of the art. The instrumental methods gave similar results, whereas significant differences were observed between the four proficiency testing schemes indicating that harmonization of the standard deviation for proficiency assessment fails to allow transferability from one proficiency testing scheme to another and that additional factor(s) contribute to variability in performance assessment.     

Determination of the standard deviation for proficiency assessment from past participant’s performances

The ??uncertainty function?? introduced by Thompson et al. estimates the reproducibility standard deviation as a function of concentration or mass fraction. This model was successfully applied to data derived from three proficiency testing schemes aiming at the quantification of cadmium, lead and mercury in blood and urine. This model allows the estimation of standard deviation for the performance assessment for proficiency testing rounds.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Detection of genetic modification in cured tobacco leaf: proficiency testing using polymerase chain reaction-based methods

The Cooperation Centre for Scientific Research Relative to Tobacco (CORESTA; Paris, France) "Task Force Genetically Modified Tobacco-Detection Methods" investigated the performance of qualitative and quantitative methods based on the polymerase chain reaction (PCR) for the detection and quantitation of genetically modified (GM) tobacco. In the 4 successful rounds of proficiency testing, the cauliflower mosaic virus 35S RNA promoter (CaMV 35S) and the Agrobacterium tumefaciens nopaline synthase terminator (NOS) were selected as target sequences. Blind-coded reference materials containing from 0.1 to 5.0% and from 0.15 to 4% GM tobacco were used in 2 rounds of qualitative and quantitative PCR, respectively. Eighteen laboratories from 10 countries participated in this study. Considering all methods and 2 rounds, the different laboratories were able to detect GM tobacco at the 0.1% level in 46 out of 58 tests in qualitative assays. The results of the proficiency test indicate that both end point screening and real-time quantitative methods are suitable for the detection of genetically modified organisms in tobacco leaf samples having a GM content of 0.1% or higher. The CORESTA proficiency study represents a first step towards the interlaboratory evaluation of accuracy and precision of PCR-based GM tobacco detection, which may lead to the harmonization of analytical procedures and to the enhancement of comparability of testing results produced by different laboratories.     

Long-term external quality assessment program for CD4+ T-lymphocyte enumeration in Thailand

The accuracy and reliability of CD4+ T-lymphocyte enumeration are crucial for HIV healthcare management. Thailand’s CD4 external quality assessment (EQA) program was implemented in 2003. This program aims to improve the quality of CD4 testing by providing the quality control material, remedial action, and technical training. In this article, the overall longitudinal performance of the participating laboratories from 2003 to 2015 is reported. The EQA blood samples were sent to the participating laboratories in Thailand and other Southeast Asian (SEA) countries. The participants were requested to enumerate CD3+ and CD4+ T-lymphocytes. The trimmed mean, standard deviation (SD), coefficient of variation (CV), and standard deviation index (SDI) of both fraction and number were calculated from the returned data. Our result indicated a continuously increasing number of participants. The response rate of 72 trials was 98 %. The outlier rates for the fraction and number of CD4+ T-lymphocytes were 3.3 % and 3.4 %, respectively. The average CV of the fraction and number of CD4+ T-lymphocytes was 7.1 % and 9.4 %, respectively. In addition, the analysis of our current EQA trial no. 072 revealed that more than 98 % of participating laboratories had SDI values between ?2 and +2 for both fraction and number of CD3+ and CD4+ T-lymphocytes. In conclusion, the implementation of the CD4 EQA program in Thailand and other SEA countries has led to a reduction in the variability of the results of the CD4 count between laboratories and a continuous improvement in laboratory performance.     

The mutual influence between <Emphasis Type="Italic">π</Emphasis>-hole pnicogen bonds and <Emphasis Type="Italic">σ</Emphasis>-hole halogen bonds in complexes of PO<Subscript>2</Subscript>Cl and XCN/C<Subscript>6</Subscript>H<Subscript>6</Subscript> (X = F,Cl, Br)

The bimolecular and termolecular complexes involving PO₂Cl and XCN/C₆H₆ (X = F, Cl, Br) were designed to form the π-hole pnicogen bonds and σ-hole halogen bonds, to compare the two types of interactions and investigate the mutual influences between them. PO₂Cl was used as simultaneous π-hole and σ-hole donor; it can interact with electron donor to form π-hole pnicogen bond and σ-hole halogen bond. The π-hole interactions are stronger than the σ-hole interactions, in both the bimolecular and the termolecular complexes. Comparing the mutual effects of the π-hole interactions and σ-hole interactions, the π-hole interaction has a greater influence on the σ-hole interaction than vice versa. With the addition of σ-hole halogen bond, the V _S,max value outside the π-hole region of PO₂Cl becomes decreasingly positive, resulting in a weaker π-hole interaction. With the addition of π-hole pnicogen bond, the V _S,max value outside the σ-hole region of PO₂Cl becomes small, also resulting in a weaker σ-hole interaction. The π-hole pnicogen bond and σ-hole halogen bond weaken each other, i.e., there is a negative cooperative effect in the termolecular complexes.     

Viscosities and Surface Tensions of Phenetole with <Emphasis Type="Italic">N</Emphasis>-Methyl-2-pyrrolidone, <Emphasis Type="Italic">N</Emphasis>,<Emphasis Type="Italic">N</Emphasis>-Dimethylformamide and Tetrahydrofuran Binary Systems at Three Temperatures

Viscosities, η, and surface tensions, σ, of binary systems of phenetole (ethoxy benzene or ethyl phenyl ether) with N-methyl-2-pyrrolidone, N,N-dimethylformamide or with tetrahydrofuran were measured over the entire mole fraction range and at (298, 303 and 308) K. The experimental data was used to compute the deviations in viscosity, Δη, and surface tension, Δσ. Values of the excess Gibbs energy of activation G*^E, surface entropy S _σ and surface enthalpy H _σ were calculated. Viscosity data of the binary systems were calculated using the Grunberg and Nissan and the three-body and four-body McAllister correlations. The Redlich–Kister method was used for evaluation of coefficients and standard deviations for Δη, Δσ and G*^E. The results were interpreted in terms of the probable effect of molecular interactions between components as well as polarity.     

Construction of a Standard Reference Plasmid for Detecting GM Cottonseed Meal

A plasmid was constructed for quantification of genetically modified (GM) cottonseed meal in the gene-specific level. The Cry1Ab/c gene was connected with the Sad1 gene by fusion PCR. The fusion gene was cloned into the pMD®19-T Simple Vector. The plasmid DNA was then digested with a restriction endonuclease SmaI to reduce the characteristic differences between the plasmid DNA and genomic DNA. For a rough quantitative analysis of GM cotton meal contents, a rapid method for measurement of the copy numbers of the transgenic Cry and cotton endogenous Sad1 gene using a real-time PCR system with the plasmid DNA as a calibrator was established. The inter-run and intra-run coefficients of variation were less than 1.48% and 2.36%, respectively. The limits of detection and quantitation of the Cry and Sad1 genes were 9 and 91 copies of pMDCS, respectively. These results prove that the standard plasmid represents a valuable alternative to genomic DNA as a certified reference material for the quantification of GM cotton and is a useful tool to establish a feasible identification management for GM cottonseed meal content in the feed industry.     

Analytical performance is improved by regular participation in proficiency testing: an analysis of data from the Aquacheck proficiency testing scheme

Analysis of historical data from the Aquacheck proficiency testing scheme demonstrates improvement in the performance of participants correlated to the frequency of participation. Analysis of the spread of results returned by all participants, defined by the percentage relative standard deviation (RSD) and the performance (Z) scores awarded to the individual participants, indicates that the overall performance for the group of participants is consistent. However, when the frequency of participation is analysed, it can be clearly seen that those laboratories who participated in the majority of the distributions available reported 10–15% more acceptable (Z-scores between +2 and ?2) results than those labs who took part with a greatly reduced frequency. When ‘more difficult’ analytes are considered, those laboratories who participated in greater than 75% of available rounds reported up to 30% more acceptable results than those laboratories who participated in 25% or fewer, of the available rounds.     

Proficiency testing in analytical laboratories: how to make it work

 This paper covers the role of proficiency testing schemes in providing an occasional but objective means of assessing and documenting the reliability of the data produced by a laboratory, and in encouraging the production of data that are "fit-for-purpose". A number of aspects of proficiency testing are examined in order to highlight features critical for their successful implementation. Aspects that are considered are: accreditation, the economics and scope of proficiency testing schemes, methods of scoring, assigned values, the target value of standard deviation σ_p, the homogeneity of the distributed material, proficiency testing in relation to other quality assurance measures and whether proficiency testing is effective. Stress is placed on the importance of any proficiency testing scheme adhering to a protocol that is recognised, preferably internationally. It is also important that the results from the scheme are transparent to both participating laboratory and its "customer". Received: 03 November 1995 Accepted: 20 November 1995     

The influence of a quality system according to ISO/IEC 17025 on the performance of Brazilian laboratories in proficiency testing in the environmental area

This paper presents a performance analysis of laboratories in proficiency testing (PT) with the main objective of evaluating whether having accreditation to ISO/IEC 17025 or not having an implemented quality system has an influence on the performance of these laboratories. This research was conducted with data from an accredited PT scheme of physical–chemical water analyses. Nine metals were considered in the analysis, with a total of 896 results. The performance indicator was z-score. The proposed analysis was conducted through statistical methods of analysis of variance, Dunnett’s test, Chi-square and cross-tabulation. It was verified that there is a significant difference between the performance of accredited laboratories, recognised laboratories, and laboratories that do not have an implemented quality system. It was established that an accredited or recognised quality system has a positive influence on the laboratory performance, resulting in a lower z-score (in module) and in a larger percentage of satisfactory results.     

Proficiency testing in food microbiology: experience from implementation of ISO/IEC 17043 and ISO/TS 22117

The proficiency testing (PT) scheme ??AQUA?? for food microbiology was organised by the Istituto Zooprofilattico Sperimentale delle Venezie (IZSVe) according to ISO/IEC 17043 and ISO/TS 22117. This paper describes the IZSVe experience on the application of the above-mentioned standards for the PTs, with focus on the Enterobacteriaceae enumeration one. Freeze-dried food matrices contaminated with American Type Culture Collection bacterial strains were used as test samples for each microbiological PT organised by IZSVe. The sample homogeneity and stability were verified prior to distribution to participants and throughout the PT, respectively. The participating laboratories analysed samples using their routine methods, and results were transmitted to IZSVe. Data and methods used by each participating laboratory were analysed in order to evaluate the laboratory performance. With reference to the Enterobacteriaceae PT, the test samples were homogeneous and stable. In addition, most laboratory results were obtained using equivalent test methods. Statistical approaches applied to analyse data generated from all participating laboratories revealed similar outcomes as no significant outlying count and only 5?% of unacceptable results were observed. Finally, the z-score, with the standard deviation that does not vary from round toround, was applied to compare and to evaluate the performance of each laboratory over time highlighting possible persistent trends over several rounds.     

The effect of the choice of method for determining assigned value on the assessment of performance in melting point analysis

The determination of melting point is a fundamental test in the Pharmaceutical industry, since it is one of the simplest techniques for the identification of a chemical substance. The melting point provides information on both identity and purity of a chemical substance and for that reason is a key test in the PHARMASSURE proficiency testing (PT) scheme. The PT scheme assesses participant??s determination of melting point, using chemicals of high purity and basing the assigned value on the robust consensus mean (median). In recent rounds, melting point reference standards have been provided as the test material and a reference value used as the assigned value for PT assessment. Comparison of the PT results over a number of rounds, using test materials with a wide range of melting points, shows the overall performance of the participant group is worse in rounds where a reference material and associated reference assigned value are used for performance assessment. When participants were assessed against the reference assigned value, a positive bias was observed in the participant??s results. Detailed information regarding the methodology used demonstrated that the majority of participants use the same analytical method, EU.Ph.2.2.14 (Council of Europe, Strasbourg, 2011), for the determination of melting point although this procedure allows flexibility in key methodological parameters, such as heating ramp rate, which may fail to ensure consistent performance across the group of participant laboratories.     

Evaluation of the use of consensus values in proficiency testing programmes

Proficiency testing (PT) is an essential tool used by laboratory accreditation bodies to assess the competency of laboratories. Because of limited resources of PT providers or for other reasons, the assigned reference value used in the calculation of z-score values has usually been derived from some sort of consensus value obtained by central tendency estimators such as the arithmetic mean or robust mean. However, if the assigned reference value deviates significantly from the ‘true value’ of the analyte in the test material, laboratories’ performance will be evaluated incorrectly. This paper evaluates the use of consensus values in proficiency testing programmes using the Monte Carlo simulation technique. The results indicated that the deviation of the assigned value from the true value could be as large as 40%, depending on the parameters of the proficiency testing programmes under investigation such as sample homogeneity, number of participant laboratories, concentration level, method precision and laboratory bias. To study how these parameters affect the degree of discrepancy between the consensus value and the true value, a fractional factorial design was also applied. The findings indicate that the number of participating laboratories and the distribution of laboratory bias were the prime two factors affecting the deviation of the consensus value from the true value.     

Interpretation of interlaboratory comparison results to evaluate laboratory proficiency

Guidelines are given for the evaluation of proficiency test (PT) results in order to increase the effectivity of PT participation. For better understanding, some statistical background is given along with some examples to show the effects of the choices made by the PT provider. The calculation method of the assigned value and the selection of the standard deviation both affect the z-score that is used by the participating laboratory to judge the quality of its performance in the PT. Therefore, the participating laboratory is advised to use the PT results with care and, if necessary, to recalculate the z-scores. Finally, advice is given on how not to follow up bad PT results along with some valuable steps that could be part of an effective follow-up procedure.     

Assessment of performance of laboratories in determining acrylamide in crispbread

A proficiency testing round was undertaken to assess the performance of laboratories to measure acrylamide in a sample of crispbread. Retail samples of crispbread were ground to a fine powder and after thorough mixing were packed in 40 g units for distribution. Ten samples were selected at random and analyzed in duplicate for acrylamide by liquid chromatography/mass spectrometry (LC/MS). Standard statistical tests showed that the material was homogeneous for the purposes of proficiency testing. Test samples were distributed to 55 laboratories in 16 countries in Europe, North America, Australia, and the Middle East. The results were analyzed by standard proficiency testing statistical procedures, and laboratories were awarded z-scores on the basis of their reported results. Based on a target standard deviation (sigmap value) taken from the Horwitz equation, for a robust mean value of 1.2 mg/kg acrylamide, satisfactory results (z-score within +/- 2 for those between 0.8 and 1.6 mg/kg) were obtained by 86% of the 37 laboratories that returned results. Only 1 laboratory was unsatisfactory and 4 had questionable results. About equal numbers of laboratories used gas chromatography (GC)/MS and LC/MS procedures with about 25% using MS/MS and one using GC with electron capture detection. There was no evident trend in performance or bias in results. GC/MS and LC/MS data were evenly distributed across the population of laboratories reporting results.     

Probabilistic comparison and assessment of proficiency testing schemes and laboratories in the somatic cell count of raw milk

The somatic cell count (SCC) of milk is one of the main indicators of the udder health status of lactating mammals and is a hygiene criterion of raw milk used to manufacture dairy products. An increase in SCC is regarded as one of the primary indicators of inflammation of the mammary gland. Therefore, SCC is relevant in food legislation as well as in the payment of ex-farm raw milk and it has a major impact on farm management and breeding programs. Its determination is one of the most frequently performed analytical tests worldwide. Routine measurements of SCC are almost exclusively done using automated fluoro-opto-electronic counting. However, certified reference materials for SCC are lacking, and the microscopic reference method is not reliable because of serious inherent weaknesses. A reference system approach may help to largely overcome these deficiencies and help to assure equivalence in SCC worldwide. The approach is characterised as a positioning system fed by different types of information from various sources. A statistical approach for comparing proficiency tests (PTs) by assessing them using a quality index P _Q and assessing participating laboratories using a quality index P _L, both deriving from probabilities, is proposed. The basic assumption is that PT schemes are conducted according to recognised guidelines in order to compute performance characteristics, such as z-scores, repeatability and reproducibility standard deviations. Standard deviations are compared with the method validation data from the ISO method. Input quantities close to or smaller than the reference data of the method validation or the assigned value of the PT result in values for P _Q and P _L close to the maximum value. Evaluation examples of well-known PTs show the practicability of the proposed approach.     

Establishing the standard deviation for proficiency assessment ( \hat{\sigma } ). in microbiology PT/EQA schemes

Microbiological testing can be challenging due to the unique nature of microorganisms. Studies have shown that values for reproducibility in microbiology are difficult to establish as they can vary depending upon a number of factors such as the organism under test, the matrix and the test method. The relative lack of consistent data on reproducibility means it can be difficult for providers of microbiology PT schemes to establish a suitable standard deviation for proficiency assessment ( $ \hat{\sigma } $ ). Methods for establishing $ \hat{\sigma } $ are described in BS ISO 13528: 2005, but these are not all appropriate in microbiology, given the lack of available data and the large range of possible variables. Using the robust standard deviation of participant results is not recommended as this can vary greatly from round to round and means that performance is not comparable over time. Use of a fixed standard deviation depends very much on the perception of the PT scheme organiser as to what constitutes fitness-for-purpose. In microbiology, is it really necessary to obtain precise results? Should the emphasis be on the correct isolation and identification of organisms, and on performance over time to detect laboratory bias rather than on enumeration results alone?     

New Insights on Non-Enzymatic Oxidation of Ganglioside GM1 Using Mass Spectrometry

Gangliosides are acidic glycosphingolipids that are present in cell membranes and lipid raft domains, being particularly abundant in central nervous systems. They participate in modulating cell membrane properties, cell–cell recognition, cell regulation, and signaling. Disturbance in ganglioside metabolism has been correlated with the development of diseases, such as neurodegenerative diseases, and in inflammation. Both conditions are associated with an increased production of reactive oxidation species (ROS) that can induce changes in the structure of biomolecules, including lipids, leading to the loss or modification of their function. Oxidized phospholipids are usually involved in chronic diseases and inflammation. However, knowledge regarding oxidation of gangliosides is scarce. In order to evaluate the effect of ROS in gangliosides, an in vitro biomimetic model system was used to study the susceptibility of GM1 (Neu5Acα2-3(Galβ1-3GalNAcβ1-4)Galβ1-4Glcβ1Cer) to undergo oxidative modifications. Oxidation of GM1 under Fenton reaction conditions was monitored using high resolution electrospray ionization-mass spectrometry (ESI-MS) and tandem mass spectrometry (ESI-MS/MS). Upon oxidation, GM1 underwent oxidative cleavages in the carbohydrate chain, leading to the formation of other gangliosides GM2 (GalNAcβ1-4Gal(Neu5Acα2-3)1-4Glcβ1Cer), GM3 (Neu5Acα2-3Galβ1-4Glcβ1Cer), asialo-GM1 (Galβ1-3GalNAcβ1-4Galβ1-4Glcβ1Cer), asialo-GM2 (GalNAcβ1-4Galβ1-4Glcβ1Cer), of the small glycolipids lactosylceramide (LacCer), glucosylceramide (GlcCer), and of ceramide (Cer). In addition, oxygenated GM1 and GM2 (as keto and hydroxy derivatives), glycans, oxidized glycans, and oxidized ceramides were also identified. Nonenzymatic oxidation of GM1 under oxidative stress contributes to the generation of other gangliosides that may participate in the imbalance of gangliosides metabolism in vivo, through uncontrolled enzymatic pathways and, consequently, play some role in neurodegenerative processes.

Open image in new window

Graphical Abstract ?

     

LC with a Teicoplanin Aglycone Chiral Sorbent for the Separation of the Enantiomers of Non-Steroidal Anti-Inflammatory Drugs: An Evaluation of Chiral Capillary Columns

A capillary column with a teicoplanin aglycone (TAG) stationary phase (CSP) was used for enantioselective separation of selected profen non-steroidal anti-inflammatory drugs in capillary liquid chromatography (cLC). The effect of variations in the mobile phase composition on the retention and enantioselective separation was examined. The best resolution was attained in the mobile phase composed of 40/60 (v/v) methanol/1.0% triethylamine acetate buffer, pH 4.0 or 4.5. Under the optimized separation conditions, five of the set of eight analytes were enantioresolved with resolution values better than 0.9. Only fenoprofen was not enantioseparated in any system tested. The optimized separation conditions were used for evaluation of three chiral capillary columns (all prepared in the same way in our laboratory) in terms of the repeatability and reproducibility of the results. The run-to-run repeatability was expressed in terms of the relative standard deviation (RSD) values, obtained from ten independent measurements, for the following parameters: the retention factor for the first eluted enantiomer (k ₁), the selectivity (α), the enantioresolution (R), the theoretical plate count per meter for the first eluted enantiomer (N ₁) and the elution curve asymmetry for the first eluted enantiomer (As ₁). None of the RSD values exceeded 8%. The column-to-column reproducibility of these parameters ranged between 1 and 9%. The results obtained with TAG based CSPs in cLC (a laboratory packed capillary column) were compared with those obtained by classical high-performance liquid chromatography (HPLC) with a commercially available column. The cLC procedure provided a better enantioresolution and the elution curves had a better symmetry.