The multivariate coefficient of variation for comparing serum protein electrophoresis techniques in external quality assessment schemes

External quality assessment (EQA) schemes are national or transnational programmes designed to control the analytical performance of clinical laboratories and to maintain inter-laboratory variability within acceptable limits. In such EQA programmes, participants are usually grouped by the type of assay technique/equipment they use. The coefficient of variation (CV) is a simple tool for comparing the inter-laboratory reproducibility of such techniques: the lower the CV, the better the analytical performance. Serum protein electrophoresis, a laboratory test profile consisting of five fractions (albumin, α ₁, α ₂, β and γ globulins) summing up to 100% of total proteins, can also be assayed in different ways depending on the media or the analytical principle. We propose a multivariate CV for comparing the performance of electrophoretic techniques in EQA, thus extending the univariate CV concept. First, the compositional nature of electrophoretic data requires a one-to-one transformation from the five-dimensional to the four-dimensional space. Next, robust estimations of the mean and the covariance matrix are needed to avoid the effect of outliers. The new approach is illustrated on electrophoretic datasets from the French and Belgian national EQA programmes.     

Statistical analysis of serum protein electrophoresis results in External Quality Assessment schemes

The goal of External Quality Assessment (EQA) schemes is to ensure that results obtained on a particular specimen in a given clinical laboratory are compatible with those obtained by other laboratories on the same specimen. Serum protein electrophoresis is a laboratory test consisting of five fractions (albumin, α₁, α₂, β and γ globulins), which sum up to 100% of total proteins. So far, in EQA schemes the five fractions have been analyzed separately as for ordinary tests like glucose or cholesterol. This approach does not consider the fractions as a whole and the linear relationship between them. A statistical approach has been developed to analyze EQA electrophoresis results from a global standpoint by using robust multivariate method to eliminate the effect of outlying profiles. As illustrated on electrophoretic data from the Belgian EQA scheme, the novel approach improves the detection of poor performing laboratories. The method will be implemented in the Belgian EQA scheme on a routine basis.     

The effects of quality-management systems on external quality-assessment performance in Finnish clinical chemistry laboratories

The effects of the implementation of quality-management systems and technological changes on the performance of 23 Finnish clinical chemistry laboratories were studied in external quality-assessment (EQA) schemes organized by Labquality Ltd., Finland, during 1993 to 1999. The investigated serum analytes were sodium, calcium, glucose, cholesterol, and alanine aminotransferase. According to the results, the improvements in analytical quality due to the quality programs were rather small. The effects to the proportions of satisfactory results in terms of precision and deviation from the target value were 5–15% (95% confidence intervals for the proportions ±1–5%). At the same time the laboratories were implementing quality-management systems new technology and characteristics of quality-control materials affected the EQA performance. Both improvements and deteriorations in analytical quality due to these factors were identified. The effects and their total outcomes differed substantially between the five analytes.     

External quality assessment schemes for clinical laboratories

 The clinical routine laboratory generally utilizes cheap, easy and rapid measurement procedures ("methods") in order to meet the requirement for the production of many analytical results (500–3000 per day) of hundreds of different types. The measurement procedures are optimized for the analysis of native patients' samples, but are frequently sensitive to deviation of the composition of the matrix from that in normal fresh samples. The inherent lack of stability of patient samples means that control samples need to be stabilized. The method of stabilization is critical. Furthermore, the method of "spiking" samples with pathological material is a matter of concern. Generally, minimally processed patients' samples should be used in external quality assessment (EQA) schemes. Consensus values are currently the most popular for use as a guide to the best results from participating laboratories in EQA schemes; these often work fairly well. However, the uncertainty and traceability of this type of value is unknown, and in some cases may even be misleading, tending to preserve bad routine methods when these are dominant in the participating laboratories. Reference measurement procedure (RMP) values are recommended to provide scientifically based information, to facilitate the proper choice of methods in the routine laboratories, and to validate the suitability of control materials in EQA schemes. The present paper provides selected examples from a study comparing consensus values with RMP values on lyophilized sera, and also presents results on a fresh frozen thawed serum for the study of commutability. Received: 8 November 1995 Accepted: 8 May 1996     

External quality assessment in microbiology: comparison of results from Belgian and Canadian laboratories with regard to their ability to identify <Emphasis Type="Italic">Streptococcus pyogenes</Emphasis>

Clinical Microbiology Proficiency Testing (CMPT; Canada) and the Institute of Public Health (IPH; Belgium) set up a joint external quality assessment (EQA) project in which they sent identical simulated clinical samples to their respective participants. Samples were sent out as throat swabs containing Streptococcus pyogenes (as pathogen) and viridans streptococci (as commensal). Results from identification by the Belgian laboratories were excellent: 99.5% detected the group A streptococci. About 10% of the Canadian laboratories reported the absence of reportable β-haemolytic streptococci, although for most of them the issue was one of non-examination of small or pin-point streptococci. Overall, Canadian and Belgian laboratories performed well in this EQA; the joint project clearly showed that although many similarities exist, there are differences between laboratories in treating EQA samples and reporting results.     

Twenty years of the Me.Tos. Project: an Italian national external quality assessment scheme for trace elements in biological fluids

The Me.Tos. Project, started in 1983 and still running, is an external quality assessment (EQA) scheme for laboratories performing specialized analyses in occupational and environmental laboratory medicine. Besides the organization of EQA exercises, initiatives for further education of the participants and the harmonization of EQA procedures at a European level are carried out. Participation in EQA schemes allows laboratories to comply with the international standards for the quality and competence of testing and clinical laboratories. The organization of the scheme includes the preparation of control materials, their distribution to the participants, according to strategies aimed to avoid identification of the samples, the statistical analysis of the results and the evaluation of laboratories' performance according to international guidelines and criteria set by the organizers. An overview of the scheme operation and the current performances of participants will be given.     

External quality assurance programmes in medical laboratories

 Medical laboratories have a long tradition of external quality assessment. Starting from pure quality control of laboratory performances, most schemes have evolved to a powerful tool for improving quality of clinical outcome of results. External quality assurance in medical laboratories not only includes laboratory performance evaluation, but also evaluation of method performance, post-marked vigilance, training and help. In the future, the quality of programmes must further be improved by accreditation of schemes and by using electronic data interchange. Received: 9 December 2000 Accepted: 14 December 2000     

Monitoring performance of nucleic acid-based diagnostic measurement system users by EQA

External quality assurance (EQA) programmes allow a laboratory to monitor independently its performance and provide feedback to identify and investigate potential areas of concern. However, both the definition of appropriate ‘performance indicators’ and their statistical analysis vary across EQA providers. Typically, performance scores have an unknown statistical distribution and hence arbitrary thresholds are used to measure performance. In this paper, we introduce a scoring system based on well-known statistical principles. The system is simple, flexible and easy to interpret. It can be used to measure performance for single samples or across a panel so providing useful and meaningful information to participants in EQA programmes. The score is illustrated using the 2005 QCMD Hepatitis B Virus Proficiency programme.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the Eurachem Workshop, Rome, 5–7 October 2008 is provided. The eight WG’s covered a range of issues concerned with current practice and future directions; how frequently should laboratories participate in PT/EQA? (WG1); developments in PT/EQA within the EU—what is required in future? (WG2); what issues do developing countries face with regards to PT/EQA? (WG3), what issues are specific to microbiology PT/EQA? (WG4); what new fields are emerging for PT/EQA? (WG5); what will be the impact of the new ISO/IEC 17043 standard? (WG6); do current PT/EQA schemes meet the needs of participants? (WG7); and what are the issues that affect the quality of proficiency test items? (WG8). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

National External Quality Assessment Schemes for microbiology, parasitology, and virology in Europe

Quality assurance is an important aspect of laboratory management. One of the activities involved is the participation in external quality assessment (EQA) schemes by the clinical laboratory. These EQA schemes should be organised according to well-defined guidelines, such as the ESO/IEC Guide 43. The present work provides an inventory of the EQA schemes organised in Europe. The schemes are related to the fields of bacteriology, parasitology and virology. For each field various surveys have been organised. Data such as the number of participants, frequency of survey and number of samples in a survey are presented. The nature and way of manufacturing the control material is specified.     

The results of HbA1c measurement and its comparison with reference method values in an EQA programme

This article describes the effect of a change in reference method for HbA_1c measurement on results in our EQA programme. Important indicators of analytical quality, particularly reproducibility, bias and uncertainty, were assessed. Basic analytical quality indicators for the measurement of HbA_1c in an EQA survey during 2000–2008 were evaluated. Three EQA surveys per year were performed. Usually there were more than 200 participating laboratories in each survey. Reproducibility, systematic differences, bias, and methodology changes before and after IFCC reference method implementation were determined. In the EQA surveys, target values traceable to the IFCC reference method are used. Estimation of the combined uncertainty of the result was a voluntary part of the surveys and was reported by 50% of participating laboratories. Reproducibility of all participants’ measurements was close to the target value of CV = 5.0% (coefficient of variation). The group of participants using HPLC methodology had reproducibility lower than 5.0% in each survey, whilst the immunochemistry group usually had CV values more than 6.0%. The differences in individual results from target values ranged in the interval ±12% for HPLC methods but more (−8 to +30%) for immunochemistry methods. During the monitored time interval the number of participants using HPLC methods increased up to 70% whilst the number of participants using immunochemistry methods decreased approximately to 30%. The combined uncertainty (u _c ) reported by participants varies between 4.0 and 4.5% for HbA_1c values between 30 and 90 mmol/mol. No significant differences between precision and systematic errors (bias) after the implementation of IFCC method of measurement for HbA_1c to routine processes for HPLC were observed. Immunochemical methods do not fulfil requirements for analytical reproducibility.     

Proficiency testing in analytical chemistry, microbiology and laboratory medicine: working discussions on current practice and future directions

A summary of the working group (WG) discussions on proficiency testing (PT) and external quality assessment (EQA) held at the EURACHEM Workshop, Istanbul, 3?C6 October 2011, is provided. The six WGs covered a range of issues concerned with current practice and future directions; implementing the requirements of ISO/IEC 17043 by PT/EQA providers (WG1); accrediting PT/EQA providers to ISO/IEC 17043 (WG2); pre- and post-analytical aspects in PT/EQA (WG3); evaluating participant performance in qualitative PT/EQA schemes (WG4); establishing PT/EQA schemes in developing countries (WG5); and establishing acceptability criteria in microbiology PT/EQA schemes (WG6). Delegates with different backgrounds were on each WG in order to capture a range of views and experience from a number of different sectors. Working group representatives included PT/EQA providers, participants in PT/EQA schemes and end-users of PT results such as accreditation bodies and regulatory authorities, from countries around the world.     

Requirements vs acceptability in proficiency testing schemes and other interlaboratory comparisons

Many laboratories take part in proficiency testing schemes, external quality assessment programmes and other interlaboratory comparisons. These have many similarities but also important differences in their modus operandi and evaluation of performance of participating laboratories. This paper attempts to highlight both the similarities and differences. It also puts particular emphasis on requirements called ”target values for uncertainty” and their meaning. Received: 24 January 2001 Accepted: 25 January 2001     

Twenty-year review of laboratory performance on enteric parasitology external quality assessment surveys 1992–2011, Ontario Canada

Quality Management Program?CLaboratory Services (QMP?CLS) is a mandatory external quality assessment (EQA) and accreditation programme for laboratories in Ontario. This report summarizes performance of Ontario laboratories on QMP?CLS enteric parasitology EQA surveys and compares this performance over time. The results reported by participating laboratories were assessed against the expected results as determined by reference or referee laboratories. The number of laboratories licensed for parasitology decreased dramatically over the 20?years. On average, more than 80?% of participants and often more than 90?% were able to detect and identify the species and stages for the majority of parasites. Many of the challenges contained multiple parasites and about half contained low numbers, which affected performance on individual samples. The level of performance by Ontario laboratories on parasitology EQA surveys was consistently high over the 20?years. The parasites that presented the most difficulty were Enteromonas hominis and Trichomonas hominis.     

Quality management, certification and accreditation in medical laboratories – the view of ECLM

 Increasing demands from health care planners and industrialists conducting clinical trials, as well as general competition, are forcing medical laboratories to seek third-party recognition of their quality management systems. There is a tendency to move from certification of a laboratory director, via certification of the laboratory quality system (ISO 9000 family), to accreditation needing proof of professional and technical competence in laboratory tasks. The requirements of accreditation are presented in several national schemes and in the European Standards series (EN 45 000) and the International Organization for Standardization's guide, ISO/IEC 25, to be amalgamated soon. The latter system provides transnational recognition through participation of the accrediting bodies in the European co-operation for Accreditation. Necessary supplementary guidelines exist for chemical laboratories (Eurachem) and medical laboratories CEAC/ECLM). Traceability and reliability of results are obtained by utilizing a global reference examination system and by participating in transdisciplinary work. The costs of achieving accreditation are considerable and mainly involve the production of quality handbooks and written work procedures by personnel. The rewards are an open system, smoother work, emphasis on prevention of mistakes, and satisfied stakeholders. Received: 5 October 1998 · Accepted: 20 October 1998     

Accreditation of proficiency-testing schemes in The Netherlands

Proficiency testing by laboratories, national accreditation bodies, and other third parties is becoming more and more considered as a standard and integral part of the quality control system. Therefore, it is of utmost importance that the quality of the provided proficiency-testing (PT) service is outstanding. If PT-schemes are set up in order to help laboratories monitor and improve their quality, PT-schemes need not only be of high quality themselves, but the organizer also needs to be able to demonstrate this. In The Netherlands formal accreditation of the organization of proficiency testing schemes is used as a tool to guarantee high quality schemes and also to enable organizers to demonstrate their competence. Since 1996, the Dutch Council for Accreditation (RvA) has used the ISO-Guide 43–1 to assess PT-organizers in The Netherlands. From January 2000, the ISO-guide 43–1 was replaced by the ILAC G13 document for assessing organizers. Up till now, four institutes have been accredited by the RvA for the organization of PT-schemes.     

The Swiss External Quality Assessment Scheme in Bacteriology and Mycology 1992–1996

 The Swiss External Quality Assessment Scheme in Bacteriology and Mycology was created in 1980 and has been organised since 1983 by the Department of Medical Microbiology in Zurich. The number of Swiss participants has steadily risen from 66 in 1989 to 92 in 1996. Twelve bacterial and fungal strains are sent to the participants in four despatches, each containing three specimens, per year. Scores are allocated per specimen and range between 0 and 1. Participants with mean scores of ≤0.75 are considered poor performers. The mean scores increased from 0.85 in 1992 to 0.91 in 1996. This improvement can be attributed to the educational effect of the external quality control scheme, since all participants receive a detailed discussion for each specimen together with their individual results. On average, both large University and Cantonal (state) laboratories as well as private laboratories show satisfactory performance. In particular, laboratories officially recognised by the Swiss Federal Office of Public Health (SFOPH) rate better than non-recognised participants. Many small regional hospital laboratories, most of them not SFOPH-recognised, are often among the poor performers. They are often managed by technical staff and lack a trained microbiologist. The recently introduced legislation in Switzerland renders participation in external quality assessment schemes compulsory, and all clinical microbiology laboratories are required to employ qualified microbiologists. This will certainly help to improve the quality standards of all laboratories performing microbiological tests. Received: 13 November 1997 · Accepted: 28 December 1997     

Do we need to accredit proficiency testing schemes?

 Proficiency testing (PT) is being increasingly used as an important quality assurance tool for laboratories. The subject of quality of the providers of PT schemes has been discussed increasingly in recent years. Some countries have implemented systems for the accreditation of PT schemes. This paper looks at the background to the accreditation of PT schemes, the likely mechanisms which could be employed for accreditation, and some of the practical aspects.     

A dedicated External Quality Assessment system for immunoassays of reproductive hormones

A dedicated External Quality Assessment (EQA) system for immunoassays of reproductive hormones (LH, FSH, prolactin, hCG, progesterone, estradiol, testosterone) and cortisol was developed. This catered for the needs of a group of laboratories conducting collaborative research with the WHO in the field of human reproduction. This system was designed to provide the maximum information on assay quality with the final aim of achieving and maintaining the highest possible comparability in monitored multi-center clinical studies. The EQA was based on the assays of 32 control samples per year in 4 quarterly assay runs and the evaluation of the following parameters: bias, between-run imprecision within a laboratory, number of outliers, frequency of returned assays of control samples, and between-laboratory imprecision. The data obtained were combined in a performance index for each laboratory, EQA run, and analyte. This index was then subjected to ranking. It may be assumed that the EQA system contributed to improvements of analytical work and/or the maintenance of a high assay quality in participating laboratories. Received: 26 July 2001 Accepted: 9 November 2001     

Long-term external quality assessment program for CD4+ T-lymphocyte enumeration in Thailand

The accuracy and reliability of CD4+ T-lymphocyte enumeration are crucial for HIV healthcare management. Thailand’s CD4 external quality assessment (EQA) program was implemented in 2003. This program aims to improve the quality of CD4 testing by providing the quality control material, remedial action, and technical training. In this article, the overall longitudinal performance of the participating laboratories from 2003 to 2015 is reported. The EQA blood samples were sent to the participating laboratories in Thailand and other Southeast Asian (SEA) countries. The participants were requested to enumerate CD3+ and CD4+ T-lymphocytes. The trimmed mean, standard deviation (SD), coefficient of variation (CV), and standard deviation index (SDI) of both fraction and number were calculated from the returned data. Our result indicated a continuously increasing number of participants. The response rate of 72 trials was 98 %. The outlier rates for the fraction and number of CD4+ T-lymphocytes were 3.3 % and 3.4 %, respectively. The average CV of the fraction and number of CD4+ T-lymphocytes was 7.1 % and 9.4 %, respectively. In addition, the analysis of our current EQA trial no. 072 revealed that more than 98 % of participating laboratories had SDI values between ?2 and +2 for both fraction and number of CD3+ and CD4+ T-lymphocytes. In conclusion, the implementation of the CD4 EQA program in Thailand and other SEA countries has led to a reduction in the variability of the results of the CD4 count between laboratories and a continuous improvement in laboratory performance.