Proficiency testing for the improvement of analytical practice

The results of an individual laboratory??s participation in proficiency testing are often taken in isolation. A single poor assessment may be investigated, or a trend of participation over time may be charted. However, the overall results of proficiency testing (over all participants and over time) may also provide some insights into aspects of the analysis being undertaken. Two examples are summarised here. Analysis of sodium in various foods appears to be difficult with no obvious method trends. Analysis of vitamin B₂ in liquid dietary supplement requires enzymatic dephosphorylation, as well as acidic digestion. In the latter case, some participants appear to have changed their method since the first reporting of the problem. Investigation of proficiency testing results and the implications for the analysis takes time and requires data sets to be retrieved from archive. This may not be the highest priority in the workload of a busy proficiency testing provider. However, the benefits to the analytical community from such investigations are great, and their reporting is to be encouraged.     

能力验证计划中防止数据串通的措施

为保证能力验证结果评价有效、可信、公平,对能力验证计划防止数据串通的措施进行了探讨。从样品设计、样品量发放、检测周期3个方面介绍了防止数据串通的措施,以CNAS PT0017–T029"离子色谱法测定水中Cl~–,SO_4^(2–)及NO_3~–三种阴离子的含量"能力验证计划为例,讨论能力验证计划防止数据串通措施的具体实施方法。结果显示,设计分割水平样品、减少样品发放量、缩短能力验证结果上报周期等几种措施可以有效地防止能力验证数据的串通,为能力验证活动的健康发展提供了参考。     

Bias analysis of proficiency testing programme results

A proficiency testing programme might involve a great number of participating laboratories coming from different countries or regions, and normally they analysed the same test materials using their own routine analytical methods. Hence, the results of a proficiency testing programme may contain valuable information which could serve purposes other than just performance evaluation. This study attempted to extract information from the results of a proficiency testing programme for the purposes of educating the participating laboratories as suggested by ISO/IEC 17043. The “bias analysis” approach introduced in this study was based on the statistical model of measurement and the nature of bias in chemical analysis. With this approach, the participating laboratories could estimate the bias associated with different settings of experimental conditions according to the statistics of subset distribution of the reported results from the participating laboratories. This would be useful for them to review the analytical procedures they used and modify their methods if needed. The approach was applied to the analysis of data obtained from a number of past proficiency testing programmes, and the findings were discussed in this paper.     

Blood lead proficiency testing: overview of the federally sponsored program in the U.S.

External proficiency testing is a proven method for improving the quality of laboratory measurements, including lead in blood. Several proficiency testing programs exist to fill this need. This article describes the U.S. federally sponsored Blood Lead Proficiency Testing Program. Three specimens, composed of blood from dosed cows, are distributed to participants each month. Target values are determined by a group of 10 referee laboratories. Participant performance has shown steady improvement over time.     

Quality control procedures for chloride and nitrate ions analysis in plant samples by ion chromatography

This paper describes useful procedures to monitor quality of chloride and nitrate ions analysis in plant samples by ion chromatography. The use of certified reference materials (CRMs) provided an efficient way to verify the accuracy of the method. Data generated by the method of analysis for chloride compared favourably with certified values. The quality system included also the systematic analysis of an internal reference sample in each batch of samples routinely analysed. The performance of the method, including extraction and measurement, over a period of 3 years was reported with control charts. The yearly variation coefficients were less than 6.5% for chloride and nitrate ions. Finally, the analytical method was evaluated through the participation of laboratory to an international proficiency testing scheme. Found results were not significantly different from published medians.     

Proficiency testing programmes as a tool in food quality assurance: Overview of Italian experiences

Proficiency testing involves the performance of test procedures on routine samples by a number of laboratories. Interlaboratory proficiency testings provide multiple benefits to participants since they play a key-role in the total quality control of laboratory activities. They serve as a means of self-improving, as a mechanism of continuing education and as a source of information for accreditation agencies. This review highlights basic principles, benefits, criteria and capabilities of a proficiency testing programme for food analysis laboratories as well as their role in the implementation of rapidly developing food control legislation.     

Electrochemical detection of nitrate,nitrite and ammonium for on-site water quality monitoring

This review examines the most recent electrochemical developments for nitrate, nitrite and ammonium detection for on-site water monitoring. There remains a high demand for effective field-based detection of the dissolved inorganic nitrogen (DIN) analytes to aid in mitigating nitrogen loading. Electrochemical approaches show increasing potential to fill this role as advancements in nanotechnology continually improve analytical performance and on-site applicability. However, translating these improvements into the field still faces the resonating challenges of reaching analytical proficiency (selectivity, sensitivity, robustness, stability), practical end-user functionality, minimal matrix interferences and cost effectiveness. Herein, we elaborate on these challenges via a critical evaluation of current studies and examine how realistic the prospects of on-site nitrate, nitrite and ammonium are. We also present recommendations in addressing these gaps to conclude the review.     

An error model for evaluation and interpretation of proficiency testing

Data from proficiency testing can be used to increase our knowledge of the performance of populations of laboratories, individual laboratories and different measurement methods. To support the evaluation and interpretation of results from proficiency testing an error model containing different random and systematic components is presented. From a single round of a proficiency testing scheme the total variation in a population of laboratories can be estimated. With results from several rounds the random variation can be separated into a laboratory and time component and for individual laboratories it is then also possible to evaluate stability and bias in relation to the population mean. By comparing results from laboratories using different methods systematic differences between methods may be indicated. By using results from several rounds a systematic difference can be partitioned into two components: a common systematic difference, possibly depending on the level, and a sample-specific component. It is essential to distinguish between these two components as the former may be eliminated by a correction while the latter must be treated as a random component in the evaluation of uncertainty. Received: 20 November 2000 Accepted: 3 January 2001     

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.     

Integration of metrological principles and performance evaluation in a proficiency testing scheme in support of the Council Directive 98/83/EC

Proficiency testing as a means of external quality assessment plays the role of independent evidence of laboratories’ performance. To enable laboratories to fulfil the requirements stated in legislation, methodology for evaluation of laboratories’ performance in proficiency testing schemes should incorporate principles of measurement results which are fit for intended use and incorporate evaluation of laboratories’ performances based on independent reference value. A proficiency testing scheme was designed to support Drinking Water Directive (98/83/EC) specifically. The methodology for performance evaluation, which takes into account a “fitness for purpose”-based standard deviation for proficiency assessment, is proposed and discussed in terms of requirements of the Drinking Water Directive. A ζ′-score, modified by application of target uncertainty was developed in a way that fulfils requirements defined in the legislation. As an illustration, results are reported for nitrate concentration in water. The approach presented can also be applied to other fields of measurements.     

Improvements in performance in medical diagnostics tests documented by interlaboratory comparison programs

There is evidence to support the notion that interlaboratory comparisons (ILCs) are an effective tool for laboratory improvement. However, despite widespread experience and anecdotal evidence of improvements there are few published studies demonstrating any benefits from ILCs– in any field of testing. Published demonstrations of benefits can help justify the growing use of ILCs. ILCs and proficiency testing have been common for many years in medical laboratories; there has been open information on the results of ILCs, and there has been standardization of results from thousands of laboratories. These studies show general improvement over time in several areas of testing in different countries. Many articles cite specific reasons for the improvements, either proven or supposed. An early version of this paper was presented at the International Laboratory Accreditation Cooperation Conference ”ILAC 2000” in Washington D.C., on 31October, 2000. Received: 10 February 2001 Accepted: 21 January 2002     

水中易释放氰化物能力验证样品的研制及应用

根据国家地表水环境质量监测网监测任务要求和环境监测的实际需要,制备了水中易释放氰化物能力验证样品。通过均匀性、稳定性检验以及量值一致性评价,研制的样品均匀性良好,在3℃~6℃冷藏避光保存条件下1年内稳定,样品配制值与多家实验室协作测定结果一致。探讨了样品在能力验证活动中的应用,共有来自全国11个省的32家实验室参加了水中易释放氰化物的能力验证计划,实验室满意率在80%以上,实验室结果出现有问题或不满意主要是由于样品前处理以及检测过程质量控制不当导致的。经检测及实验室反馈的数据验证,该能力验证样品能够应用于能力验证活动。     

Comparison of different approaches to the statistical evaluation of proficiency tests

The objective of this study was to compare different z-score calculation for the results from proficiency testing programmes. The comparison was carried out on the data from proficiency tests for four different matrices and for all analytes included in the proficiency testing programmes organised by UKZUZ institute from the year 2005 to 2007. It represented 99 samples (three periods per year and eleven samples distributed in each period); the number of determined analytes was 108 in each period. The total number of investigated data sets was 953. Three different approaches to z-score calculation were compared. The results gave reasonably comparable evaluation in most cases but sometimes UKZUZ method proved to be rather strict. The procedure using the robust average and robust standard deviation calculated according to Algorithm A described in ISO 13528 was found to be the most suitable for matrices and level of measurands in UKZUZ proficiency tests.     

Review of the new edition of ISO 13528

The international standard ISO 13528 is an important reference document for proficiency testing providers and testing laboratories alike, which provides guidance on the use of statistical methods in proficiency testing. With a view to bringing the document in harmony with the ISO/IEC 17043:2010, the first edition of ISO 13528 was revised and the new edition was published in September 2015. This paper attempted to review and discuss the major changes introduced in the new edition of ISO 13528, particularly the part on statistical design with respect to the objective of proficiency testing scheme and the considerations on assigned value determination and performance scoring. Moreover, procedures provided in the new edition of ISO 13528 for qualitative proficiency testing schemes and various robust statistical methods would also be briefly discussed in the paper.     

Application of a new AMIQAS computer program for integrated quality control, method evaluation and proficiency testing

Summary The computer program AMI - Quality Assessment Scheme (AMIQAS) was designed to give easy statistical management of analytical data in relation to method evaluation, internal quality control and proficiency testing. The program is designed in accordance with the recommendation of the ISO 5725 guideline and other official recommendations. The program consists of a database, test for outliers, statistics and graphs for evaluation of an analytical method for chemical, biological and environmental analysis, different control charts to be used in day-to-day internal quality control (Shewhart charts and z-charts). Finally the program has facilities to perform proficiency testing (also called external quality control) and produce the relevant evaluation reports for the documentation of method performance and quality control of results of measurements. The present paper demonstrates the applicability of the AMIQAS program for integrated method evaluation, internal quality control and proficiency testing using the determination of the concentration of lead in human whole blood as the numerical example.     

Traceability and its role in interlaboratory comparisons (proficiency testing programmes), modeled on trace element determination in biological materials

Traceability is not always evident in proficiency testing programs, although this is a requirement in ISO/IEC Guide?43-1. The assigned, or "true", value in most programs is not traceable to an independent entity. The test materials should generally be similar in nature to those routinely tested by participating laboratories. This is far from always the case and it is important to realize that if the difference is large, there may be no traceability to the testing program. It is also important that results from participation in proficiency tests are cited when papers are published, in order to enhance reliability/credibility of the published data.     

Regional cooperation in proficiency testing: strategy planning and practical experiences in Western Balkan countries

The article describes the project of regional cooperation in proficiency testing within CARDS 2006 (assistance programme to the Western Balkans), which was coordinated by European Committee for Standardization, European Commission and European Free Trade Association and conducted by the University of Ljubljana, Faculty of Electrical Engineering, Laboratory of Metrology and Quality. The later is the holder of the national standard for thermodynamic temperature as well as an accredited proficiency testing provider and also accredited as a calibration, testing and inspection laboratory. It is also responsible for graduate study programme Quality Engineering. Proficiency testing was considered as a core activity of the entire project, but additionally as a very appropriate platform for knowledge transfer of EU technical legislation in selected sectors, metrology, testing and quality assurance in general. The project had a high added value for participating laboratories in countries of Western Balkans, because it required an active role of all participating laboratories, assured participation of staff from national metrology institutions, all national accreditation bodies, and was supported by national quality infrastructures, such as standardization, governmental institutions etc. in the entire Western Balkans region. The activities stimulated progress in the area of conformity assessment, added knowledge to the implementation of technical directives and development of the technical part of national legislation and therefore facilitated the process toward the accession to the European Union. The main objective was therefore to organize proficiency testing in those sectors (construction, consumer protection and environment) that were vital in the pre-accession process of countries seeking full implementation of EU technical legislation.     

Traceability and its role in interlaboratory comparisons (proficiency testing programmes), modeled on trace element determination in biological materials

 Traceability is not always evident in proficiency testing programs, although this is a requirement in ISO/IEC Guide 43-1. The assigned, or "true", value in most programs is not traceable to an independent entity. The test materials should generally be similar in nature to those routinely tested by participating laboratories. This is far from always the case and it is important to realize that if the difference is large, there may be no traceability to the testing program. It is also important that results from participation in proficiency tests are cited when papers are published, in order to enhance reliability/credibility of the published data.

     

Harmonization and transferability of performance assessment: experience from four serum aluminum proficiency testing schemes

Proficiency testing schemes monitor laboratory performance and provide a stimulus for improvement in accuracy. Where several schemes operate in the same analytical sector, there are risks that assessments of performance may be in conflict. Performance assessment for the determination of trace elements such as aluminum in serum is particularly important due to the high risk of contamination and therefore erroneous results. The objectives of this work were (1) to compare several mathematical models to establish a predefined standard deviation for proficiency assessment and (2) to evaluate the influence of instrumental methods and proficiency testing scheme on the assessment of performance for serum aluminum measurements. For this purpose, three samples were sent to the participants of four proficiency testing schemes. Assigned values were calculated according to algorithm A according to ISO 13528 and standard deviation for proficiency assessment according to three methods based on individual variability, state of the art or previous proficiency testing results. The method based on individual variability produced a more stringent standard deviation compared to analytical imprecision based on the state of the art. The instrumental methods gave similar results, whereas significant differences were observed between the four proficiency testing schemes indicating that harmonization of the standard deviation for proficiency assessment fails to allow transferability from one proficiency testing scheme to another and that additional factor(s) contribute to variability in performance assessment.