Proficiency testing: binary data analysis

A method for evaluating qualitative proficiency testing (PT) of laboratories conducting binary tests is proposed. The method is based on the scale-invariant item response model proposed by the authors in earlier publications. We consider the case where the laboratories under the PT conduct test consisting of a set of test items/species presenting different, but unknown beforehand levels of difficulty when trying to detect a particular property of theirs, and we need to evaluate/compare both the intrinsic abilities of the participating laboratories and the level of difficulty of the test items. We assume that the responses to different test items do not affect one another and discuss how to get and interpret the most likely estimates/scores. The method is illustrated by the example presented in a recent publication by our colleagues from QuoData GmbH and can be considered as an alternative to that proposed in their publication method of scoring.