Decision threshold adjustment in class prediction |
| |
Authors: | J J Chen C-A Tsai H Moon H Ahn J J Young C-H Chen |
| |
Institution: | 1. Division of Biometry and Risk Assessment , National Center for Toxicological Research, Food and Drug Administration , Jefferson, Arkansas 72079jchen@nctr.fda.gov;3. Institute of Statistical Science , Academia Sinica, Taipei, 11529, Taiwan;4. Division of Biometry and Risk Assessment , National Center for Toxicological Research, Food and Drug Administration , Jefferson, Arkansas 72079;5. Department of Applied Mathematics and Statistics , Stony Brook University , Stony Brook, NY, 11794 |
| |
Abstract: | Standard classification algorithms are generally designed to maximize the number of correct predictions (concordance). The criterion of maximizing the concordance may not be appropriate in certain applications. In practice, some applications may emphasize high sensitivity (e.g., clinical diagnostic tests) and others may emphasize high specificity (e.g., epidemiology screening studies). This paper considers effects of the decision threshold on sensitivity, specificity, and concordance for four classification methods: logistic regression, classification tree, Fisher's linear discriminant analysis, and a weighted k-nearest neighbor. We investigated the use of decision threshold adjustment to improve performance of either sensitivity or specificity of a classifier under specific conditions. We conducted a Monte Carlo simulation showing that as the decision threshold increases, the sensitivity decreases and the specificity increases; but, the concordance values in an interval around the maximum concordance are similar. For specified sensitivity and specificity levels, an optimal decision threshold might be determined in an interval around the maximum concordance that meets the specified requirement. Three example data sets were analyzed for illustrations. |
| |
Keywords: | Concordance Cross validation Receiver operating characteristic curve Sensitivity and specificity Weighted k-NN |
|
|