Direct importance estimation for covariate shift adaptation |
| |
Authors: | Masashi Sugiyama Taiji Suzuki Shinichi Nakajima Hisashi Kashima Paul von Bünau Motoaki Kawanabe |
| |
Affiliation: | 1.Department of Computer Science,Tokyo Institute of Technology,Meguro-ku, Tokyo,Japan;2.Department of Mathematical Informatics,The University of Tokyo,Bunkyo-ku, Tokyo,Japan;3.Nikon Corporation,Kumagaya, Saitama,Japan;4.IBM Research,Tokyo Research Laboratory,Yamato, Kanagawa,Japan;5.Department of Computer Science,Technical University Berlin,Berlin,Germany;6.Fraunhofer FIRST.IDA,Berlin,Germany |
| |
Abstract: | A situation where training and test samples follow different input distributions is called covariate shift. Under covariate shift, standard learning methods such as maximum likelihood estimation are no longer consistent—weighted variants according to the ratio of test and training input densities are consistent. Therefore, accurately estimating the density ratio, called the importance, is one of the key issues in covariate shift adaptation. A naive approach to this task is to first estimate training and test input densities separately and then estimate the importance by taking the ratio of the estimated densities. However, this naive approach tends to perform poorly since density estimation is a hard task particularly in high dimensional cases. In this paper, we propose a direct importance estimation method that does not involve density estimation. Our method is equipped with a natural cross validation procedure and hence tuning parameters such as the kernel width can be objectively optimized. Furthermore, we give rigorous mathematical proofs for the convergence of the proposed algorithm. Simulations illustrate the usefulness of our approach. |
| |
Keywords: | Covariate shift Importance sampling Model misspecification Kullback– Leibler divergence Likelihood cross validation |
本文献已被 SpringerLink 等数据库收录! |
|