Infrared and visible image fusion based on visual saliency map and weighted least square optimization |
| |
| Institution: | 1. School of Automation, Beijing Institute of Technology, Beijing 100081, China;2. Beijing Aerospace Automatic Control Institute, 100854 Beijing, China;1. Computer Vision and Systems Laboratory, Laval University, Quebec City, Quebec, G1K 7P4, Canada;2. Las.E.R. Laboratory, Dept. of Industrial and Information Engineering and Economics (DIIIE), University of L''Aquila, Monteluco di Roio - L''Aquila (AQ), 67100, Italy;3. Visiooimage Inc, Infrared Vision Systems, 2604 Lapointe, Quebec City, Quebec, G1W 1A8, Canada;1. Department of Electronic & Computer Engineering, Ngee Ann Polytechnic, 535 Clementi Rd, S599489, Singapore;2. Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Malaysia;3. Department of Biomedical Engineering, School of Science and Technology, SIM University, 599491, Singapore;1. Department of Physics, Xiamen University, Xiamen 361005, Fujian, People''s Republic of China;2. Department of Electrical and Computer Engineering, National University of Singapore, Singapore 119260, Singapore;3. Key Laboratory of Semiconductor Materials Science, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, People''s Republic of China |
| |
| Abstract: | The goal of infrared (IR) and visible image fusion is to produce a more informative image for human observation or some other computer vision tasks. In this paper, we propose a novel multi-scale fusion method based on visual saliency map (VSM) and weighted least square (WLS) optimization, aiming to overcome some common deficiencies of conventional methods. Firstly, we introduce a multi-scale decomposition (MSD) using the rolling guidance filter (RGF) and Gaussian filter to decompose input images into base and detail layers. Compared with conventional MSDs, this MSD can achieve the unique property of preserving the information of specific scales and reducing halos near edges. Secondly, we argue that the base layers obtained by most MSDs would contain a certain amount of residual low-frequency information, which is important for controlling the contrast and overall visual appearance of the fused image, and the conventional “averaging” fusion scheme is unable to achieve desired effects. To address this problem, an improved VSM-based technique is proposed to fuse the base layers. Lastly, a novel WLS optimization scheme is proposed to fuse the detail layers. This optimization aims to transfer more visual details and less irrelevant IR details or noise into the fused image. As a result, the fused image details would appear more naturally and be suitable for human visual perception. Experimental results demonstrate that our method can achieve a superior performance compared with other fusion methods in both subjective and objective assessments. |
| |
| Keywords: | Multi-scale decomposition Image fusion Rolling guidance filter Visual saliency map Weighted least square optimization |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
