Night vision dense crowd counting based on mid-term fusion of thermal imaging features北大核心CSCD

为了提高人群计数模型对尺度和光噪声的鲁棒性,设计了一种多模态图像融合网络。提出了一种针对夜间人群统计模型,并设计了一个子网络Rgb-T-net,网络融合了热成像特征和可见光图像的特征,增强了网络对热成像和夜间人群特征的判断能力。模型采用自适应高斯核对密度图进行回归,在Rgb-T-CC数据集上完成了夜视训练和测试。经验证网络平均绝对误差为18.16,均方误差为32.14,目标检测召回率为97.65%,计数性能和检测表现优于当前最先进的双峰融合方法。实验结果表明,所提出的多模态特征融合网络能够解决夜视环境下的计数与检测问题,消融实验进一步证明了融合模型各部分参数的有效性。

Night vision dense crowd counting based on mid-term fusion of thermal imaging features

In order to improve the robustness of crowd counting model to scale and optical noise, a multimodal image fusion network was designed. A statistical model for night crowd was proposed, and a sub network Rgb-T-net was designed. The network integrated the characteristics of thermal imaging and visible image, and the ability of network to judge the characteristics of thermal imaging and night crowd was enhanced. The proposed model used the adaptive Gaussian checking density diagram for regression, and the night vision training and testing were completed on the Rgb-T-CC data set. Through verification, the average absolute error of the network is 18.16, the mean square error is 32.14, and the recall rate of target detection is 97.65%. The counting performance and detection performance are superior to the current most advanced bimodal fusion method. The experimental results show that the proposed multimodal feature fusion network can solve the counting and detection problem in night vision environment, and the ablation experiment further proves the effectiveness of parameters of the fusion model.