A Multi-Modal Fusion Method Based on Higher-Order Orthogonal Iteration Decomposition |
| |
Authors: | Fen Liu
Jianfeng Chen
Weijie Tan
Chang Cai
|
| |
Institution: | 1.School of Marine Science and Technology, Northwestern Polytechnical University, Xi’an 710072, China; (F.L.); (C.C.);2.College of Mathematics and Computer Science, Yan’an University, Yan’an 716000, China;3.State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China; |
| |
Abstract: | Multi-modal fusion can achieve better predictions through the amalgamation of information from different modalities. To improve the performance of accuracy, a method based on Higher-order Orthogonal Iteration Decomposition and Projection (HOIDP) is proposed, in the fusion process, higher-order orthogonal iteration decomposition algorithm and factor matrix projection are used to remove redundant information duplicated inter-modal and produce fewer parameters with minimal information loss. The performance of the proposed method is verified by three different multi-modal datasets. The numerical results validate the accuracy of the performance of the proposed method having 0.4% to 4% improvement in sentiment analysis, 0.3% to 8% improvement in personality trait recognition, and 0.2% to 25% improvement in emotion recognition at three different multi-modal datasets compared with other 5 methods. |
| |
Keywords: | multi-modal fusion tensor iteration decomposition dimensionality reduction |
|
|