This study proposes a novel general image fusion framework based on cross-domain long-range learning and Swin Transformer, termed as SwinFusion. On the one hand, an attention-guided cross-domain module is devised to achieve sufficient integration of complementary information and global interaction. More specifically, the proposed method involves an intra-domain fusion unit based on self-attention and an inter-domain fusion unit based on cross-attention, which mine and integrate long dependencies within the same domain and across domains. Through long-range dependency modeling, the network is able to fully implement domain-specific information extraction and cross-domain complementary information integration as well as maintaining the appropriate apparent intensity from a global perspective. In particular, we introduce the shifted windows mechanism into the self-attention and cross-attention, which allows our model to receive images with arbitrary sizes. On the other hand, the multi-scene image fusion problems are generalized to a unified framework with structure maintenance, detail preservation, and proper intensity control. Moreover, an elaborate loss function, consisting of SSIM loss, texture loss, and intensity loss, drives the network to preserve abundant texture details and structural information, as well as presenting optimal apparent intensity. Extensive experiments on both multi-modal image fusion and digital photography image fusion demonstrate the superiority of our SwinFusion compared to the state-of-the-art unified image fusion algorithms and task-specific alternatives. Implementation code and pre-trained weights can be accessed at .

Jiayi Ma;Linfeng Tang;Fan Fan;Jun Huang;Xiaoguang Mei;Yong Ma

Electronic Information School,Wuhan University,Wuhan 430072,China

自动化学报（英文版）

[1]Jiayi Ma;Linfeng Tang;Fan Fan;Jun Huang;Xiaoguang Mei;Yong Ma-.SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer)[J].自动化学报（英文版）,2022(07):1200-1217

SwinFusion

Cross,Long,range,Learning,General,Image,via,Transformer,This,study,proposes,novel,fusion,framework,long,learning,termed,On,one,hand,attention,guided,module,devised,achieve,sufficient,integration,complementary,information,global,interaction,specifically,proposed,method,involves,intra,unit,self,which,mine,integrate,dependencies,within,same,across,domains,Through,dependency,modeling,network,able,fully,implement,extraction,well,maintaining,appropriate,apparent,intensity,from,perspective,In,particular,introduce,shifted,windows,mechanism,into,allows,our,receive,images,arbitrary,sizes,other,multi,scene,problems,generalized,unified,structure,maintenance,preservation,proper,control,Moreover,elaborate,loss,function,consisting,SSIM,texture,drives,preserve,abundant,details,structural,presenting,optimal,Extensive,experiments,both,modal,digital,photography,demonstrate,superiority,compared,state,algorithms,task,alternatives,Implementation,code,trained,weights,can,be,accessed

0.543454