Visual localization is a crucial component in the application of mobile robot and autonomous driving. Image retrieval is an efficient and effective technique in image-based localization methods. Due to the drastic variability of environmental conditions, e.g., illumination changes, retrieval-based visual localization is severely affected and becomes a challenging problem. In this work, a general architecture is first formulated probabilistically to extract domain-invariant features through multi-domain image translation. Then, a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner. The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models with and without Grad-SAM loss. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMU-Seasons dataset. The strong generalization ability of our approach is verified with the RobotCar dataset using models pre-trained on urban parts of the CMU-Seasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision, especially under challenging environments with illumination variance, vegetation, and night-time images. Moreover, real-site experiments have been conducted to validate the efficiency and effectiveness of the coarse-to-fine strategy for localization.

Hanjiang Hu;Hesheng Wang;Zhe Liu;Weidong Chen

Department of Automation,Shanghai Jiao Tong University,Shanghai 200240,China;Department of Automation,Institute of Medical Robotics,Key Laboratory of System Control and Information Processing of Ministry of Education,Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education,Shanghai Jiao Tong University,Shanghai 200240,China;Department of Computer Science and Technology,University of Cambridge,Cambridge CB30FD,United Kingdom

自动化学报（英文版）

[1]Hanjiang Hu;Hesheng Wang;Zhe Liu;Weidong Chen-.Domain-Invariant Similarity Activation Map Contrastive Learning for Retrieval-Based Long-Term Visual Localization)[J].自动化学报（英文版）,2022(02):313-328

RobotCar

Domain,Invariant,Similarity,Activation,Map,Contrastive,Learning,Retrieval,Based,Long,Term,Visual,Localization,localization,crucial,component,application,mobile,robot,autonomous,driving,Image,retrieval,efficient,technique,methods,Due,drastic,variability,environmental,conditions,illumination,changes,visual,severely,affected,becomes,challenging,problem,this,work,architecture,first,formulated,probabilistically,extract,domain,invariant,features,through,multi,translation,Then,novel,gradient,weighted,similarity,activation,mapping,loss,Grad,SAM,incorporated,finer,high,accuracy,We,also,new,adaptive,triplet,boost,contrastive,learning,embedding,self,supervised,manner,final,coarse,pipeline,implemented,sequential,combination,models,without,Extensive,experiments,have,been,conducted,validate,effectiveness,proposed,approach,CMU,Seasons,dataset,strong,generalization,our,verified,using,trained,urban,parts,Our,performance,even,outperforms,state,baselines,medium,precision,especially,under,environments,variance,vegetation,night,images,Moreover,real,site,efficiency,strategy

0.614174