Photometric stereo aims to reconstruct 3D geometry by recovering the dense surface orientation of a 3D object from multiple images under differing illumination.Traditional methods normally adopt simplified reflectance models to make the surface orientation computable.However,the real reflectances of surfaces greatly limit applicability of such methods to real-world objects.While deep neural networks have been employed to handle non-Lambertian surfaces,these methods are subject to blurring and errors,especially in high-frequency regions(such as crinkles and edges),caused by spectral bias:neural networks favor low-frequency representations so exhibit a bias towards smooth functions.In this paper,therefore,we propose a self-learning conditional network with multi-scale features for photometric stereo,avoiding blurred reconstruction in such regions.Our explorations include:(i)a multi-scale feature fusion architecture,which keeps high-resolution representations and deep feature extraction,simultaneously,and(ii)an improved gradient-motivated conditionally parameterized convolution(GM-CondConv)in our photometric stereo network,with different combinations of convolution kernels for varying surfaces.Extensive experiments on public benchmark datasets show that our calibrated photometric stereo method outperforms the state-of-the-art.

Yakun Ju;Yuxin Peng;Muwei Jian;Feng Gao;Junyu Dong

Department of Computer Science and Technology,Ocean University of China,Qingdao 266100,China;Wangxuan Institute of Computer Technology,Peking University,Beijing 100871,China;School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan 250002,China

计算可视媒体（英文）

[1]Yakun Ju;Yuxin Peng;Muwei Jian;Feng Gao;Junyu Dong-.Learning conditional photometric stereo with high-resolution features)[J].计算可视媒体（英文）,2022(01):105-118

reflectances,crinkles,conditionally,CondConv

Learning,photometric,stereo,high,resolution,features,Photometric,aims,geometry,by,recovering,dense,orientation,from,multiple,images,under,differing,illumination,Traditional,methods,normally,adopt,simplified,models,make,computable,However,real,surfaces,greatly,limit,applicability,such,world,objects,While,deep,neural,networks,have,been,employed,handle,Lambertian,these,are,subject,blurring,errors,especially,frequency,regions,edges,caused,spectral,bias,favor,low,representations,exhibit,towards,smooth,functions,In,this,paper,therefore,propose,self,learning,scale,avoiding,blurred,reconstruction,Our,explorations,include,fusion,architecture,which,keeps,extraction,simultaneously,ii,improved,gradient,motivated,parameterized,convolution,GM,our,different,combinations,kernels,varying,Extensive,experiments,public,benchmark,datasets,show,that,calibrated,outperforms,state,art

0.612679