Action recognition is an important research topic in video analysis that remains very challenging.Effective reco-gnition relies on learning a good representation of both spatial information(for appearance)and temporal information(for motion).These two kinds of information are highly correlated but have quite different properties,leading to unsatisfying results of both connecting independent models(e.g.,CNNN-LSTM)and direct unbiased co-modeling(e.g.,3DCNN).Besides,a long-lasting tradition on this task with deep learning models is to just use 8 or 16 consecutive frames as input,making it hard to extract discriminative motion features.In this work,we propose a novel network structure called ResLNet(Deep Residual LSTM network),which can take longer inputs(e.g.,of 64 frames)and have convolutions collaborate with LSTM more effectively under the residual structure to learn better spatial-temporal representations than ever without the cost of extra computations with the proposed embedded vari-able stride convolution.The superiority of this proposal and its ablation study are shown on the three most popular benchmark datasets:Kinetics,HMDB51,and UCF101.The proposed network could be adopted for various features,such as RGB and optical flow.Due to the limitation of the computation power of our experiment equipment and the real-time require-ment,the proposed network is tested on the RGB only and shows great performance.

Tian WANG;Jiakun LI;Huai-Ning WU;Ce LI;Hichem SNOUSSI;Yang WU

Institute of Artificial Intelligence,Beihang University,Beijing 100191,China;School of Automation Science and Electrical Engineering,Beihang University,Beijing 100191,China;College of Electrical and Information Engineering,Lanzhou University of Technology,Lanzhou 730050,China;Institute Charles Delaunay-LM2S FRE CNRS 2019,University of Technology of Troyes,Troyes 10010,France;Institute for Research Initiatives,Nara Institute of Science and Technology,Nara 630-0192,Japan

计算机科学前沿

[1]Tian WANG;Jiakun LI;Huai-Ning WU;Ce LI;Hichem SNOUSSI;Yang WU-.ResLNet:deep residual LSTM network with longer input for action recognition)[J].计算机科学前沿,2022(06):39-47

ResLNet,gnition,unsatisfying,CNNN

deep,residual,network,longer,action,recognition,Action,important,research,topic,video,analysis,that,remains,very,challenging,Effective,relies,learning,good,both,spatial,information,appearance,temporal,motion,These,kinds,are,highly,correlated,but,have,quite,different,properties,leading,results,connecting,independent,models,direct,unbiased,modeling,3DCNN,Besides,lasting,tradition,this,task,just,use,consecutive,frames,making,hard,extract,discriminative,features,In,novel,structure,called,Deep,Residual,which,can,take,inputs,convolutions,collaborate,more,effectively,under,better,representations,than,ever,without,cost,computations,proposed,embedded,able,stride,superiority,proposal,its,ablation,study,shown,three,most,popular,benchmark,datasets,Kinetics,HMDB51,UCF101,could,adopted,various,such,RGB,optical,flow,Due,limitation,power,our,experiment,equipment,real,require,tested,only,shows,great,performance

0.597944