It is significant to efficiently support artificial intelligence ( AI) applications on heterogeneous mobile platforms, especially coordinately execute a deep neural network ( DNN) model on multiple computing devices of one mobile platform. This paper proposes HOPE, an end-to-end heterogeneous inference framework running on mobile platforms to distribute the operators in a DNN model to differ- ent computing devices. The problem is formalized into an integer linear programming ( ILP) problem and a heuristic algorithm is proposed to determine the near-optimal heterogeneous execution plan. The experimental results demonstrate that HOPE can reduce up to 36 . 2％ inference latency ( with an average of 22. 0％) than MOSAIC, 22. 0％ (with an average of 10. 2％) than StarPU and 41. 8％ (with an average of 18. 4％) than μLayer respectively.

XIA Chunwei;ZHAO Jiacheng;CUI Huimin;FENG Xiaobing

Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,P.R.China;School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100190,P.R.China

高技术通讯（英文版）

[1]XIA Chunwei;ZHAO Jiacheng;CUI Huimin;FENG Xiaobing-.HOPE: a heterogeneity-oriented parallel execution engine for inference on mobiles)[J].高技术通讯（英文版）,2022(04):363-372

mobiles,MOSAIC,StarPU

HOPE,heterogeneity,oriented,parallel,execution,engine,inference,It,significant,efficiently,support,artificial,intelligence,applications,heterogeneous,platforms,especially,coordinately,execute,deep,neural,network,DNN,model,multiple,computing,devices,one,This,paper,proposes,end,framework,running,distribute,operators,differ,problem,formalized,into,integer,linear,programming,ILP,heuristic,algorithm,proposed,determine,optimal,plan,experimental,results,demonstrate,that,reduce,latency,average,than,Layer,respectively

0.544703