Nowadays,high-performance computing(HPC)clusters are increasingly popular.Large volumes of job logs recording many years of operation traces have been accumu-lated.In the same time,the HPC cloud makes it possible to access HPC services remotely.For executing applications,both HPC end-users and cloud users need to request specific resources for different workloads by themselves.As users are usually not familiar with the hardware details and software layers,as well as the performance behavior of the underlying HPC systems.It is hard for them to select optimal resource configurations in terms of performance,cost,and energy efficiency.Hence,how to provide on-demand services with intelligent resource allocation is a critical issue in the HPC community.Prediction of job characteristics plays a key role for intelligent resource allocation.This paper presents a survey of the existing work and future directions for prediction of job characteristics for intelligent resource allocation in HPC systems.We first review the existing techniques in obtaining performance and energy consumption data of jobs.Then we survey the techniques for single-objective oriented predictions on runtime,queue time,power and energy consumption,cost and optimal resource configuration for input jobs,as well as multi-objective oriented predictions.We conclude after discus-sing future trends,research challenges and possible solutions towards intelligent resource allocation in HPC systems.

Zhengxiong HOU;Hong SHEN;Xingshe ZHOU;Jianhua GU;Yunlan WANG;Tianhai ZHAO

Center for High Performance Computing,School of Computer Science,Northwestern Polytechnical University,Xi'an 710072,China;School of Computer Science and Engineering,Sun Yat-Sen University,Guangzhou 510275,China

计算机科学前沿

[1]Zhengxiong HOU;Hong SHEN;Xingshe ZHOU;Jianhua GU;Yunlan WANG;Tianhai ZHAO-.Prediction of job characteristics for intelligent resource allocation in HPC systems:a survey and future directions)[J].计算机科学前沿,2022(05):16-32

Prediction,characteristics,intelligent,allocation,HPC,systems,survey,future,directions,Nowadays,high,performance,computing,clusters,increasingly,popular,Large,volumes,logs,recording,many,years,operation,traces,have,been,accumu,lated,In,same,cloud,makes,possible,access,services,remotely,For,executing,applications,both,users,need,request,specific,resources,different,workloads,by,themselves,usually,not,familiar,hardware,details,software,layers,well,behavior,underlying,It,select,optimal,configurations,terms,cost,energy,efficiency,Hence,how,provide,demand,critical,issue,community,plays,key,role,This,paper,presents,existing,We,first,review,techniques,obtaining,consumption,data,jobs,Then,single,objective,oriented,predictions,runtime,queue,power,input,multi,conclude,after,discus,trends,research,challenges,solutions,towards

0.522484