Decision tree(DT)plays an important role in pattern recognition and machine learning,which is widely used for regression tasks because of its natural interpretability.Nevertheless,the traditional decision tree is constructed by recursive Boolean division.The discrete decision-making process in DT makes it non-differentiable,and causes the problem of hard decision boundary.To solve this problem,a probability distribution model—Staired-Sigmoid is proposed in this paper.The Staired-Sigmoid model is used to differentiate the decision-making process,by which the samples can be assigned to two sub-trees more finely.Based on Staired-Sigmoid,we further propose the soft decision tree(SDT)for regression tasks,where the samples are assigned to different sub-nodes according to a continuous probability distribution.This process is differentiable,and all parameters in SDT can be optimized by gradient descent algorithms.Owing to its constructing rules,SDT is more stable than decision tree,and it is easier to overcome the problem of overfitting.We validate SDT on several datasets obtained from UCI.Experiments demonstrate that SDT achieves better performance than decision tree,and it significantly alleviates the overfitting.

Nengjing GUO;Jianfeng HUANG

The Troops 61206,PLA,Beijing 100041,China;School of Earth and Space Sciences,Peking University,Beijing 100871,China

系统科学与信息学报（英文版）

[1]Nengjing GUO;Jianfeng HUANG-.Soft Decision Tree for Regression)[J].系统科学与信息学报（英文版）,2022(05):518-530

Staired

Soft,Decision,Tree,Regression,plays,important,role,pattern,recognition,machine,learning,which,widely,used,regression,tasks,because,its,natural,interpretability,Nevertheless,traditional,decision,constructed,by,recursive,Boolean,division,discrete,making,process,makes,differentiable,causes,problem,hard,boundary,To,solve,this,probability,distribution,model,Sigmoid,proposed,paper,differentiate,samples,assigned,two,sub,trees,more,finely,Based,we,further,soft,SDT,where,are,nodes,according,continuous,This,parameters,optimized,gradient,descent,algorithms,Owing,constructing,rules,stable,than,easier,overcome,overfitting,We,validate,several,datasets,obtained,from,UCI,Experiments,demonstrate,that,achieves,better,performance,significantly,alleviates

0.536931