Decision Tree

The study compared the performance of decision trees, random forests, XGBoost, RNN, DNN, and Siamese neural networks in atrial fibrillation detection

Post-hoc explanation methods include SHAP and LIME, and intrinsic interpretable models include linear models and decision trees.

Algorithms such as logistic regression, decision trees and support vector machines produce decision boundaries of differing shapes.

Some regulators tend to require that financial AI use inherently interpretable models (such as logistic regression and decision trees), which directly limits the room for deploying complex models.

Traditional models such as SVM or decision trees are extremely sensitive to label noise, because their capacity and optimization path lack the preference for learning commonalities first and memorizing special cases later.

Decision trees are widely used in scenarios that require explainable grounds, such as medical diagnosis and credit approval.

A decision model is essentially a reusable set of judgment logic, which can be as simple as a decision tree or a set of weighted scoring rules.

CLIC trains an interpretable decision tree to distinguish between the code collections of two LLMs.

The biggest problem with a single decision tree is high variance and susceptibility to overfitting; ensemble learning compensates for this shortcoming through the trade-off of higher bias for lower variance.

The construction of a decision tree is essentially a recursive process: at each node, data is split in two based on a certain feature and threshold, and this process is repeated on the subsets until a stopping condition is met.

The time complexity of brute-force searching for the optimal split point in a decision tree is O(n·m·log n), where n is the number of samples and m is the number of features, and this is the main bottleneck in decision tree training speed.

Common techniques for limiting decision tree overfitting include restricting the maximum tree depth (max_depth), requiring nodes to contain at least N samples before splitting (min_samples_split), and post-pruning.

Bagging (Bootstrap Aggregating) trains multiple decision trees by sampling training data with replacement and aggregating predictions, effectively reducing variance and improving stability.

决策树及其集成版本（随机森林、XGBoost）对特征缩放不敏感，不需要像SVM或神经网络那样进行标准化预处理

决策树的分裂准则（信息增益、基尼不纯度、均方误差）本质上是排序统计量的函数，只关心阈值两侧的类别比例而非样本间距离

决策树使用沿坐标轴的轴对齐分割，对特征的单调变换和局部拉伸不敏感，因此对保持类别排序的变换具有天然不变性

该SWD特征变换实验在决策树上表现良好，但在逻辑回归、SVM、KNN等算法上效果糟糕

决策树的分裂准则本质上是排序统计量的函数，对任何保持类别排序的单调变换具有不变性