Introduction to DynForest methodology

The tree building

For each tree \(b\) \((b = 1, ..., B)\), we aim to partition the subjects into groups that are homogeneous regarding the outcome \(Y\). The tree building process is summarized in figure 1.

We first draw a bootstrap sample from the original dataset of \(N\) subjects. The subjects excluded by the bootstrap constitute the out-of-bag (OOB) sample, noted \(OOB^b\) for the tree \(b\). Then, at each node \(d \in \mathcal{D}\), we recursively repeat the following steps using the \(N^{(d)}\) subjects located at node \(d\):

1. An ensemble of \(\mathcal{M}^{(d)}=\{\mathcal{M}_x^{(d)},\mathcal{M}_y^{(d)}\}\) candidate predictors are randomly selected among \(\{\mathcal{M}_x,\mathcal{M}_y\}\) (see figure 1B). The size of \(\mathcal{M}^{(d)}\) is defined by the hyperparameter \(mtry\);
2. For each longitudinal predictor in \(\mathcal{M}_y^{(d)}\):
   1. We independently model the trajectory using a flexible linear mixed model (Laird and Ware 1982) according to time (specified by the user)
   2. We derive an ensemble \(\mathcal{M}_{y\star}^{(d)}\) of individual time-independent features. These features are the individual random-effects \(\widehat{b}_i^{(d)}\) of the linear mixed model predicted from the repeated data for individual \(i\)
   3. We define \(\mathcal{M}_\star^{(d)}=\{\mathcal{M}_x^{(d)},\mathcal{M}_{y \star}^{(d)}\}\) our new ensemble of candidate features
3. For each candidate feature \(W \in \mathcal{M}_\star^{(d)}\):
   1. We build a series of splits \(c_W^{(d)}\) according to the feature values if continuous, or modalities otherwise (see figure 1C), leading to two groups.
   2. We quantify the distance between the two groups according to the nature of \(Y\):
      • With \(Y\) continuous, we compute the weighted within-group variance with the proportion of subjects in each group as weights
      • With \(Y\) categorical, we compute the weighted within-group Shannon entropy (Shannon 1948) (i.e. the amount of uncertainty) with the proportion of subjects in each group as weights
      • With \(Y\) survival competing events, we compute the log-rank statistic test (Peto and Peto 1972)
      • With \(Y\) survival competing events, we compute the Fine & Gray statistic test (Gray 1988)
4. We split the subjects into the two groups that minimize (for continuous and categorical outcome) or maximize (for survival outcome) the distance defined previously. We denote \(\{W_0^d,c_0^d\}\) the optimal couple used to split the subjects where they are assigned to the left and right daughter nodes \(2d\) and \(2d + 1\), respectively (see figure 1D and A).

To end the recursive procedure described previously, a stopping criterion should be met. We define two stopping criteria: nodesize the minimal number of subjects in the nodes and minsplit the minimal number of events required to split the node. minsplit is only defined with survival outcome. In the following, we call leaves the nodes that cannot be split.

In each leaf \(h \in \mathcal{H}\), a summary \(\pi^{h^b}\) is computed using the individual belonging to the leaf \(h\). The leaf summary is defined according to the outcome. We return:

• the mean, with \(Y\) continuous
• the modality with the highest probability, with \(Y\) categorical
• the cumulative incidence function over time estimated by Nelson-Aalen (Nelson 1969; O. Aalen 1976), with \(Y\) survival competing events
• the cumulative incidence function over time estimated by Aalen-Johansen (O. O. Aalen and Johansen 1978), with \(Y\) survival competing events

Figure 1: Overall scheme of the tree building in DynForest with (A) the tree structure, (B) the node-specific treatment of time-dependent predictors to obtain time-fixed features, (C) the dichotomization of the time-fixed features, (D) the splitting rule.

Individual prediction of the outcome

Out-Of-Bag prediction error

Using the OOB individual predictions, an OOB prediction error can be internally assessed, in particular to tune the hyperparameters of the random forest. The OOB prediction error estimates the difference between observed and predicted values and is defined according to the nature \(Y\) by:

• the mean square error (MSE), with \(Y\) continuous
• the missclassification error, with \(Y\) categorical
• the Integrated Brier Score (IBS) (Sène et al. 2016), with \(Y\) survival

We want to minimize as much as possible the OOB prediction error by tuning mtry, nodesize and minsplit hyperparameters.

Explore the most predictive variables