Feature Transformation: Tree Ensemble vs SVM

Questions

Questions:

  • AiLA’s CML strategy uses a feature transformation step using classification, intended to help with non-linear relationships and regularization related the potentially large number of features [details found in CML modelling section].
  • What is the benefit of the feature transformation?
  • What happens if we replace a Tree Ensemble method with SVM?

Approach:

  • Focus on model precision and number of positively classified days as relevant metrics w.r.t strategy performance.
  • Control comparison by using same data samples and architecture setup, only differing by the feature transform step.

Precision Outperformance:

  • Fraction of true label instances in sample predicted by CML strategy, minus fraction in whole data sample.
  • Majority of assets yield higher true instance fraction and results consistent between validation and holdout data.

Consistency Checks

  • The study includes slight differences w.r.t. the [CML modelling section] analysis, such as including 2021 data and fixed set of hyper parameters.
  • However, generally a good consistency was found between the prediction results, given these slight differences.

Prediction Performance

  • Prediction results found similar between using tree-based feature transform (FT) and no FT, e.g. smaller variance vs slightly larger mean.
  • The larger variance observed when not using any FT relates to tendency of classifying fewer positive days (see next slide).

Feature Transform Impact

  • Feature transform (FT) is based on the Tree Ensemble model,
    • Reduces the number of features (proportional to nr of trees) by at least one order of magnitude.
    • Similar prediction precision/performance of full CML strategy as without using any feature transform.
    • However, consistently with a larger fraction of days positively classified by the model.
  • Given the small prediction edge a large number of “trading days” is required for a relevant strategy performance, which makes the larger fraction of positive days in favor of using the tree-based FT vs no FT.
  • Feature transform based on SVM model,
    • Typically, a number of support vectors (SV) slightly more than half of the number of data points used.
    • Number of features, constructed using SVs together with kernel and dual coefficients, often exceeded the number of original features.
    • No area of improvement w.r.t the tree-based FT was observed using the SVM version.

Conclusions

  • The implications from using a feature transform in the CML strategy was investigated, which currently is based on an ensemble tree model.
  • The study was made at the model prediction level to avoid differences introduced by downstream strategy aspects, however, focused on metrics expected to be most relevant for the final strategy performance.
  • The current FT, based on an ensemble tree model, reduces the number of features dramatically and resulted in a similar model performance compared with not using any FT, however, yielding a larger number of positively classified days.
  • This is considered an advantage given that the small prediction edge requires many “trading days” for a relevant strategy performance.
  • The comparison also included a FT based on a SVM model, however, no areas of improvement were observed compared to the tree-based FT.