Hybrid blend
A meta-learner sitting on top of every other architecture in the leaderboard. Its job is to answer one question: do the sequence models add anything on top of the trees? The honest answer is no.
Scores
How it works
For each target the script loads the OOF predictions of the GBM stack, the transformer, the BiGRU, and the LSTM, and tries three different meta-learners on those columns: a non-negative L1 minimiser, NNLS (non-negative least squares), and a logistic / ridge CV. The CV-validated best of the three is kept as the official hybrid number.
The result every time is the same: the blender lands on the GBM stack with weight ~1.0 and assigns the sequence models essentially zero weight. On gender the small positive weights it tries to give the sequence models even hurt: the hybrid lands at 0.875 vs 0.877 for the GBM stack alone. On age and mood it ties the GBM stack to the third decimal. The takeaway: at this dataset size the trees consume all the signal the sequence models have.
Configuration
- Base predictorsGBM stack, transformer, BiGRU, LSTM (4 OOF columns per target)
- Blenders triedL1 minimisation, NNLS, logistic CV (gender) / ridge CV (regression)
- SelectionBest held-out CV score per target
- CV5-fold StratifiedKFold (gender), KFold (age, mood), seed 42
- ConstraintNon-negative weights, sum unconstrained