Bidirectional GRU
A two-layer bidirectional GRU with learned attention pooling over the 21 sequence steps, fused with a 14-dim side-feature MLP. The strongest of the three sequence models on gender and age, ties the LSTM on mood.
Scores
How it works
Each step in the 21-element sequence is projected from 40 dims to d_model=64, then run through a 2-layer bidirectional GRU (64 hidden per direction, so 128 per step out). Instead of taking the final hidden state, a learned-query additive attention pools the 21 step outputs into a single 128-dim context vector. A small MLP maps 14 hand-engineered session-level side features into 128 dims; the two 128-vectors are concatenated and pushed through a fuse layer before the three task heads.
Trained multi-task with weighted BCE + MSE losses. AdamW with linear warmup and cosine decay. Best validation snapshot kept on gender BCE.
Configuration
- d_model64
- Hidden64 per direction, 2 layers
- PoolingLearned-query additive attention over 21 steps
- Side features14, projected to 128 and fused
- Params179,139
- Dropout0.3
- OptimiserAdamW, lr 6e-4, warmup 3 epochs, cosine decay over 50 epochs
- CV5-fold StratifiedKFold on gender, seed 42