Better System Trader 185: How to avoid creating trading strategies that degrade quickly - Timothy Masters
Apr 12, 2021
AI Snips
Chapters
Transcript
Episode notes
Protect Your Out-Of-Sample Data
- Avoid reusing the same out-of-sample dataset multiple times during development.
- Save out-of-sample data for one final unbiased test to prevent selecting the luckiest model.
Selection Picks The Luckiest System
- Out-of-sample performance equals true skill plus luck, so comparing systems picks the luckier one.
- Selecting the luckiest system creates optimistic bias and causes deterioration in live trading.
Use Permutation Tests Early
- Use Monte Carlo permutation tests to detect overfitting without touching out-of-sample data.
- Retrain on many permuted datasets and ensure original performance stands out above permuted runs.