Contents:

  1. What is Reinforcement Learning?

  2. Choosing a policy

  3. Calculating the value of an action

  4. Practical consideration

    • “Gamification” of trading

    • How is the system trained (each game independent)?

    • Reward-function engineering

    • What features do we use for the neural network?

    • How to test the system?

    • What type of ANN should be used?

  5. Demo and results

  6. Lessons learned

    • RL can be very sample inefficient

    • Reward function design is hard

    • Rewards in trading are sparse

    • Local optima are difficult to escape

    • RL could just be overfitting peculiar chart patterns

    • Results are unstable and hard to reproduce

  7. Why is it so hard?

    • Financial series are very noisy

    • Financial systems are dynamic - rules keep changing

    • Rules evolve by the very act of understanding them

    • Computing power is still limited

    • New algorithms are yet to be discovered

  8. Improving performance

    • Noise = Unexplained returns

    • Adding predictive factors to improve performance

  9. Future work - let the machine select the features

  10. Questions & Answers