MSc dissertation: PPO study
My MSc dissertation work studying Proximal Policy Optimization in depth.
researchshipped
Problem
PPO is widely used but its strong results depend heavily on implementation details that are rarely spelled out. My dissertation set out to study the algorithm carefully and understand which design choices genuinely affect performance.
Approach
- Worked through the theory behind the clipped surrogate objective and why it constrains policy updates.
- Examined the practical levers - advantage normalisation, GAE lambda, clip range, value-loss weighting and entropy - and how each changes training behaviour.
- Backed the analysis with my own from-scratch implementation so claims could be tested directly rather than taken on trust.
Tech
PyTorchReinforcement learningPPOAcademic research
Status
Completed as part of my MSc. The accompanying open-source PPO implementation is public; the dissertation document itself is academic work.