Implementing PPO from scratch in PyTorch
A walk through every moving part of Proximal Policy Optimization - GAE, the clipped surrogate objective, the value loss, the entropy bonus and the training loop - and why each one is there.
Technical notes on the things I build - reinforcement learning internals and agentic systems on Claude and MCP.
A walk through every moving part of Proximal Policy Optimization - GAE, the clipped surrogate objective, the value loss, the entropy bonus and the training loop - and why each one is there.
The architecture behind a fleet of coordinating agents: mailbox messaging, persistent memory, MCP tools and human-in-the-loop control - what worked and what was genuinely hard.