Skip to content

Can RL dose anaesthesia better than a tuned PID?

Reinforcement learning against a well-tuned PID controller for anaesthetic dosing, tested on simulated patients it had never seen.

Year
2026
Domain
Robot learning
Role
Solo, coursework rebuilt properly
Stack
Python, PyTorch, Gymnasium, Soft Actor-Critic
Fig. 1One unseen test patient. The dots are the noisy, delayed signal the controllers actually see.

The problem

During surgery an anaesthetist keeps adjusting the dose so the patient stays at the right depth. The sensor is slow, noisy and delayed, and every patient responds differently. That makes it a hard control problem, and a fair test of whether learning beats a classical controller.

What I built

A simulated patient population, a training environment with a realistic delayed sensor, and three controllers: a tuned PID, pure reinforcement learning, and a learned correction on top of the PID. Training, tuning and test patients were kept separate.

How it went

The learned correction on top of the PID matched it. Pure reinforcement learning did worse, after first finding a loophole in the task that I had to close.

Figures

Each point is a test patient: where the learned correction helped and where it didn't.
Fig. 2Each point is a test patient: where the learned correction helped and where it didn't.
Anaesthetic depth over a full case for six test patients.
Fig. 3Anaesthetic depth over a full case for six test patients.
Code and write-up ↗