I’m a PhD candidate at the University of Cambridge interested in Reinforcement Learning (RL) and its real-world applications. I’m part of Jonathan Cullen’s group–the Resource Efficiency Collective, I’m affiliated with the Alan Turing Institute, Cambridge’s AI for Environmental Risks centre, I’ve collaborated with the Cambridge Centre for the Study of Existential Risk (CSER), and work closely with the Control and Verification Group in Oxford’s Department of Computer Science.

I’m interested in solving problems that maximise expected human prosperity. I see reinforcement learning (RL) as a critical technology in this pursuit, and my research focuses on building new RL systems that can be deployed in the real world toward useful ends.

From the set of helpful problems to tackle, I’ve so far been focusing on climate change mitigation. I’m interested in building agents that can reduce carbon emissions at a globally-relevant scale; this has meant building RL agents that could be deployed in every building in the world to minimise carbon emissions – see here. Creating agents that can generalise at this scale is non-trivial, and requires us to deal with challenges ignored by traditional approaches to RL like non-stationary environments, non-episodic tasks, small datasets, inadequate simulators etc. I’m interested in exploring algorithmic innovations that can help us deal with these challenges, often in the domains of: model-based RL, sequence-to-sequence learning, continual learning, probabilistic modelling, planning etc.

If you’re into this sort of thing, I’m always keen to chat so send me an email.

Scott

Updates

[19/11/22] Our paper Low Emission Building Control with Zero-Shot Reinforcement Learning was accepted at AAAI 2023. I’ll present it in Washington DC next February. [paper]

[17/10/22] I’m interning as a Research Scientist with Foresight Data Machines, working on RL for autonomous control in heavy industry.

[29/09/22] Our pre-print: Transforming agrifood production systems and supply chains with digital twins was accepted in Nature Science of Food. [paper]

[26/09/22] I went viral on TikTok for wearing pink crocs. [link]

[28/06/22] New pre-print: Low Emission Building Control with Zero-Shot Reinforcement Learning. [paper]

[06/08/21] I’ve become an aligned PhD student with Cambridge’s Artificial Intelligence for the Study of Environmental Risk (AI4ER) Centre for Doctoral Training. [link]

[21/07/21] A story covering my acceptance onto the Alan Turing Institute’s PhD Enrichment Scheme was published on the University of Cambridge’s website. [link]

[19/05/21] I’ve been accepted onto the 2022 Alan Turing Enrichment Scheme. I’ll spend most of next year at Alan Turing HQ in the British Library, London continuing my PhD research. [link]