We introduce the challenging Sterkfontein Caves dataset comprising ten underground scenes from a UNESCO World Heritage Site, and use it to find a new simple baseline method that beats existing low-light reconstruction methods upon it. Each scene is of complex surface geometry and high-frequency texture from cave rock structures, including human-made markings on the rock face. The captured images exhibit varied or uncontrolled lighting over a large dynamic range, with glare artefacts, and low signal-to-noise ratios from the challenging dark real-world capture scenario. We propose a view synthesis benchmark for low-light RAW and sRGB reconstruction. All tested NeRF and Gaussian splatting baseline methods struggle on this data, with the best performing method in terms of reliability and average PSNR being our Raw-Nerfacto method. We discuss these errors in detail to find directions of future work for the community in overcoming the significant challenges that remain in low-light high-detail scenes.

We describe a research programme aimed at the construction of a single, generally intelligent agent—one competent across all tasks rather than skilled at any one. Such an agent must act in the real world through a rich sensorimotor interface, with sensors and actuators general enough for any task it may face. But that same richness makes any single task nearly impossible to learn directly. We argue that the resolution lies in abstraction, and survey our work towards this goal: discovering skills from experience, deriving compact models from those skills, and transferring both to new tasks. Skills, in particular, can be composed so that new tasks are solved with no further learning. Together, these threads work towards a single agent capable, in principle, of every task its body allows.

An important problem in reinforcement learning is designing agents that learn to solve tasks safely in an environment. A common solution is to define either a penalty in the reward function or a cost to be minimised when reaching unsafe states. However, designing reward or cost functions to incentivise safe behaviours with maximal probability is non-trivial and can increase with the complexity of the problem. To address this, we propose a simple algorithm for agents to automatically learn the penalty for unsafe transitions while learning to solve the given task. Precisely, the agent keeps track of its lowest and highest estimated state-values, then simply uses their difference as the penalty for unsafe transitions. Importantly, our approach can be used with any base RL algorithm since it only modifies the rewards perceived by an agent, and introduces no new hyperparameters that would require tuning. Our experiments demonstrate the effectiveness of this approach in enabling agents to learn safe policies in high-dimensional continuous control environments.

We consider the problem of unsupervised skill segmentation and hierarchical structure discovery in reinforcement learning. While recent approaches have sought to segment trajectories into reusable skills or options, most rely on action labels, rewards, or handcrafted annotations, limiting their applicability. We propose a method that segments unlabelled trajectories into skills and induces a hierarchical structure over them using a grammar-based approach. The resulting hierarchy captures both low-level behaviours and their composition into higher-level skills. We evaluate our approach in high-dimensional, pixel-based environments, including Craftax and the full, unmodified version of Minecraft. Using metrics for skill segmentation, reuse, and hierarchy quality, we find that our method consistently produces more structured and semantically meaningful hierarchies than existing baselines. Furthermore, as a proof of concept, we demonstrate that these discovered hierarchies accelerate and stabilise learning on downstream reinforcement learning tasks.

We consider how to construct state abstractions compatible with a given set of abstract actions, to obtain a well-formed abstract Markov decision process (MDP). We show that the Bellman equation suggests that abstract states should represent distributions over states in the ground MDP; we characterize the conditions under which the resulting process is Markov and approximately model-preserving, derive algorithms for constructing and planning with the abstract MDP, and apply them to a visual maze task. We generalize these results to the factored actions case, characterizing the conditions that result in factored abstract states and apply the resulting algorithm to Montezuma’s Revenge. These results provide a powerful and principled framework for constructing neurosymbolic abstract Markov decision processes.