Neural activity encoding recent experiences is replayed during sleep and rest to promote consolidation of memories. However, precisely which features of experience influence replay prioritisation to optimise adaptive behaviour remains unclear. Here, we trained adult male rats on a novel maze-based reinforcement learning task designed to dissociate reward outcomes from reward-prediction errors. Four variations of a reinforcement learning model were fitted to the rats' behaviour over multiple days. Behaviour was best predicted by a model incorporating replay biased by reward-prediction error, compared to the same model with no replay, random replay or reward-biased replay. Neural population recordings from the hippocampus and ventral striatum of rats trained on the task evidenced preferential reactivation of reward-prediction and reward-prediction error signals during post-task rest. These insights disentangle the influences of salience on replay, suggesting that reinforcement learning is tuned by post-learning replay biased by reward-prediction error, not by reward per se. This work therefore provides a behavioural and theoretical toolkit with which to measure and interpret the neural mechanisms linking replay and reinforcement learning.

Axons of dopaminergic neurons express gamma-aminobutyric acid type-A receptors (GABARs) and nicotinic acetylcholine receptors (nAChRs), which are positioned to shape striatal dopamine release. We examine how interactions between GABARs and nAChRs influence dopaminergic axon excitability. Axonal patch-clamp recordings reveal that potentiation of GABARs by benzodiazepines suppress dopaminergic axon responses to cholinergic interneuron transmission. In imaging experiments, we use the first temporal derivative of axonal calcium signals to distinguish between direct stimulation of dopaminergic axons and nAChR-evoked activity. Inhibition of GABARs with gabazine selectively enhance nAChR-evoked axonal calcium signals but does not alter the strength or dynamics of acetylcholine release, suggesting that the enhancement is mediated primarily by GABARs on dopaminergic axons. Unexpectedly, we find that a widely used GABAR antagonist, picrotoxin, inhibits axonal nAChRs and should be used cautiously for striatal circuit analysis. Overall, we demonstrate that GABARs on dopaminergic axons regulate integration of nicotinic input to shape axonal excitability.

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes from preceding cues or actions. For example, reward outcomes may be linked to specific stimuli and actions, corresponding to stimulus- and action-based learning. But how does the brain arbitrate between such models? Here, we combined multiple computational approaches to quantify concurrent learning in male monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for a dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala in model arbitration. We demonstrated that the amygdala adjusts the initial balance between the two learning systems and is essential for updating arbitration according to the correct model, which in turn alters the interaction between arbitration and learning that governs the time course of learning and choice behavior. In contrast, VS lesions lead to an overall reduction in stimulus-value signals. This role of the amygdala reconciles existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

Head-mounted miniaturized two-photon microscopes enable cellular-resolution recording of neural activity deep in the mouse brain during unrestrained behavior. Two-photon microscopy, however, is traditionally limited in frame rate by the necessity of scanning the excitation beam over a large field-of-view (FOV). Here, we present two types of multiplexed miniaturized two-photon microscopes (M-MINI2Ps) that preserve spatial resolution while increasing frame rate by simultaneously imaging two FOVs and demixing them temporally or computationally. We demonstrate large-scale (500 × 500 μm FOV) multiplane calcium imaging in visual and prefrontal cortices of freely moving mice during spontaneous exploration, social behavior, and auditory stimulus. The increased speed of M-MINI2Ps also enables two-photon voltage imaging at 400 Hz over a 380 × 150 μm FOV in freely moving mice. With compact footprints and compatibility with the open-source MINI2P, M-MINI2Ps enable high-speed recording of rapid neural dynamics and large-volume population activity in freely moving mice, providing a powerful tool for systems neuroscience.

In order to understand the retinal network, it is essential to identify functional connectivity among retinal neurons. For this purpose, imaging neuronal activity through fluorescent indicator proteins has been a promising approach, offering simultaneous measurements of neuronal activities from different regions of the circuit. In this study, we used genetically encoded indicators─Bongwoori-R3 for voltage or GCaMP6f for calcium─to visualize membrane voltage or calcium dynamics, respectively, as spatial maps within individual retinal ganglion cells from retinal tissues of photoreceptor-degenerated mice. Retinal voltage imaging was able to show current-evoked somatic spiking as well as subthreshold voltage changes, while calcium imaging showed changes in calcium concentrations evoked by current pulses in retinal ganglion cells. These results indicate that the combination of fluorescent protein sensors and high-speed imaging methods permits the imaging of electrical activity with cellular precision and millisecond resolution. Hence, we expect our method will provide a potent experimental platform for the study of retinal signaling pathways, as well as the development of retinal stimulation strategies in visual prosthesis.

Flexible control of motor timing is crucial for behaviour. Before volitional movement begins, the frontal cortex and striatum exhibit ramping spiking activity, with variable ramp slopes anticipating movement onsets. This activity in the cortico-basal ganglia loop may function as an adjustable 'timer,' triggering actions at the desired timing. However, because the frontal cortex and striatum share similar ramping dynamics and are both necessary for timing behaviours, distinguishing their individual roles in this timer function remains challenging. Here, to address this, we conducted perturbation experiments combined with multi-regional electrophysiology in mice performing a flexible lick-timing task. Following transient silencing of the frontal cortex, cortical and striatal activity swiftly returned to pre-silencing levels and resumed ramping, leading to a shift in lick timing close to the silencing duration. Conversely, briefly inhibiting the striatum caused a gradual decrease in ramping activity in both regions, with ramping resuming from post-inhibition levels, shifting lick timing beyond the inhibition duration. Thus, inhibiting the frontal cortex and striatum effectively paused and rewound the timer, respectively. These findings are consistent with a model in which the striatum is part of a network that temporally integrates input from the frontal cortex and generates ramping activity that regulates motor timing.

All mammals exhibit flexible decision policies that depend, at least in part, on the cortico-basal ganglia-thalamic (CBGT) pathways. Yet understanding how the complex connectivity, dynamics, and plasticity of CBGT circuits translate into experience-dependent shifts of decision policies represents a longstanding challenge in neuroscience. Here we present the results of a computational approach to address this problem. Specifically, we simulated decisions during the early learning process driven by CBGT circuits under baseline, unrewarded conditions using a spiking neural network, and fit an evidence accumulation model to the resulting behavior. Using canonical correlation analysis, we then replicated the identification of three control ensembles (responsiveness, pliancy and choice) within CBGT circuits, with each of these subnetworks mapping to a specific configuration of the evidence accumulation process. We subsequently simulated learning in a simple two-choice task with one optimal (i.e., rewarded) target and found that, during early stages of learning, feedback-driven dopaminergic plasticity on cortico-striatal synapses effectively increases reward rate over time. The learning-related changes in the decision policy can be decomposed in terms of the contributions of each control ensemble, whose influence is driven by sequential reward prediction errors on individual trials. Our results provide a clear and simple mechanism for how dopaminergic plasticity shifts subnetworks within CBGT circuits so as to increase reward rate by strategically modulating how evidence is used to drive decisions.