doi: 10.1038/s41467-018-06561-y.
Rats adopt the optimal timescale for evidence integration in a dynamic environment
Affiliations
- PMID: 30323280
- PMCID: PMC6189050
- DOI: 10.1038/s41467-018-06561-y
Item in Clipboard
Rats adopt the optimal timescale for evidence integration in a dynamic environment
Alex T Piet et al.
Nat Commun.
.
Display options
Format
Display options
Format
Abstract
Decision making in dynamic environments requires discounting old evidence that may no longer inform the current state of the world. Previous work found that humans discount old evidence in a dynamic environment, but do not discount at the optimal rate. Here we investigated whether rats can optimally discount evidence in a dynamic environment by adapting the timescale over which they accumulate evidence. Using discrete evidence pulses, we exactly compute the optimal inference process. We show that the optimal timescale for evidence discounting depends on both the stimulus statistics and noise in sensory processing. When both of these components are taken into account, rats accumulate and discount evidence with the optimal timescale. Finally, by changing the volatility of the environment, we demonstrate experimental control over the rats' accumulation timescale. The mechanisms supporting integration are a subject of extensive study, and experimental control over these timescales may open new avenues of investigation.
Conflict of interest statement
The authors declare no competing interests.
Figures
Dynamic Clicks Task structure and example trial. a Schematic of task events and timing. A center light illuminates indicating the rat may initiate a trial by poking its nose into a center port. Auditory clicks are generated from state-dependent Poisson processes (the two states are schematized by light green and light blue backgrounds) and played concurrently from left and right speakers. The hidden state toggles between two states according to a telegraph process with hazard rate h. When the auditory clicks end, and the center light turns off, the rats must infer which of the two states the trial ended in and report their decision by poking into one of two reward ports. Trials have random durations so the rat must be prepared to answer at all time points. b An example trial. The hidden state transitions randomly, and the auditory clicks (black triangles) are generated accordingly. The optimal inference process (black line; see text for its derivation) accumulates clicks, and discounts accumulated evidence proportionally to the volatility of the environment and click statistics. For the optimal process, a choice is generated at the end of the trial according to whether the optimal inference variable is above or below 0
Optimal discounting rates depends on click reliability and can be well-approximated by linear discounting. a, b The reliability κ of each click depends on the Poisson click rates r1 and r2, as well as the click mislocalization probability n. a
κ as a function of click rates with no sensory noise (black), and with the average level of sensory noise reported in Brunton, 2013 (pink). Vertical dashed line shows the click rates used in the study. b The reliability of each click depends on how consistently each click can be correctly localized to the side that generated it. At 50% mislocalization each click contains no information about the current state, so κ = 0. The pink dot uses the average level of sensory noise. The grey dot uses an intermediate level of the sensory noise. All values displayed here use the click rates used in the study (38 Hz/2 Hz). c Discounting functions for the three sensory noise levels in (b) (same colors). Increasing sensory noise causes the discounting functions to weaken. Horizontal lines show average clks/sec in each of the two states. d Histogram of changes of mind produced by the optimal inference equation. Timing is relative to the last change in the hidden state. (Black) Inference without sensory noise, (pink) inference with average rat level of sensory noise. e The optimal nonlinear discounting function can be approximated by a linear discounting function. If the linear discounting function is tuned appropriately, accuracy is close to the full nonlinear function. f Comparison between optimal nonlinear discounting function (blue) and the best linear approximation (black), in terms of average accuracy for different noise levels. The best linear approximation is effectively equivalent. Arrow indicates parameter values used in (d). g The best linear discounting rate λ as a function of sensory noise. Increasing sensory noise decreases the discounting rate. The best linear function is found numerically on a set of 30k trials, which produces some variability for different noise levels. Pink dot indicates average rat sensory noise
Rats discount evidence. a Reverse correlation curves for an example rat reveals how clicks at each time point influence the rat’s decision. Error bars show standard deviation. b Reverse correlation curves for 14 rats. Error bars are omitted for clarity. c Reverse correlation curves for a range of simulated linear discounting agents. Black to white lines indicate increasing discounting rates (λ). Only the reverse correlation curve for the right choice are shown for clarity. Each curve was fit with an exponential function (example red). The fit parameters are used in (d). d Exponential fit to each discounting agent recovers the generative linear discounting rate. Example in (c) show with red dot. Error bars show 95% confidence intervals of exponential fit
Rats discount evidence with the optimal timescale. a Example reverse correlation curve for one rat (blue and green), and the reverse correlation curve from the optimal linear inference agent (pink) with the average rat sensory noise from Brunton, 2013. The optimal linear inference agent was simulated on the same trials the rat performed. Shaded area shows one standard deviation. b Quantification of discounting timescales. Rat integration timescales plotted with optimal agents with no sensory noise (black), or with sensory noise (pink). The variability in optimal discounting rates is a result of measuring the reverse correlation curves on a different set of trials each rat actually performed. Error bars omitted for clarity
Trial-by-trial model captures rat behavior, and shows optimal linear discounting. a Example reverse correlation curves generated by the trial-by-trial model (light red) compared with a rat’s behavior (blue and green). Shaded area is one standard deviation. b–d Error bars show 95% confidence intervals. b Best fitting discounting rates for rats trained on the dynamic task (light red), and for rats trained in a static environment (blue, data and fits from Brunton, 2013). c Each rat’s noise level and discounting rate (light red) compared to the optimal linear trade-off (black). d Each rat’s evidence discounting parameter compared to the accuracy maximizing discounting level. e The average accuracy for the model fit to each rat’s behavior, and optimized to maximize accuracy
Rats adapt to changing environmental conditions. a Schematic outlining the experimental design. Four rats were moved from a 0.5 Hz hazard rate to 0 Hz, then back to 0.5 Hz. Rats stayed in each environment for multiple daily training sessions, with a minimum of 25 sessions. b Quantification of evidence discounting rates before, during, and after the switch for the combined rat dataset using the trial-by-trial model. c Evidence discounting rate during the switch for one rat using the trial-by-trial model on consecutive blocks of 7500 trials. Only the evidence discounting term was fit. b, c Error bars show 95% confidence intervals
References
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources