Notebooks

Neural Coding

Last update: 21 Apr 2025 21:17
First version: 20 September 2001

And the statistics of neural spike trains more generally.

Since I've written about the neural coding problem, at great length, in my review of Spikes (see below), I won't repeat myself here.

Things to try to understand: Distributed and population codes. How much can be understood about coding without also understanding computation?

Things to do: Causal-state reconstruction on real neural spike data. (Done; see below.) Transducer state reconstruction; states of the inferred transducer = classes of stimuli (+ internal histories) which make a difference to the cell. The information coherence measure should indicate the quantity of distributed information in spike-trains. Calculate for actual neuronal circuits; does this interpretation make sense?

    Recommended, big picture:
  • Larry Abbott and Terry Sejnowski (eds.), Neural Codes and Distributed Representations
  • Chris Eliasmith and Charles Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems
  • Liam Paninski, Jonathan Pillow, and Jeremy Lewi, "Statistical models for neural encoding, decoding, and optimal stimulus design", to appear in P. Cisek, T. Drew and J. Kalaska (eds.), Computational Neuroscience: Progress in Brain Research [PDF preprint]
  • Alexandre Pouget, Peter Dayan and Richard S. Zemel, "Inference and Computation with Population Codes", Annual Review of Neuroscience 26 (2003): 381--401
  • Fred Rieke, David Warland, Rob de Ruyter van Steveninck and William Bialek, Spikes: Exploring the Neural Code [Review: Cells That Go ping, or, the Value of the Three-Bit Spike]
    Recommended, close-ups:
  • Jose M. Amigo, Janusz Szczepanski, Elek Wajnryb and Maria V. Sanchez-Vives, "Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity", Neural Computation 16 (2004): 717--736 [Normally, I have strong views on using Lempel-Ziv to measure entropy rates, but here they are using the 1976 Lempel-Ziv definitions, not the 1978 ones. The difference is subtle, but important; 1978 leads to gzip and practical compression algorithms, but very bad entropy estimates; 1976 leads, as they show numerically, to quite good entropy rate estimates, at least for some processes. Thanks to Dr. Szczepanski for correspondence about this paper.]
  • Riccardo Barbieri, Loren M. Frank, David P. Nguyen, Michael C. Quirk, Victor Solo, Matthew A. Wilson and Emery N. Brown, "Dynamic Analyses of Information Encoding in Neural Ensembles", Neural Computation 16 (2004): 277--307
  • M. J. Barber, J. W. Clark and C. H. Anderson, "Neural Representation of Probabilistic Information," Neural Computation 15 (2003): 1843--1864, cond-mat/0108425
  • David Brillinger
    • "Nerve Cell Spike Train Data Analysis: A Progression of Technique," Journal of the American Statistical Association 87 (1992): 260--270
    • and Allessandro E. P. Villa, "Assessing Connections in Networks of Biological Neurons", pp. 77--92 in D. R. Brillinger, L. T. Fernholz and S. Morgenthaler (eds.), The Practice of Data Analysis: Essays in Honor of John W. Tukey [PS]
  • A. E. Brockwell, A. L. Rojas and R. E. Kass, "Recursive Bayesian Decoding of Motor Cortical Signals by Particle Filtering", Journal of Neurophysiology 91 (2004): 1899--1907 [Very nice, especially since they've combining data from multiple experiments. It is a little disappointing that they set up a state-space model, but then only use the state to enforce a kind of weak continuity constraint on the decoding, rather than trying to capture the actual computations going on. But I should talk to them about that... Appendix A gives a very clear and compact explanation of particle filtering.]
  • Uri T. Eden, Loren M. Frank, Riccardo Barbieri, Victor Solo and Emery N. Brown, "Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering", Neural Computation 16 (2005): 971-988
  • Nicol S. Harper and David McAlpine, "Optimal neural population coding of an auditory spatial cue", Nature 430 (2004): 682--686
  • Jerome Y. Lettvin, H. R. Maturana, Warren S. McCulloch and W. H. Pitts, "What the Frog's Eye Tells the Frog's Brain", Proceedings of the IRE, 47 (1959): 1940--1951 [Reprinted in McCulloch's Embodiments of Mind, among other places, and available online, e.g., here]
  • Thomas Naselaris, Kendrick N. Kay, Shinji Nishimoto and Jack L. Gallant, "Encoding and decoding in fMRI", Neuroimage 56 (2011): 400--410, PMC3037423
  • Jonathan W. Pillow, Yashar Ahmadian and Liam Paninski, "Model-Based Decoding, Information Estimation, and Change-Point Detection Techniques for Multineuron Spike Trains", Neural Computation 23 (2011): 1--45
  • R. Quian Quiroga, L. Reddy, G. Kreiman, C. Koch and I. Fried, "Invariant visual representation by single neurons in the human brain", Nature 435 (2005): 1102--1107
  • Tatyana Sharpee, Nicole C. Rust and William Bialek, "Analyzing neural responses to natural signals: maximally informative dimensions", Neural Computation 16 (2004): 223--250, physics/0212110
  • S. P. Strong, Roland Koberle, Rob de Ruyter van Steveninck and William Bialek, "Entropy and Information in Neural Spike Trains," Physical Review Letters 80 (1998): 197--201
  • Eric E. Thomson and William B. Kristan, "Quantifying Stimulus Discriminability: A Comparison of Information Theory and Ideal Observer Analysis", Neural Computation 17 (2005): 741--778 [A useful warning against a too-common abuse of information theory. Thanks to Eric for providing me with a pre-print.]
  • Shreejoy J. Tripathy, Krishnan Padmanabhan, Richard C. Gerkin, and Nathaniel N. Urban, "Intermediate intrinsic diversity enhances neural population coding", Proceedings of the National Academy of Sciences (USA) 110 (2013): 8248--8253
  • Jonathan D. Victor and Keith P. Purpura, "Metric-Space Analysis of Spike Trains: Theory, Algorithms and Application," Network: Computation in Neural Systems 8 (1997): 127--164
  • Vincent Q. Vu, Bin Yu, Robert E. Kass, "Information In The Non-Stationary Case", arxiv:0806.3978 ### Modesty forbids me to recommend:
  • Rob Haslinger, Kristina Klinkner and CRS, "The Computational Structure of Spike Trains", Neural Computation 22 (2010): 121--157, arxiv:1001.0036 ### To read:
  • Craig A. Atencio, Tatyana O. Sharpee, and Christoph E. Schreiner, "Hierarchical computation in the canonical auditory cortical circuit", Proceedings of the National Academy of Sciences (USA) 106 (2009): 21894--21899
  • E. D. Adrian, Physical Background of Perception [Adrian was one of the first --- maybe the first? --- to record spike trains from neurons, and realize they were how neurons communicate]
  • Blaise Aguera y Arcas and Adrienne Fairhall, "What causes a neuron to spike?" physics/0301014
  • Blaise Aguera y Arcas, Adrienne L. Fairhall and William Bialek, "Computation in a single neuron: Hodgkin and Huxley revisited," physics/0212113
  • Kazuyuki Aihara and Isao Tokuda, "Possible neural coding with interevent intervals of synchronous firing," Physical Review E 66 (2002): 026212
  • Vijay Balasubramanian and Michael J. Berry, "Metabolically Efficient Codes in The Retina," cond-mat/0105128
  • Vijay Balasubramanian, Don Kimber and Michael J. Berry, "Metabolically Efficient Information Processing," cond-mat/0105127
  • C. T. Bergstrom and M. Rosvall, "The transmission sense of information", arxiv:0810.4168
  • John A. Berkowitz and Tatyana O. Sharpee, "Quantifying Information Conveyed by Large Neuronal Populations", Neural Computation 31 (2019): 1015--1047
  • Michele Bezzi, Mathew E. Diamond and Alessandro Treves, "Redundency and synergy arising from correlations in large ensembles," cond-mat/0012119
  • Michele Bezzi, Ines Samengo, Stefan Leutgeb and Sheri Mizumori, "Measuring information spatial densities," cond-mat/0111150
  • William Bialek and Rob R. de Ruyter van Steveninck, "Features and dimensions: Motion estimation in fly vision", q-bio.NC/0505003
  • Naama Brenner, Steven P. Strong, Roland Koberle, William Bialek and Rob R. de Ruter van Steveninck, "Synergy in a Neural Code," Neural Computation, 12 (2000): 1531--1552
  • Lars Buesing and Wolfgang Maass, "A Spiking Neuron as Information Bottleneck", Neural Computation 22 (2010): 1961--1992
  • Daniel A. Butts, Chong Weng, Jianzhong Jin, Chun-I Yeh, Nicholas A. Lesica, Jose-Manuel Alonso and Garrett B. Stanley, "Temporal precision in the neural code and the timescales of natural vision", Nature 449 (2007): 92--95
  • C. E. Carr and M. A. Friedman, "Evolution of Time Coding Systems," Neural Computation 11 (1999): 1--20
  • Guillermo A. Cecchi, Mariano Sigman, Josée-Manuel Alonso, Luis Martínez, Dante r. Chialvo and Marcelo O. Magnasco, "Noise in Neurons is Message-Dependent," cond-mat/0004492
  • Mircea I. Chelaru and Valentin Dragoi, "Efficient coding in heterogeneous neuronal populations", Proceedings of the National Academy of Sciences 105 (2008): 16344--16349
  • Yuzhi Chen, Wilson S. Geisler and Eyal Seidemann, "Optimal decoding of correlated neural population responses in the primate visual cortex", Nature Neuroscience 9 (2006): 1412--1420 [This sounds cool, and of course I shouldn't comment before reading more than just the abstract, but of course I will anyway. "This optimal decoder consistently outperformed the monkey in the detection task, demonstrating the sensitivity of our techniques": yes, but doesn't that by the same token inidcate their irrelevance to understanding the monkey's neural code?]
  • Marshall Crumiller, Bruce Knight, Yunguo Yu and Ehud Kaplan, "Estimating the amount of information conveyed by a population of neurons" [PDF preprint via Dr. Kaplan]
  • Isabel Dean, Nicol S Harper and David McAlpine, "Neural population coding of sound level adapts to stimulus statistics", Nature Neuroscience 8 (2005): 1684--1689
  • Coralie de Hemptinne, Sylvie Nozaradan, Quentin Duvivier, Philippe Lefevre, and Marcus Missal, "How Do Primates Anticipate Uncertain Future Events?", Journal of Neuroscience 27 (2007): 4334--4341
  • Valeria Del Prete, "A replica free evaluation of the neuronal population information with mixed continuous and discrete stimuli: from the linear to the asymptotic regime," cond-mat/0301457
  • David R. Euston and Bruce L. McNaughton, "Apparent Encoding of Sequential Context in Rat Medial Prefrontal Cortex Is Accounted for by Behavioral Variability", The Journal of Neuroscience 26 (2006): 13143--13155
  • Hugo G. Eyherabide, Ariel Rokem, Andreas V. M. Herz, Ines Samengo, "Burst firing is a neural code in an insect auditory system", arxiv:0807.2550
  • Adrienne L. Fairhall, Geofrrey D. Lewen, William Bialek and Robert R. de Ruyter van Steveninck, "Efficiency and Ambiguity in an Adaptive Neural Code," Nature 412 (2001): 787--792
  • Michael Famulare and Adrienne Fairhall, "Feature Selection in Simple Neurons: How Coding Depends on Spiking Dynamics", Neural Computation 22 (2010): 581--598
  • F. Gabbiani
  • Surya Ganguli, Dongsung Huh, and Haim Sompolinsky, "Memory traces in dynamical systems", Proceedings of the National Academy of Sciences (USA) 105 (2008): 18970--18975
  • Yun Gao, Ioannis Kontoyiannis, Elie Bienenstock, "From the entropy to the statistical structure of spike trains", arxiv:0710.4117
  • Ralf M. Haefner and Matthias Bethge, "Evaluating neuronal codes for inference using Fisher information", NIPS 23 (2010) [PDF]
  • Yi Hao, Alon Orlitsky, "Data Amplification: Instance-Optimal Property Estimation", arxiv:1903.01432
  • Kenneth D. Harris, "Neural Signatures of Cell Assembly Organization", Nature Reviews Neuroscience 6 (2005): 399--407
  • V. Hok, E. Save, P. P. Lenck-Santini and B. Poucet, "Coding for spatial goals in the prelimbic/inframlimbic area of the rat frontal cortex", PNAS 102 (2005): 4602--4607
  • Toshihiko Hosoya, Stephen A. Baccus and Markus Meister, "Dynamic predictive coding by the retina", Nature 436 (2005): 71--77
  • Wentao Huang and Kechen Zhang, "Information-Theoretic Bounds and Approximations in Neural Population Coding", Neural Computation 30 (2018): 885--944
  • Quentin J. M. Huys, Richard S. Zemel, Rama Natarajan, and Peter Dayan , "Fast Population Coding", Neural Computation 19 (2007): 404--441
  • Ole Jensen, "Information Transfer Between Rhythmically Coupled Networks: Reading the Hippocampal Phase Code," Neural Computation vol. 13 no. 12 (December 2001)
  • Christof Koch, Biophysics of Computation: Information Processing in Single Neuron
  • Philipp Knüsel, Reto Wyss, Peter König and Paul F.M.J. Verschure, "Decoding a Temporal Population Code", Neural Computation 16 (2004): 2079--2100
  • R. Krahe, G. Kreiman, F. Gabbiani, C. Koch, W. Metzner, "Stimulus encoding and feature extraction by multiple sensory neurons" [Reprint]
  • Nikolaus Kriegeskorte, Visual Population Codes: Towards a Common Multivariate Framework for Cell Recording and Functional Imaging
  • Petr Lansky and Priscilla E. Greenwood, "Optimal Signal Estimation in Neuronal Models", Neural Computation 17 (2005): 2240--2257
  • G. D. Lewen, W. Bialek and R. R. de Ruyter van Steveninck, "Neural coding of naturalistic motion stimuli," physics/0103088
  • Longnian Lin, Remus Osan, Shy Shoham, Wenjun Jin, Wenqi Zuo, and Joe Z. Tsien, "Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus", PNAS 102 (2005): 6125--6130
  • Christian K. Machens, "Adaptive sampling by information maximization," physics/0112070
  • Gary Marsat and Gerald S. Pollack, "A Behavioral Role for Feature Detection by Sensory Bursts", The Journal of Neuroscience 26 (2006): 10542--10547
  • Laura Martignon, Gustavo Deco, Kathryn Laskey, Mathew Diamond, Winrich Freiwald and Eilon Vaadia, "Neural Coding: Higher-Order Temporal Patterns in the Neurostatistics of Cell Assemblies," Neural Computation 12 (2000): 2621--2653
  • Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott, "Point singularities and suprathreshold stochastic resonance in optimal coding", cond-mat/0409528
  • Panzeri and Schultz, "A Unified Approach to the Study of Temporal, Correlational, and Rate Coding," Neural Computation 13 (2001): 1311--1349
  • Phillips and Singer, In Search of Common Foundations for Cortical Computation
  • Jonathan W. Pillow, Liam Paninski, Valerie J. Uzzell, Eero P. Simoncelli, and E. J. Chichilnisky, "Prediction and Decoding of Retinal Ganglion Cell Responses with a Probabilistic Spiking Model", The Journal of Neuroscience 25 (2005): 11003--11013
  • Jonathan W. Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher, Alan M. Litke, E. J. Chichilnisky and Eero P. Simoncelli, "Spatio-temporal correlations and visual signalling in a complete neuronal population", Nature 454 (2008): 995--999
  • Alessio Plebe and Vivian M. De La Cruz, "Neural Representations Beyond ``Plus X''", Minds and Machines 28 (2018): 93--117
  • K. Prank, F. Gabbiani and G. Brabant, "Coding efficiency and information rates in transmembrane signaling" [Abstract]
  • D. S. Reich, F. Mechler and J. D. Victor, "Independent and Redundant Information in Nearby Cortical Neurons", Science 294 (2001): 2566--2568
  • Hugh P. C. Robinson and Annette Harsch, "Stages of spike time variability during neuronal responses to transient inputs," Physical Review E 66 (2002): 061902
  • Enrico Rossoni and Jianfeng Feng, "Decoding spike train ensembles: tracking a moving stimulus", Biological Cybernetics 96 (2007): 99--112 [Improvements for some non-stationary situations through censored maximum likelihood estimation]
  • Rob de Ruyter van Steveninck and William Bialek, "Timing and Counting Precision in the Blowfly Visual System," physics/0202014
  • Ines Samengo, "Information loss in an optimal maximum likelihood decoding," physics/0110074
  • Elad Schneidman, William Bialek and Michael J. Berry, II, "An information theoretic approach to the functional classification of neurons," physics/0212114
  • Maoz Shamir and Haim Sompolinsky, "Nonlinear Population Codes", Neural Computation 16 (2004): 1105--1136
  • Tatyana Sharpee and William Bialek, "Neural Decision Boundaries for Maximal Information Transmission", q-bio.NC/0703046
  • Kyle H. Srivastava, Caroline M. Holmes, Michiel Vellema, Andrea R. Pack, Coen P. H. Elemans, Ilya Nemenman, and Samuel J. Sober, "Motor control by precisely timed spike patterns", Proceedings of the National Academy of Sciences (USA) 114 (2017): 1171--1176
  • Richard B. Stein, E. Roderich Gossen and Kelvin E. Jones, "Neuronal Variability: Noise or Part of the Signal?", Nature Reviews Neuroscience 6 (2005): 389--397
  • Michael Stiber, "Spike timing precision and neural error correction: local behavior", q-bio.NC/0501021
  • Giulio Tononi and Olaf Sporns, "Measuring information integration", Biomedcentral Neuroscience 4 (2003): 31 [Really more neural information theory than neural coding as such]
  • Nicholas Watters and George N. Reeke, "Neuronal Spike Train Entropy Estimation by History Clustering", Neural Computation 26 (2014): 1840--1872
  • Brian D. Wright, Kamal Sen, William Bialek and Allison J. Doupe, "Spike timing and the coding of naturalistic sounds in a central auditory area of songbirds," physics/0201027
  • Si Wu and Shun-ichi Amari, "Computing with Continuous Attractors: Stability and Online Aspects", Neural Computation 17 (2005); 2215--2239

permanent link for this note RSS feed for this note

Notebooks :

AltStyle によって変換されたページ (->オリジナル) /