NYU // Learning and Memory // Spring 2023 // Gureckis and Hartley

#1. Introduction/Overview

Introduction to class for people contemplating registering. Overview of syllabus, instructors, requirements, grading, etc... Introduction of students.

#2. What is learning exactly?

Historical ideas and the birth of the modern science of learning. Additional topics include learning/performance, innate behaviors versus adaptation (nature/nurture), critical periods, models and mechanisms, and levels of analysis.

Readings:

1. Textbook reading: Eichenbaum, Chapter 1 - The nature of learning and memory
2. Meltzoff, A.N., Kuhl, P.K., Movellan, J. and Sejnowski, T.J. (2009) "Foundations for a New Science of Learning" Science, 325, 284-288. [PDF]
3. Zador, A. (2019) A critique of pure learning and what artificial neural networks can learn from animal brains. Nature Communications, 10, 3770. [PDF]

#3. Basic concepts in the neuroscience of learning and memory

In the following weeks we will explore a number of basic phenomena of learning. However, it is helpful to begin by casting these ideas against the backdrop of contemporary neuroscience. Today's lecture will be a basic whirl-wind tour of the neural processes thought to underly learning and memory. We'll talk about the function of neurons, the specialization of function in the brain, basic learning mechanisms (hebbian learning, LTP), and modern techniques for studying learning and memory (fMRI, EEG, etc...).

Readings:

1. Textbook reading: Eichenbaum, Chapter 2 - The neural bases of learning and memory
2. Scoville, W.B. and Milner, B. (1957) "Loss of Recent Memory After Bilateral Hippocampal Lesions" Journal of Neurology, Neurosurgery and Psychiatry, 20, 11-21. [PDF]
3. Squire, L.R. (1992) "Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory" Journal of Cognitive Neuroscience, 4 (3), 232-243. [PDF]
4. Josselyn, S.A. and Tonegawa, S. (2020). Memory engrams: Recalling the past and imagining the future. Science, 367(6473), eaaw4325. [PDF]

#4. Unsupervised and perceptual learning

This lecture will cover non-associative forms of learning (habituation/sensitation), unsupervised learning, perceptual learning, latent learning, feature learning, stimulus-stimulus learning, statistical learning, imprinting, priming, repetition suppression.

Readings:

1. Textbook reading: Eichenbaum, Ch. 3&4 - Simple Forms of Learning and Memory/Perceptual Learning and Memory
2. Aslin, R.N. and Newport, E.L. (2012) Statistical learning: From acquiring specific items to forming general rules. Current Directions in Psychological Science, 21 (3), 170–176. [PDF]
3. Goldstone, R.L. (1998) "Perceptual Learning" Annual Review of Psychology, 49, 585-612. [PDF]
4. Barlow, H.B. (1989) "Unsupervised Learning" Neural Computation, 1, 295-311. [PDF]

#5. Classical conditioning I

Pavlov, basic procedure, phenomena and terms (CS/US, etc...), basic findings, blocking and overshadowing, etc..., Resorla-Wagner model, Pearce-Hall model and the role of attention/associability in classical conditioning, basic neural substrates of classical conditioning, interactions with other learning systems (e.g., role of hippocampus in trace conditioning).

Readings:

1. Textbook reading: Eichenbaum, Ch. 5 - Procedural Learning I: Classical Conditioning
2. Rescorla, R.A. (1998) "Pavlovian Conditioning: It's not what you think it is" American Psychologist, 43(4), 151-160. [PDF]
3. Rescorla, R.A. and Wagner, A.R.(1971) "A Theory of Pavlovian Conditioning: Variations in the Effectiveness of Reinforcement and Non-reinforcement" in Black, A.H. & Prokasy, W.F. (eds.), Classical conditioning II: Current research and theory (pp. 64-99). New York: Appleton-Century-Crofts. [PDF]
4. Courville, A., Daw, N.D., Touretsky, D.S. (2006) Bayesian theories of conditioning in a changing world. Trends in Cognitive Sciences, 10(7). 294-300. [PDF]

#6. Classical conditioning II

modern theories including causal interpretations of classical conditioning, context-dependent learning, second-order condition (temporal-difference model and relationship to Rescorla-Wagner), neural basis of prediction errors

Readings:

1. Niv, Y. and Schoenbaum, G. (2008) "Dialogues on prediction errors" Trends in Cognitive Science, 12(7), 265-72. [PDF]
2. Schultz, W., Dayan, P. & Montague, P.R. (1997) "A neural substrate of prediction and reward" Science, 275, 1593. [PDF]
3. Gershman, S.J., and Blei, D. and Niv, Y. (2009) "Context, learning, and extinction" Psychological Review, 117(1), 197-209. [PDF]

#7. Instrumental conditioning I

law of effect, role of reinforcement, stimulus control, choice behavior, matching law, melioration, concurrent schedules, self control/impulsivity, habits and planning, superstitious responding (special thanks to nathaniel daw for sharing slides and thoughts on the instrumental section)

Readings:

1. Textbook reading: Eichenbaum, Ch. 6 - Procedural Learning II: Habits and Instrumental Conditioning
2. Herrnstein, R.J. (1970) "On the law of effect" Journal of the Experimental Analysis of Behavior, 13, 243-266. [PDF]
3. Skinner, B.F. (1948) "Superstition in the Pigeon" Journal of Experimental Psychology, 38, 168-172. [PDF]
4. Dickinson, A. (1985) "Actions and Habits: The Development of Behavioral Autonomy" Philosophical Transactions of the Royal Society of London. Series B, Biological, 38, 168-172. [PDF]

#8. Instrumental conditioning II

computational reinforcement learning, model-based and model-free learning algorithms

Readings:

1. Silver, D., Sigh, S., Precup, D. and Sutton, R.S. (20210) Reward is enough. Artificial Intelligence, 299, 103535. [PDF]
2. Tolman, E.C. (1948) "Cognitive Maps in Rats and Men" Psychological Review, 55(4), 189-208. [PDF]
3. Daw, N., Gershman, S.J, Seymour, B., Dayan, P. and Dolan, R.J. (2011) Model-based influences on humans' choices and striatal prediction errors. Neuron, 69, 6, 1204-1215. [PDF]

#9. Generalization and discrimination

Pearce (configural) vs. R-W (elemental), stimulus generalization, attention learning, context dependent learning

Readings:

1. Mitchell, T.M. (1980). The need for biases in learning generalizations (Report CBM- TR-5-110). New Brunswick, NJ: Rutgers University, Department of Computer Science. [PDF]
2. Shepard, R.N. (1987) "Toward a universal law of generalization for psychological science" Science, 237(4820), 1317-1323. [PDF]
3. Dunsmoor, JE and Murphy, GL (2015) Categories, concepts, and conditioning: how humans generalize fear Trends in cognitive sciences 19 (2), 73-77. [PDF]

#10. Episodic memory

introduction, episodic memory, hippocampus and space, flexibility, interactions with striatum and cortex

Readings:

1. Textbook reading: Eichenbaum, Ch. 8 - Cognitive memory
2. Davachi, L., Mitchell, J.P., and Wager, A.D. (2003) Multiple routes to memory: Distinct medial temporal lobe processes build item and source memories. Proceedings of the National Academy of Science, 100 (4) 2157-2162 [PDF]
3. Lisman, J., Buzsáki, G., Eichenbaum, H., Nadel, L., Ranganath, C., & Redish, A.D. (2017) Viewpoints: how the hippocampus contributes to memory, navigation and cognition. Nature Neuroscience 20, 1434–1447. [PDF]
4. Stachenfeld, K.L., Botvinick, M.M., and Gershman, S.J. (2017) The hippocampus as a predictive map. Nature Neuroscience, 20(11), 1643-1653. [PDF]
5. Shiffrin, R.M. and Steyvers, M. (1997) A model for recognition memory: REM -- retrieving effectively from memory. Psychonomic Bulletin & Review, 4, 145-166. [PDF]

#11. Memory consolidation and complementary learning systems

consolidation, complementary learning systems hypothesis, catastrophic interference, specificity and abstraction, recognition versus recall

Readings:

1. Textbook reading: Eichenbaum, Ch. 11 - Memory consolidation
2. McClelland, J.L. and McNaughton, B.L., and O'Reilly RC. (1995) Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory Psychological Review, 102(3), 419-457 [PDF]
3. Tse D., Langston R.F., Kakeyama M., Bethus I., Spooner P.A., Wood E.R., Witter M.P. and Morris R.G.M. (2007) "Schemas and memory consolidation" Science, 316 (5821), 76 [PDF]
4. Leutgeb JK, Leutgeb S, Moser MB, Moser EI. (2007) Pattern separation in the dentate gyrus and CA3 of the hippocampus. Science, 315(5814):961-6. [PDF]

#12. Context and memory

events, temporal context models, decay versus interference

Readings:

1. Kurby, C.A. and Zacks, J.M., (2008) Segmentation in the perception and memory of events. Trends in Cognitive Science, 12(2), 72-79.
2. Howard, M.W. and Kahana, M.J. (2001) A distributed representation of temporal context. Journal of Mathematical Psychology. 46(3), 269-299.

#13. Semantic memory

semantic memory, models of semantic memory, episodic-semantic memory interactions

Readings:

1. Textbook reading: Eichenbaum, Ch. 10 - Semantic Memory
2. Tenenbaum, J.B., Kemp, C., Griffiths, T.L., and Goodman, N.D. (2011) How to grow a mind: Statistics, structure and abstraction. Science, 331(6022), 1279-1285
3. Jones, M. N., Willits, J. A., & Dennis, S. (2015). Models of semantic memory. In J. R. Busemeyer & J. T. Townsend (Eds.) Oxford Handbook of Mathematical and Computational Psychology.
4. Bhatia, S. and Richie, R. (2022) Transformer networks of human conceptual knowledge. Psychological Review

#14. The science of being a better learner

how to use science of learning and memory to be a better student?

Readings:

1. Schacter, D. L. (1999). The seven sins of memory: Insights from psychology and cognitive neuroscience.American Psychologist, 54(3), 182–203. [PDF]
2. Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J., and Willingham, D.T. (2013) Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology Psychological Science in the Public Interest, 14(1), 4-58. [PDF]
3. Karpicke, J.D. (2012) Retrieval-Based Learning: Active Retrieval Promotes Meaningful Learning Current Directions in Psychological Science 21(2),157-163. [PDF]