Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 8df3c68

Browse files
committed
notes updates
1 parent 30c698b commit 8df3c68

File tree

4 files changed

+17
-2
lines changed

4 files changed

+17
-2
lines changed

‎_notes/assets/ttt_lm.jpeg‎

383 KB
Loading[フレーム]

‎_notes/neuro/comp_neuro.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1025,6 +1025,7 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
10251025
- ![fedorenko_fig1](../assets/fedorenko_fig1.png)
10261026
- language areas engage during both comprehension and production; are input and output modality-independent
10271027
- damage to left-hemisphere frontal/temporal brain areas leads to aphasia (deficits in language comprehension and production)
1028+
- Language is widely distributed throughout the brain ([drijvers, small, & skipper, 2025](https://www.nature.com/articles/s41583-024-00903-0)) - respond that rather than a "language network", the ‘language network’ could more simply be conceived of as a collection of hierarchically organized auditory association cortices communicating with functional connectivity hubs that coordinate a whole-brain distribution of contextually determined and, thus, highly variable ‘peripheral’ regions
10281029
- Semantic encoding during language comprehension at single-cell resolution ([jamali...fedorenko, williams, 2024](https://www.nature.com/articles/s41586-024-07643-2))
10291030
- interpreting brain encoding models
10301031
- [Brains and algorithms partially converge in natural language processing](https://www.nature.com/articles/s42003-022-03036-1#Sec9) (caucheteux & king, 2022)
@@ -1040,7 +1041,7 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
10401041
- Seminal language-semantics fMRI study ([huth...gallant, 2016](https://www.nature.com/articles/nature17637)) - build mapping of semantic concepts across cortex using word vecs
10411042
- Crafting Interpretable Embeddings for Language Neuroscience by Asking LLMs Questions ([benara et al. 2024](https://openreview.net/pdf?id=mxMvWwyBWe))
10421043
- A generative framework to bridge data-driven models and scientific theories in language neuroscience ([antonello et al. 2024](https://arxiv.org/abs/2410.00812))
1043-
- [(caucheteux, gramfort, & king, facebook, 2022)](https://www.nature.com/articles/s41598-022-20460-9) - predicts fMRI with gpt-2 on the narratives dataset
1044+
- Deep language algorithms predict semantic comprehension from brain activity [(caucheteux, gramfort, & king, facebook, 2022)](https://www.nature.com/articles/s41598-022-20460-9) - predicts fMRI with gpt-2 on the narratives dataset
10441045
- GPT‐2 representations predict fMRI response + extent to which subjects understand corresponding narratives
10451046
- compared different encoding features: phoneme, word, gpt-2 layers, gpt-2 attention sizes
10461047
- brain mapping finding: auditory cortices integrate information over short time windows, and the fronto-parietal areas combine supra-lexical information over long time windows
@@ -1065,7 +1066,12 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
10651066
- decoding models
10661067
- [Semantic reconstruction of continuous language from non-invasive brain recordings](https://www.biorxiv.org/content/10.1101/2022.09.29.509744v1) (lebel, jain, & huth, 2022) - reconstruct continuous natural language from fMRI
10671068
- [Decoding speech from non-invasive brain recordings](https://arxiv.org/abs/2208.12266) (defossez, caucheteux, ..., remi-king, 2022)
1068-
- Bilingual language processing relies on shared semantic representations that are modulated by each language ([chen...klein, gallant, deniz, 2024](https://www.biorxiv.org/content/10.1101/2024.06.24.600505v1)) - shared semantic representations are modulated by each language
1069+
- multilingual stuff
1070+
- Bilingual language processing relies on shared semantic representations that are modulated by each language ([chen...klein, gallant, deniz, 2024](https://www.biorxiv.org/content/10.1101/2024.06.24.600505v1)) - shared semantic representations are modulated by each language
1071+
- An investigation across 45 languages and 12 language families reveals a universal language network ([malik-moraleda...fedorenko, 2022](https://www.nature.com/articles/s41593-022-01114-5#data-availability))
1072+
- Multilingual Computational Models Reveal Shared Brain Responses to 21 Languages ([gregor de varda, malik-moraleda...tuckute, fedorenko, 2025](https://www.biorxiv.org/content/10.1101/2025.02.01.636044v1))
1073+
- Constructed languages are processed by the same brain mechanisms as natural languages ([malik-moraleda...fedorenko, 2023](https://www.biorxiv.org/content/10.1101/2023.07.28.550667v2))
1074+
10691075

10701076
## theories of explanation
10711077

‎_notes/research_ovws/ovw_llms.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -420,6 +420,13 @@ See related papers in the [📌 interpretability](https://csinva.io/notes/resear
420420
- Diffusion-LM Improves Controllable Text Generation ([lisa li, thickstun, gulrajani, liang, & hashimoto, 2022](https://arxiv.org/abs/2205.14217))
421421
- Mixture of Soft Prompts for Controllable Data Generation ([chen, lee, ..., yu, 2023](https://arxiv.org/pdf/2303.01580.pdf)) - trains a small model on data from a big frozen LLM that is then more controllable
422422

423+
### test-time training
424+
425+
- Learning to (Learn at Test Time): RNNs with Expressive Hidden States ([sun...guestrin, 2024](https://arxiv.org/abs/2407.04620))
426+
- ![ttt_lm](../assets/ttt_lm.jpeg)
427+
- Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate ([wang...chen, 2025](https://arxiv.org/abs/2501.17703))
428+
- s1: Simple test-time scaling ([muennighof...hashimoto, 2025](https://arxiv.org/pdf/2501.19393))
429+
423430
# misc
424431

425432
## adaptation / transfer
@@ -770,6 +777,7 @@ Editing is generally very similar to just adaptation/finetuning. One distinction
770777
- Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves ([deng...gu, 2023](https://arxiv.org/abs/2311.04205))
771778
- SafeDecoding ([xu...poovendran, 2024](https://arxiv.org/pdf/2402.08983#page=3.89))
772779
- Hierarchical instruction following ([wallace..beutel, 2024](https://arxiv.org/abs/2404.13208))
780+
- Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming ([anthropic 2025](https://arxiv.org/pdf/2501.18837)) - use constitution to generate synthetic harmful/harmless texts and train classifiers on them
773781

774782

775783
**Attacks**
@@ -1571,6 +1579,7 @@ mixture of experts models have become popular because of the need for (1) fast s
15711579
## clinical/medical papers
15721580

15731581
- Self-Verification Improves Few-Shot Clinical Information Extraction ([gero et al. 2023](https://arxiv.org/abs/2306.00024))
1582+
- Universal Abstraction: Harnessing Frontier Models to Structure Real-World Data at Scale ([wong...poon, 2025](https://arxiv.org/abs/2502.00943)) - specialized prompt template for extracting attributes using LLM
15741583
- MedCalc-Bench: Evaluating Large Language Models for Medical Calculations ([khandekar...lu, 2024](https://arxiv.org/abs/2406.12036)) - create examples / questions from popular MDCalc guidelines
15751584
- LLMs are Few-Shot Clinical Information Extractors ([agrawal...sontag, 2022](https://arxiv.org/abs/2205.12689)) - use GPT3
15761585
- Health system-scale language models are all-purpose prediction engines ([NYU 2023](https://www.nature.com/articles/s41586-023-06160-y))

‎assets/cv_chandan.pdf‎

25 Bytes
Binary file not shown.

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /