Journal article
Mechanistic Interpretability of In-Context Learning: A Survey of Known Circuits and Open Problems
In-context learning (ICL) — the ability of a transformer language model to acquire a new input-output mapping from a handful of demonstrations in its prompt, with no weight updates — remains the most striking and least understood capability of large language models. Mechanistic interpretability…