How Guessing the Next Word Becomes a Conversation — The Minimum Foundation for Reading Language Models

What the Primer Established, and the Questions That Remain

If you are asked to guess the next word in this sentence, most people come up with a few candidates. This is what language models are trained to do. "How Generative AI Answers — A Sentence Built One Piece at a Time" called this next-token prediction. Based on tokens—the pieces that a language model chops text into and handles—it looks at the tokens placed in front and assigns a probability to each candidate for the token that will come next.

There is an experiment where people and language models were given this task to do identically. Whether measured by the rate at which the single most plausible choice was correct, or by how much probability was given to the correct answer, humans were consistently worse than a comparatively small language model (arXiv:2212.11281, preprint). The same paper starts from the fact that the language model was not trained to answer questions or write code, but to guess the next token. This result is a difference in the task they were trained on, and humans have never practiced that task. The training objective and the abilities we expect are separate axes, and no comparison can be read without checking what the paper measured.

The rest of the foundation was built by the primers. "How Generative AI Answers" followed pre-training, which repeats next-token guessing with an enormous amount of text; fine-tuning, which polishes it once more with selected examples; attention, which calculates how much to pay attention to which part of the input; and the Transformer, which uses that as its skeleton. Proposed in 2017 as a structure built with attention alone, the Transformer made calculations easy to split and run for translation tasks and had a much shorter training time (DOI: 10.65215/2q58a426). Attempts to explain why it has that exact shape are ongoing after the fact, and one preprint argues that shape follows when you mirror statistics of words appearing together in a corpus against the context of sentences (arXiv:2511.13780, preprint). "Neural Networks and Parameters — What It Means to Nudge Hundreds of Billions of Dials" established the observation that scaling up the model reduces loss—meaning the degree to which the model's answer strays from the correct answer—along with the debate surrounding whether abilities suddenly emerge in large models.

This article starts on top of that. There is only a single training objective, but the things we use are not things that received only that training. We follow the space between these in three questions. What traps does the token-chopping method create, what improves and how when you scale up the model, and what is the phenomenon where the model changes just through examples in a chat window without retraining? Between those, we take a closer look at the step of layering human preferences, and at the end, we record places where answers are still divided, keeping them divided.

Most of the studies cited here are preprints posted on arXiv, meaning versions uploaded before peer review. A preprint does not mean it is wrong, but it is not read with the same weight as results that have gone through review. The figures in this article are all values that the paper measured at that time under those conditions, and such values age quickly in this field.

The Trap Created by the Way We Cut into Tokens

In 2013, the authors who introduced the method of turning words into bundles of numbers wrote down its limitations themselves. The expression is indifferent to word order and fails to represent idioms, so that combining the meanings of "Canada" and "Air" does not yield "Air Canada" (DOI: 10.48550/arxiv.1310.4546, preprint). How you choose to cut text into units has been a choice that alters results ever since.

Today's language models cut text into tokens. In the words of "How Generative AI Answers", tokens are the units by which language models chop up and handle text, sometimes being a single word, and other times a fragment of a word or a punctuation mark. Subwords are pieces cut smaller than a word, where common words tend to remain whole pieces while rare words are split into multiple pieces. Language models usually pick pieces from a pre-determined list of subwords to cut text, and tokenizers are the tools that dictate how to cut text into tokens this way.

This cutting comes with a cost. The resulting cuts are easily shaken by typos or changes in text length, and models are mostly blind to which letters are contained inside a single token. A study calling this problem "the curse of tokenization" showed through three types of questions that large language models remain weak against this issue. Scaling up the model helps, but the biases created by typos and format changes remain, and the authors report that a technique of slightly shaking up the cutting method during training eases this problem (arXiv:2406.11687, preprint).

The cost of cutting differs across languages. If you translate the same text into multiple languages and count the tokens, the lengths vary wildly, with differences reaching up to 15 times in some cases. This gap remains even in tokenizers built to support multiple languages evenly. As a result, commercial service fees, processing times and delays, and the amount of text you can feed into a model at once become uneven across language communities (arXiv:2305.15425, preprint). This means you might pay a higher price depending on the language you use, even when asking the same thing.

It would seem that a tokenizer is better when token boundaries align well with morpheme boundaries, which are units of meaning in words. There is a study that measured this degree of alignment across 70 languages and compared it with the scores achieved by 5 pretrained models on 7 tasks. The degree of alignment with morpheme boundaries did not explain differences in scores very well (arXiv:2507.06378, preprint). There is still no consensus on what makes good tokenization.

One trap is even more subtle. In most language models, subword pieces carry the leading space in front of a word. Therefore, where a word ends is not determined by looking at that word's piece alone; you have to check whether the next piece starts with a space. To measure the difficulty humans feel when reading text, researchers often sum up the probabilities of pieces to use as the probability of a word, and it has been proved that this method can produce distributions where the sum of probabilities exceeds 1 (arXiv:2406.10851, preprint). The fact that basic rules probabilities must follow are broken like this shows that tokens are different from the units of language we think of. The authors showed that this problem can be solved with a simple correction that shifts the probability of the trailing space over to the current word.

Tokens are not words. If you hold on to this single line, the observation that models are bad at counting letters is not so surprising. This is because the unit models see is a piece that is a bundle of letters, and they are mostly blind to the letters contained inside a single piece.

What Gets Better When You Make It Bigger? — A Prescription Revised Two Years Later

The scaling laws established in "Neural Networks and Parameters" trace the trend of how loss drops when you increase a model's size, the amount of training data, and the computation spent on training. A 2020 preprint reported that the loss decreases in a power-law shape across all three factors and that some trends continue over a range spanning more than seven orders of magnitude, arguing that this relationship lets you decide how to divide a fixed computation budget (arXiv:2001.08361, preprint). It attached a formula that calculates in advance just how much better things will get to the intuitive idea that "making it bigger makes it better."

That prescription was revised two years later. A 2022 study concluded that the large language models of the time were severely under-trained relative to their data, tracing the cause to a trend of growing only the model while leaving the amount of data the same. After training more than 400 models—ranging from 70 million to over 16 billion parameters—on 5 billion to 500 billion tokens, the most efficient use of a computation budget required increasing the model size and the number of training tokens at the same rate. True to that prediction, a model that used the same computation budget as a 280-billion-parameter model but cut parameters to 70 billion and quadrupled the data outperformed the 280-billion, 175-billion, and 530-billion-parameter models across multiple evaluation tasks (arXiv:2203.15556, preprint). What to note here is the revised method. It found the optimal point anew using more than 400 models, and verified that optimal point with an actual model.

Scaling laws are observations rather than laws, and the prescriptions drawn from those observations can be revised. What the 2022 study revised was not the trend that loss decreases, but the ratio for dividing a fixed computation budget between the model and the data. That is why when you read claims about scale, you start by looking at which resource was increased and what was held fixed while measuring. Depending on whether only parameters were increased, data was increased alongside them, or the computation budget was kept constant, the same phrasing takes on different meanings.

This is also a place to examine what "better" means. The loss measured by scaling law research is typically cross-entropy. Cross-entropy is a loss that grows larger the lower the probability the model assigns to the correct token. This value measures only the magnitude of the probability given to the correct answer, without checking whether the correct answer ranks ahead of other candidates. Yet in actual text generation—much like a method that always picks the top-ranked candidate—that ranking often matters. Research has therefore emerged that re-establishes scaling laws using the probability of the correct token falling within the top candidate ranks as the yardstick (arXiv:2510.20387, preprint). The authors view this law as complementing rather than replacing the picture seen through cross-entropy.

Even if loss decreases, the performance on tasks you actually assign might not follow along. Downstream tasks are individual tasks like translation that are actually assigned to a model after pretraining. When pretraining data and translation task data were sufficiently similar, increasing pretraining steadily improved translation scores as well, and even allowed for prior prediction. When the two were moderately misaligned, translation scores fluctuated or even worsened the more pretraining was continued, yet cross-entropy measured on the downstream task kept improving throughout that time (arXiv:2402.04177, preprint). The loss getting better and the things we want getting better are two different matters.

Why this shape appears has not been settled either. A common explanation is that a power-law structure already exists within the data. Yet in transformers trained to predict random walk paths on a graph, scaling laws appeared even though the data contained no such structure (arXiv:2601.10684, preprint). Scaling laws are empirical regularities that repeatedly turn up when measured under certain conditions with certain yardsticks, and there is as yet no agreed-upon explanation for why they hold true.

What Comes After Pre-training — Alignment That Imposes Human Preferences

A 2022 preprint begins with the sentence that making language models large does not automatically make them follow user intent better. Even large models can output text that is untrue, harmful, or simply unhelpful, and in the authors' words, these models are not aligned with the user (arXiv:2203.02155, preprint). Alignment is the work of fitting what a model outputs to user intent and human values.

What next-token prediction aims for is the words likely to follow in the text it learned from, and that text mixes unhelpful words with untrue words. A model that has only finished pre-training follows such continuations just as they are. The two refinement steps followed in "How Generative AI Answers" — fine-tuning to make the model follow human-written exemplary answers, and Reinforcement Learning from Human Feedback (RLHF), which corrects it using rankings graded by humans — are procedures meant to bridge this gap. We saw there that the answers of a 1.3-billion-parameter model refined this way were preferred, in human evaluations of the requests the study collected, over the answers of GPT-3, which had 175 billion parameters and was 100 times larger, though simple mistakes still remained (arXiv:2203.02155). The meaning this result has in this article is that scale may not be the only axis. What is done after pre-training can greatly change the model users experience.

The first step is often called instruction tuning. Instruction tuning is fine-tuning done with examples pairing instructions with matching answers. How instructions should be mixed in what proportions is not yet well understood. A study that divided instructions into three types — natural language processing tasks, coding, and general conversation — and changed the mixing proportions reports that certain types can be advantageous for one use while harmful to another (arXiv:2312.10793, preprint). The second step, Reinforcement Learning from Human Feedback, has older roots. As a variant of reinforcement learning that learns from human feedback instead of human-designed scoring rules, it stands on the preceding field of reinforcement learning using human preferences as signals, and a significant portion of its foundational techniques comes from the fields of machine control and robotics (arXiv:2312.14925, preprint).

This procedure contains an assumption. Most approaches treat humans as entities who are rational, if somewhat noisy, and in particular provide unbiased feedback. There is the objection that this human model is overly simplistic, and that models of humans should be different for each person, vary depending on the context, and change over time (arXiv:2206.13316, preprint). The question of whose preferences also remains. Epistemological and ethical arguments have emerged stating that Reinforcement Learning from Human Feedback must incorporate multiple perspectives together (arXiv:2407.17482, preprint). Alignment is a technical procedure while at the same time holding within it the question of whose values are to be put inside.

There is also the observation that alignment takes something away. Mode collapse is a phenomenon where a model's outputs narrow down to a few patterns and lose diversity. A study analyzing 4,374 stories drawn from three language models reports that successive editions of one model family showed increasingly severe mode collapse (arXiv:2402.04477, preprint). If a model is tuned excessively during the alignment process, the ability to mimic the voices of many different writers is restricted, and it becomes unable to take on multiple perspectives. Although it is an observation about a specific point in a specific model family, it is enough to shake the conventional wisdom that alignment only adds capabilities.

When reading a paper, first check whether that paper is measuring a model that has only finished pre-training, or a model that has gone through instruction tuning and alignment. The two are different items, and the results of benchmarks — which are collections of test questions created to make multiple AI models solve the same problems and compare their scores — can also change according to this distinction.

Learning Without Retraining, Using Only Examples in the Chat Window — In-Context Learning

A preprint introducing GPT-3 in 2020 reported a strange phenomenon. As they scaled up the language model, performance using just a few examples improved significantly, sometimes rivaling the best methods of the time that were fine-tuned for specific tasks. Not a single parameter was fixed; they simply wrote out a task description and a few demonstration examples in text (DOI: 10.48550/arxiv.2005.14165, preprint). The same paper noted that there are datasets where this approach still struggles, as well as datasets suffering from methodological issues stemming from the mass-scale learning of internet text.

In-context learning is a phenomenon where a model solves new tasks using only the examples in the text placed before it, without modifying its parameters. Although the word "learning" is in the name, it does not involve turning dials to reduce loss like the learning followed in "Neural Networks and Parameters — What It Means to Nudge Hundreds of Billions of Dials". Not a single parameter changes. What changes is only the preceding text that the model is currently looking at. A prompt is the text placed in front of a model to get a response. This is the root of the fact that how you write a prompt changes the result.

There are multiple explanations for why this happens. One theory focuses on the fact that natural language data contains structures that combine small operations. Under linguistic assumptions, if pre-training data has enough of that structure, they derived an information-theoretic limit showing that in-context learning capabilities emerge from ordinary next-token prediction (arXiv:2303.07971, preprint). There is also the observation that this capability can appear and then disappear. In-context learning appears when the training data has specific distributional properties, but can diminish with further training, and one study used a simplified model to theoretically pinpoint the conditions under which this emergence and disappearance occur (arXiv:2410.23042, preprint).

The most widely known technique derived from in-context learning is chain-of-thought prompting. Chain-of-thought prompting is a method where examples detailing intermediate reasoning steps leading to the answer are included in the prompt, making the model answer by writing out intermediate steps as well. They reported that this significantly improves performance on complex reasoning, and the gains could be dramatic. Giving just eight such examples to a 540,000,000,000-parameter model reached the state-of-the-art accuracy at the time on an elementary school math word problem benchmark, even outperforming a GPT-3 fine-tuned with a verifier attached (arXiv:2201.11903, preprint). The condition under which that paper observed this capability was a sufficiently large model.

Models do not uniformly use the text put into the chat window. Even models specifically trained to handle long inputs fail to properly capture information placed right in the middle of the input. One study connected this to the U-shaped positional bias of attention. Regardless of relevance, tokens placed at the beginning and end of the input receive more attention. The same study reported up to 15 percentage points higher performance than existing methods on various tasks of answering with retrieved text by using a technique to correct this bias (arXiv:2406.16008, preprint). This means it is a property stemming from the architecture, but one that can be mitigated.

Examples placed at the beginning can also interfere with what follows. In transformers trained on synthetic regression tasks, the more misleading examples there were at the front of the prompt, the more the prediction error clearly increased for the tasks that followed. Adding more correct examples reduced the error, but the magnitude of the reduction shrank the more they were added. Researchers called this the stickiness of context, and reported that the order in which they were trained heavily dictated resilience (arXiv:2604.23371, preprint). Under those experimental conditions, not only what you put into the prompt, but also what had already been placed there could cloud the answers that followed.

Peeking at What a Model Learned — The Limits of Probing

"Neural Networks and Parameters — What It Means to Nudge Hundreds of Billions of Dials" noted that humans cannot read what parameters are doing together, and tools to look inside are not yet finished. In language models, one of those tools is probing. Probing is a method where you detach the numbers that a model creates in its inner layers while processing an input—that is, representations—and try to guess what information is contained within them using a small model.

There is research showing that a model's outward behavior and its inner representations can diverge. When researchers investigated language models fine-tuned with random correct answers to "forget" a specific task, the degree of forgetting varied extremely by task. On some tasks, they gave nonsense answers even to entirely new problems, but on tasks like physical commonsense reasoning or scientific QA, the effect was limited only to the examples they were made to forget, and very similar new problems were still solved accurately. And even the broadly generalized forgetting was shallow. Even after forgetting, a linear probe learned on top of that model's representation—a probe made of the simplest straight-line-form calculations—still performed that task. The authors write that these results show it is difficult and unpredictable to selectively erase specific skills through fine-tuning (arXiv:2409.02228, preprint).

Behavior and representation can be misaligned. Just because a model behaves as though it cannot do something does not mean that information is not inside. Conversely, just because a probe reads something out does not immediately mean the model is actually using it. The in-context learning theory research seen earlier probed Transformers trained in a controlled, small experimental environment, showing that in-context learning is supported by representations of the compositional structure of inputs (arXiv:2303.07971). Results like these are also read as what the probe read out in that environment.

A stronger method is to change the training data directly. Research addressing how language models learn exceptions to English passives narrowed down the answer by removing, changing, and inserting sentences of the type pointed to by the hypothesis in the training corpus. The authors left a methodological lesson that completely controlling the inputs received by a learner is crucial for such questions (arXiv:2407.04593, preprint). This method can only be used if training can be run again from the beginning. This means it cannot be applied to models whose training corpus and training procedures are not public.

Can Emergence Be Judged?

"Neural Networks and Parameters — What It Means to Nudge Hundreds of Billions of Dials" defined emergent abilities as abilities absent in small models but present in large ones, and placed two claims side by side. They are a 2022 claim that such abilities cannot be predicted by extending the performance trends of smaller models (arXiv:2206.07682, preprint), and a 2023 counterargument that what looks like emergence can arise from the grading yardsticks chosen by researchers (arXiv:2304.15004, preprint).

Further explanations emerged afterwards. A theory published in the same year, 2023, concluded that the abilities called emergent are the combined result of in-context learning, model memory, and language knowledge, rather than true emergence. This theory was presented alongside more than 1,000 experiments (arXiv:2309.01809, preprint). A 2025 review that broadly examined this phenomenon revealed that even the ways of establishing concepts by vetting existing definitions disagree with one another. The conclusion is cautious. Emergent abilities are still poorly understood, and continue to generate misunderstandings regarding their definition, nature, predictability, and implications (arXiv:2503.05788, preprint).

To sum up the shape of the dispute: the counterargument that took issue with metrics stopped at suggesting emergence might not be a fundamental property of scaling up models, while the theory explaining it through combinations went as far as claiming that the purported emergence was not true emergence at all. The review organizes it by stating that we still do not know well. Either way, the judgment hinges on what was measured and how. The sentence "This ability emerged at this scale" is hard to accept at face value before verifying what metrics and what prompts were used to measure it.

What Is This Machine a Model Of?

The most fundamental spot is also empty. On one side stands the view that acknowledging language models have learned how to use language could open a breakthrough in language science. It adds that this price might require discarding some old ideas used to evaluate linguistic knowledge (arXiv:2512.12447, preprint).

On the other side is the argument that transformers are not a model of human ability, but a model of the corpus we trained them on. Cognitive science holds that human language capacity relies on computational formats stronger than linear ones, whereas the transformer architecture supports at most linear formats. The author adds that this is not a very pessimistic story. Language is a means to express inner thoughts, and given the right context, it is also a kind of "discourse machine" that generates new words, meaning that just as we learned to use this technology in one way, language models have learned to use it in a very different way (arXiv:2508.18598, preprint).

Both perspectives look at the same data and arrive at different conclusions. This article cannot decide which side is right. However, whichever side you choose, that choice is the result of an argument, and observations have not yet provided a settled answer.

Four Things to Check When Opening a Paper

• Look at which model was measured. The results vary depending on whether it is a model that has only finished pre-training or one that has gone through instruction tuning and alignment.
• Look at what was measured. Check whether it is loss or downstream task performance, and whether that metric changes continuously or drops off abruptly.
• Look at the scale axis. Check whether only parameters were increased, whether data was also increased, or whether the compute budget was kept constant.
• Look at whether the prompts were controlled. Check whether in-context learning and chain-of-thought became mixed into the results.

Where to Look Next

The history of deep learning up to the transformer is in "From Backpropagation to AlphaFold: What Deep Learning Stood On". Why plausible-sounding yet wrong answers come out continues in "Why Do LLMs 'Lie'?". Whether this machine can be said to understand or think is covered by "Does AI Really 'Think'? — A Question Researchers Are Still Fighting Over". How effective in-context learning and chain of thought are in actual use can be seen in "Does Prompting Actually Work? — The Science of Asking AI Well", and the problem of metrics changing conclusions in "The Score Went Up. But What Went Up?".

Source Papers

• "Distributed Representations of Words and Phrases and their Compositionality" (2013, preprint) — DOI: 10.48550/arxiv.1310.4546 — Distributed vector representations of words, subsampling and negative sampling, and its inherent limitations of being "indifferent to word order and incapable of expressing idioms."
• "Attention Is All You Need" (2017) — DOI: 10.65215/2q58a426 — A structure completely omitting recurrence and convolution in favor of attention alone, enabling parallelization and reduced training time.
• "Language Models are Few-Shot Learners" (2020, preprint) — DOI: 10.48550/arxiv.2005.14165 — Specified few-shot performance through text interaction alone without gradient updates or fine-tuning, along with self-critiques regarding methodological issues in dataset and web corpus training that still struggle.
• "Scaling Laws for Neural Language Models" (2020, preprint) — arXiv:2001.08361 — Loss scales as a power law with respect to model size, data size, and compute; the effects of width and depth are minimal, and fixed budgets should be optimally allocated.
• "Training Compute-Optimal Large Language Models" (2022, preprint) — arXiv:2203.15556 — The under-training of large models at the time, and a revision stating that model size and token count should be increased equally.
• "Training language models to follow instructions with human feedback" (2022, preprint) — arXiv:2203.02155 — Formalization of the alignment problem, instruction fine-tuning plus RLHF, where outputs from a model 100 times smaller were preferred, yet simple errors persist.
• "Emergent Abilities of Large Language Models" (2022, preprint) — arXiv:2206.07682 — The definition of emergence and claims of unpredictability.
• "Are Emergent Abilities of Large Language Models a Mirage?" (2023, preprint) — arXiv:2304.15004 — An alternative explanation that emergence may be an artifact of metric selection, backed by a three-pronged verification.
• "Are Emergent Abilities in Large Language Models just In-Context Learning?" (2023, preprint) — arXiv:2309.01809 — Explained as a combination of in-context learning, memory, and linguistic knowledge, tested over 1,000 experiments.
• "Emergent Abilities in Large Language Models: A Survey" (2025, preprint) — arXiv:2503.05788 — Discrepancies in definitions, conditional evaluations, and the conclusion that the phenomenon is still poorly understood.
• "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (2022, preprint) — arXiv:2201.11903 — Prompting that provides demonstrations of intermediate reasoning steps as examples, improving arithmetic, commonsense, and symbolic reasoning.
• "A Theory of Emergent In-Context Learning as Implicit Structure Induction" (2023, preprint) — arXiv:2303.07971 — Compositional structure recombination, an information-theoretic upper bound showing in-context learning emerges from general next-token prediction, and compositional structure representations confirmed via probing.
• "Toward Understanding In-context vs. In-weight Learning" (2024, preprint) — arXiv:2410.23042 — Distributional properties that cause the emergence and disappearance of in-context learning, and theoretical analysis of gating models.
• "When Context Sticks: Studying Interference in In-Context Learning" (2026, preprint) — arXiv:2604.23371 — The "stickiness" of context, where earlier examples hinder adaptation to subsequent tasks.
• "Tokenization Falling Short: On Subword Robustness in Large Language Models" (2024, preprint) — arXiv:2406.11687 — The "curse of tokenization," typo and formatting variation biases, and vulnerabilities that persist despite scale.
• "Language Model Tokenizers Introduce Unfairness Between Languages" (2023, preprint) — arXiv:2305.15425 — Up to 15-fold token length gaps across languages, creating inequalities in cost, latency, and context capacity.
• "Evaluating Morphological Alignment of Tokenizers in 70 Languages" (2025, preprint) — arXiv:2507.06378 — Morphological alignment does not substantially explain performance variance.
• "Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilities" (2024, preprint) — arXiv:2406.10851 — Leading whitespace can cause the sum of word probabilities to exceed 1.
• "Language models are better than humans at next-token prediction" (2022, preprint) — arXiv:2212.11281 — Humans consistently performed worse in both top-accuracy and perplexity experiments.
• "Self-Attention as Distributional Projection" (2025, preprint) — arXiv:2511.13780 — Interpreting self-attention as the sequence-context projection of corpus co-occurrence statistics.
• "Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization" (2024, preprint) — arXiv:2406.16008 — U-shaped positional attention bias and the omission of information in the middle, regardless of relevance.
• "Relative-Based Scaling Law for Neural Language Models" (2025, preprint) — arXiv:2510.20387 — A scaling law using relative order—which cross-entropy misses—as its metric.
• "Scaling Laws for Downstream Task Performance of Large Language Models" (2024, preprint) — arXiv:2402.04177 — Downstream cross-entropy improves even while downstream metrics worsen when distributions mismatch.
• "On the origin of neural scaling laws: from random graphs to natural language" (2026, preprint) — arXiv:2601.10684 — Scaling laws appear even when data lacks power-law structure.
• "A Survey of Reinforcement Learning from Human Feedback" (2023, preprint) — arXiv:2312.14925 — The foundations of RLHF, its roots in preference-based reinforcement learning, and control/robotics techniques.
• "Humans are not Boltzmann Distributions" (2022, preprint) — arXiv:2206.13316 — The simplicity of the assumption of bias-free rational feedback, and the need for personal, contextual, and dynamic human models.
• "Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?" (2024, preprint) — arXiv:2407.17482 — Epistemological and ethical arguments for pluralism in RLHF.
• "Demystifying Instruction Mixing for Fine-tuning Large Language Models" (2023, preprint) — arXiv:2312.10793 — Combining instruction types can produce contradictory effects across different applications.
• "Detecting Mode Collapse in Language Models via Narration" (2024, preprint) — arXiv:2402.04477 — An analysis of 4,374 papers showing that alignment overfitting constrains authorship generalization.
• "Unforgettable Generalization in Language Models" (2024, preprint) — arXiv:2409.02228 — Extreme task-specific variance in forgetting, alongside findings that linear probes still perform tasks even after forgetting.
• "Manipulating language models' training data to study syntactic constraint learning" (2024, preprint) — arXiv:2407.04593 — A methodology for arbitrating between hypotheses by manipulating training corpora.
• "Large language models have learned to use language" (2025, preprint) — arXiv:2512.12447 — The possibilities and costs that acknowledging the learning of language use opens up for linguistics.
• "What do language models model? Transformers, automata, and the format of thought" (2025, preprint) — arXiv:2508.18598 — The argument that transformers model a corpus rather than human capabilities, and language as a "discourse machine."