The Score Went Up. But What Went Up?

Time to Doubt the Ruler

Almost all claims about language models are boiled down to a single number. It is the number showing how many points they scored on a certain test. A language model is a model that takes text and produces the text that follows, and that test is a benchmark. As put in "How AI Gets Things Wrong" from AI from Scratch, a benchmark is a collection of test questions designed to make several AI models solve the same problems and compare their scores. When a new model comes out, a scoreboard comes out with it, and when this number goes up, we say its capability has gone up.

Even when talking about how often hallucinations occur, where reasoning breaks down, or what changes when you change the prompt—the instruction handed to the model—the evidence is ultimately a number from some test. So, in the end, we have to look at that test paper.

The sentence this article doubts is "If the score goes up, capability has gone up." Even if the score goes up, we may not know what has gone up. This is a problem inherent to the act of measurement itself. A test score is an estimate of capability, in other words an estimate, and multiple layers of assumptions lie between the estimate and the capability. This article looks at six places where those assumptions break down: contamination where test questions get mixed into training data, saturation where the test becomes too easy and differences disappear, test papers with wrong answers, construct validity asking whether the test measures the capability written on its name tag, graders, and reproducibility asking whether the same value comes out when measured again.

We will also clarify the evidence beforehand. Almost all the literature cited here is preprints released on arXiv. A preprint is a version released before undergoing peer review. One position paper on contamination (arXiv:2310.18018) is confirmed in the primer "How AI Gets Things Wrong" as the version published in conference proceedings (EMNLP Findings). Some of the rest may also have been published in peer-reviewed journals afterward, but this article did not check whether that is the case. The fact that an article doubting the ruler stands on literature whose review status was not checked is itself an example of what this article is trying to say. Evidence has grades, and evidence that does not disclose its grade is hiding its own grade. All figures below can be read as "values that the study observed and reported itself at that point in time under its own settings."

First, What the Benchmark Did

Before doubting the ruler, it is fair to state what that ruler did.

In the days when there was no shared test paper, everyone produced their own numbers with their own data, and those numbers could not be compared with one another. A 2013 preprint released a benchmark corpus of roughly 1 billion words to measure the progress of language models predicting the next word using statistics. The baseline, the starting point for comparison, was an old statistical model (5-gram) that predicted the next word by looking at just a few preceding words. Scores were measured by perplexity, a value indicating how confused the model is about the next word, where lower means better guesses. The baseline perplexity was 67.6, and combining various techniques reduced it by 35% (arXiv:1312.3005). It proposed a method of releasing a baseline and having everyone compete on top of it.

Benchmarks are the mechanism that has driven this field this far. This article is not saying that mechanism should be discarded. It looks at the weight that mechanism is currently bearing.

Contamination — When Test Questions Find Their Way Into Training Data

The first problem was established up to its definition and actual measurement in the "Students Who Saw the Test Questions in Advance" section of "How AI Gets Things Wrong". Benchmark contamination is the phenomenon where test questions, or text very similar to them, are mixed into a model's training data beforehand, and a contaminated model can receive a score higher than its actual ability on that test. Because of this, incorrect scientific conclusions may be published and correct ones discarded, but the scale of the problem is not easy to measure and remains unknown (arXiv:2310.18018, 2023 statement, published in EMNLP Findings). This article looks at the next question: whether contamination can be erased and detected.

There are loopholes starting from the removal methods. Contamination removal mostly relies on string matching, a method of checking whether text snippets identical to the test questions exist in the training data as-is. An n-gram overlap check, which looks at how many consecutively placed words overlap, is representative. Just by rephrasing the test questions in other words or translating them into another language, one can evade this kind of contamination removal. When stronger removal tools were run, 8 to 18% of a code generation benchmark overlapped across two large public corpora, and the same contamination appeared in synthetic datasets created by GPT-3.5 and GPT-4 (arXiv:2311.04850, 2023 preprint). The authors saw this as a risk where contamination can flow in even if no one inserts it intentionally. In a study that created paired questions with the same distribution for each item and tested whether models were significantly more confident on the original items, almost all public models and benchmarks tested were suspected of contamination to varying degrees (arXiv:2406.18326, 2024 preprint).

Are the Tools That Detect Contamination Reliable?

Contamination detection methods each stand on top of assumptions. A 2024 survey reviewed 50 contamination detection papers, divided those assumptions into eight categories, and tested three of those assumptions using membership inference. Membership inference is a method that tries to guess whether a single piece of text was included in a model's training data. The result is uncomfortable. Membership inference relying on these assumptions could perform similarly to random guessing on datasets used for pre-training. The authors interpreted this as the possibility that those language models are learning the distribution of the data rather than memorizing individual cases. They also pointed out that if there is a distribution shift between seen data and unseen data—meaning a difference in the distributions from which the two sets of data emerge—such methods fail easily (arXiv:2410.18966).

If this interpretation is correct, the weight is heavy. To divide contamination into "saw/did not see", a picture where the model remembers individual cases is required. If the model learns distributions, the dichotomy of "saw/did not see" itself is not well-defined.

Another study applied five detection methods to four models that were state-of-the-art at the time in 2024 and eight challenging datasets. The detection methods at that point had non-trivial limitations in both assumptions and practical application. Contamination introduced along with answer augmentation during instruction fine-tuning—the step of training a model one more time with examples pairing instructions and their corresponding answers—was particularly difficult to detect. And the consistency of judgments among state-of-the-art detection techniques was limited (arXiv:2409.09927, 2024 preprint). When different detectors give different answers, there is no referee to judge which side is correct.

Contamination is real, and one detection study suspected it to varying degrees in almost all public models and benchmarks tested, and it is not erased by string matching. However, as "How AI Gets Things Wrong" pointed out, many models including the most advanced ones showed few traces of contamination, and not all scores come from memorization. Even the detectors issuing those suspicions give conflicting judgments. When a detector says "clean", what that phrase even means has not yet been established. The claim that "this score is uncontaminated" is a claim that remains largely unverified within the scope these studies have shown.

Saturation — A Test Where Everyone Crowds Near the Ceiling

The second problem arises when a test becomes too easy. Saturation is a state where the scores of models crowd near the ceiling, making it difficult to tell the difference between models with that benchmark. A 2026 study that analyzed 60 language model benchmarks across 14 properties related to saturation found that roughly half showed saturation, with older benchmarks showing a higher proportion (arXiv:2602.16763). What influenced the power to withstand saturation was expert curation, where experts selected and refined questions, rather than whether the test data was made public.

Once they hit the ceiling, the remaining differences between models shrink. To figure out whether a shrunk difference is a real difference in ability or random fluctuation, you need to know the size of the fluctuation. Measuring that size was rare. A 2024 study measured the variance of benchmarks—in other words, the margin by which scores fluctuate when measured again with slightly changed conditions. These include fluctuations depending on the random starting values called seeds when learning begins, and whether scores steadily rise during learning. Simple changes, such as turning multiple-choice questions into fill-in-the-blank formats, often reduced variance in smaller-scale models. On the other hand, sophisticated methods such as item analysis or item response theory (which we will revisit in the alternatives later) drawn from literature dealing with human testing struggled to meaningfully reduce variance (arXiv:2406.10229).

There are also holes in statistical habits. An error bar is a mark drawn next to a score indicating the range where the actual value is likely to lie. Error bars and significance tests in language model evaluations usually lean on the Central Limit Theorem—the theorem that if there are enough questions, the fluctuation of average scores follows a bell-shaped distribution. A 2025 position paper pointed out that this method is appropriate for benchmarks with thousands of examples, but generally severely underestimates uncertainty in small, specialized benchmarks, and recommended frequentist and Bayesian methods to use instead (arXiv:2503.01747). This means error bars are drawn narrower than they actually are. The authors note that such small benchmark situations are becoming increasingly common.

Saturation can also obscure what you were trying to measure. In actual use, the difference between 99.9% and 99.999% changes the digit place of the number of failures. To estimate the probability of such rare failures with a narrow confidence interval—a narrow range where the true value is likely to lie—you have to run models more than the standard method of evenly sampling and running inputs (Monte Carlo) can handle. A 2026 study used the property that failures concentrate on specific inputs to sample toward inputs where failures are frequent, greatly reducing the required number of runs. As a result, even models whose accuracies are indistinguishable on standard benchmarks could have considerably different estimated failure rates (arXiv:2605.11209). The same score does not mean the same thing.

A common response is to make harder tests. An expanded version of a widely used multi-subject knowledge benchmark added reasoning-centric questions, increased the number of choices from 4 to 10, and removed noisy questions. Accuracy dropped by 16% to 33% compared to the original, and the margin by which scores fluctuated according to 24 prompt formats decreased from 4% to 5% to 2% (arXiv:2406.01574).

Yet this response itself goes in circles. A 2025 paper called this cycle Ouroboros, the snake biting its own tail. When a model's capabilities rise, coupled in many cases with the possibility that the dataset went into training, resulting in saturation, a harder replacement becomes necessary once more. The authors' question is the same as the question of this article. Does surpassing a benchmark really show reasoning ability, or are we chasing numbers divorced from the ability we claimed to measure (arXiv:2511.01365)?

Test Papers with Wrong Answers

Let us assume a test that is neither contaminated nor saturated. Even then, items where the answer itself is wrong can remain.

A 2024 study had humans re-evaluate and score 5,700 questions across all 57 subjects in a widely used multi-subject knowledge benchmark. Based on this, it estimated that 6.49% of all items in this benchmark contain errors, and in the analyzed virology questions, 57% had errors. When graded with the re-evaluated items, the performance metrics significantly deviated from what was originally reported (arXiv:2406.04127).

Nor is this a problem restricted to language models. A 2021 study found errors in answer labels (labels) within the test sets—the items set aside exclusively for measuring capability—of 10 of the most widely used datasets in computer vision, natural language, and audio. Algorithms picked candidates likely to be wrong, and humans verified them through crowdsourcing; an average of 51% of the flagged candidates were actually incorrect. The errors averaged at least 3.3% across the 10 datasets, and reached at least 6% in the validation set of one representative image dataset.

The most critical observation is that errors overturn rankings. Which model to use is conventionally decided by looking at test accuracy, but after correcting labels and remeasuring, the rankings could change. If the proportion of test cases with originally incorrect labels increased by just 6% (as denoted in the paper), a shallower residual network outperformed a deeper residual network. A residual network is a type of neural network with deeply stacked layers. In real-world data with high label noise, models with smaller capacity may be more practically useful (arXiv:2103.14749). Errors in the test paper do more than just shave off a few points; they can change the order entirely.

Sometimes, even when the answer is correct, what was meant to be measured is not measured. Annotation artifacts are traces left on items during the process of humans creating data, serving as clues that allow models to find the correct answer without actually solving the task. A 2023 preprint examined this phenomenon—where large pretrained models achieve high scores while relying on such traces left in common across training, validation, and test data to find answers—in a natural language inference task that matches logical relationships between two sentences. The authors exposed the limits of performance using deliberately twisted examples (contrast sets and adversarial examples), demonstrated a bias arising from how annotators created the training data, and proposed a data augmentation technique to correct it (arXiv:2302.04700).

A model that got the answer right by relying on artifacts did not get the item wrong. It simply did not do the thing we intended to measure.

Construct Validity — Does the Test Measure the Ability on the Name Tag?

This is the core of this article. The first three problems ask, "Was this test administered properly?" The fourth problem asks whether this test measures the specific ability written on its name tag.

Construct validity is the degree to which a test actually measures the concept it claims to measure, known as the construct. This is a term from psychometrics, the field that deals with how to measure human abilities and traits. Because we cannot measure things like intelligence or anxiety directly with a ruler, we create questions, collect answers, and assume those answers represent the concept we want to measure. Checking whether that assumption holds up is construct validity. "Reasoning ability," "safety," and "language understanding" are precisely such concepts, and benchmark names include words like these without any reservations.

A 2025 preprint applied this framework across machine learning benchmarks generally. The authors' thesis is this: Benchmark scores at best measure performance relative to a specific evaluation dataset and concrete training tasks. To draw scientific conclusions about tasks in general, such as image classification, from here requires additional assumptions about the theoretical structure of the training tasks, the evaluation function, and the data distribution. The authors borrowed psychometric theory to make those assumptions explicit as conditions for construct validity (arXiv:2510.23191).

How well those conditions are actually met was examined by a 2025 systematic review. When 29 expert reviewers examined 445 language model benchmarks from major natural language processing and machine learning conferences, patterns emerged in what they aimed to measure, what tasks they used, and how they graded that undermined the validity of the resulting claims. The authors put forward eight core recommendations and execution guidelines (arXiv:2511.04703).

Causal inference benchmarks are a concrete example. According to a 2024 review, among benchmarks intended to evaluate a language model's ability to reason about cause and effect, many are likely solved simply by retrieving and pulling out knowledge from that field. The authors highlighted the shift around that time in benchmarks toward asking about intervention (what happens if we change something) and counterfactuals (what would it have been like if things had been different then), and laid out criteria for useful benchmarks (arXiv:2407.08029). This means a number named "causal inference score" may actually be a knowledge retrieval score.

The medical side is the same. Medical language model research cites benchmark performance as evidence for bold claims that a model contains clinical knowledge or reasons like a doctor. A 2025 position paper pointed out that those benchmarks are mostly arbitrarily composed of medical licensing exam questions, and reported a significant gap in construct validity by evaluating at a proof-of-concept level with actual clinical data (arXiv:2503.10694). That doing well on a licensing exam and doing well at patient care are not the same thing is a fact we already know about humans, too.

One longstanding concern is also attached here. There is a concern that if many people repeatedly choose and tweak models while looking at the scores of the same test questions set aside to measure true skill (a holdout set), the results can become tailored to those questions. Exactly how large that magnitude is was not measured by the studies cited in this article. If this concern is correct, the moment a test becomes the target, the test gradually stops being a test.

The Grader — What Gets Mixed Into Scores When a Language Model Grades

To evaluate texts where there is no single right answer, such as whether a summary is good or an answer is helpful, a grader is needed. Human graders are expensive and slow, so grading is handed over to another language model, and this model is called a model judge. A 2026 meta-analysis extracting information from 14,171 papers across four major conferences and analyzing 3,334 natural language generation papers found that in 2025, model judges appeared in more papers than human evaluations. Because model judges have grown rapidly, verification matching them against human judgments has not kept pace, with fewer than 8% of such papers having performed that kind of verification. Although model judges show a correlation with overall quality, they diverge on detailed criteria such as fluency (arXiv:2601.07648).

When values assigned by a model judge become benchmark scores, the judge's habits also get mixed into the scores, and depending on which benchmark is used for verification, the judge's ranking also changes. In a 2026 study comparing 21 judge models across three benchmarks, their rankings shifted by up to 14 places. The same study points out that the simple agreement rate commonly used when verifying judges fails to subtract the proportion that match by chance, systematically inflating the judge's discriminatory power. The "Consistency-Bias Paradox," where a judge that consistently makes the same judgments (retest reliability exceeding 0.95) heavily displays positional bias where judgments are pulled depending on where the answer is placed, was also observed in two actually deployed judges (arXiv:2606.19544). Consistently giving the same answer and measuring correctly are different things.

Human evaluations, which serve as the reference point, are also not solid. When 10 papers reporting human evaluations were randomly selected from among 2020 papers at a major conference, only one paper had problem definitions, methods, and evaluations fully aligned (arXiv:2104.05361, in that sample of 10 papers). In a 2023 survey, only 13% of papers were viable replication candidates, and the authors called the conclusion that the majority of human evaluations are difficult to repeat and reproduce or have major flaws a "grim picture," while also writing that it is an opportunity to redesign them (arXiv:2305.01633). Because there are tasks where model judges are closer to human judgments than existing automated metrics are (arXiv:2502.06193, 2025 software engineering tasks), how much to trust them must be determined separately for each task. The judge's biases and the fluctuations in repeated judgments are handed over to "The Grader Is Also a Language Model — How Far Can Its Verdict Be Trusted?".

Reproducibility — Retaking the Same Test

The final issue is the most fundamental. It is whether testing the same model again with the same benchmark yields the same score.

The expanded benchmark seen in Saturation reduced score fluctuations across 24 different prompt formats from 4–5% down to 2% (arXiv:2406.01574). Read the other way around, before the improvement, simply changing the prompt format moved the score by that much. Variance depending on the seed and the phenomenon where scores do not consistently rise during training have also been measured (arXiv:2406.10229). Reproducibility when a model of the same name changes over time is left to "Not the Model I Signed Off On".

Still Measuring — Alternatives Inside the Literature

Passing through six problems, cynicism seems like the natural response. Yet what this literature actually does is far from cynical. The papers cited above mostly do not stop at pointing out problems, but propose the next rulers instead. There are five branches.

The first branch is dynamic evaluation. Dynamic benchmarks are benchmarks that keep newly creating questions instead of using a fixed test sheet. In a 2021 platform, question creators attempt to make examples that the target model gets wrong, but other people do not get wrong. Pointing out that models at the time in 2021 excelled at benchmark tasks yet failed on simple challenge examples and wavered in real-world situations, the authors proposed a structure where dataset creation, model development, and model evaluation directly inform one another (arXiv:2104.14337). A 2025 survey reviewing this trend points out the intrinsic limitations of methods supplementing fixed benchmarks, and singles out the lack of a standardized criterion for evaluating dynamic benchmarks as a decisive blank (arXiv:2502.17521). A new ruler has been made, but there is no ruler yet to test that ruler.

The second branch is design that withstands contamination. A 2024 benchmark brought questions from a broader field against unintended leakage and established three de-contamination rules. For intentional leakage, it split into a validation set and a test set with similar difficulty and subject distributions, making only the validation set public while keeping the test set private (arXiv:2412.15194). Yet the saturation study above observed that what influenced the power to endure saturation was expert curation, not whether the test data was made public or not (arXiv:2602.16763). One looks at contamination, the other at saturation, so the two results do not directly collide. However, the conclusion that "keeping things private is the answer" is still premature.

The third branch is the path returning to psychometrics. Item Response Theory is a method that sees every question as having different difficulty and discrimination (the power to sort out higher and lower ability), estimating hidden ability from response patterns. A 2025 framework used this theory to build adaptive tests that choose the next question based on the result of the previous question, reporting that it reduced the required number of questions by up to 90%. Even among targets with the same accuracy, ability estimates diverged, moving 23 to 31% of over 3,000 evaluation targets by more than 10 spots in the rankings (arXiv:2511.04689). However, the variance study above reported that methods like Item Response Theory struggled to meaningfully reduce variance (arXiv:2406.10229). Whether a tool useful for sparing questions also reduces fluctuation must be examined separately.

The fourth branch is sampling targeting rare failures. This is the approach that to measure reliability in a saturated benchmark, one should look at failure rates instead of accuracy, concentrating samples on inputs where failures cluster (arXiv:2605.11209). Statistical methods to use instead of the central limit theorem on small benchmarks have also been proposed (arXiv:2503.01747).

The fifth branch is attaching an expiration date to scores. According to a 2026 position paper, when evaluation results from automated metrics, model judges, human evaluation, and benchmarks are combined into an average, confidence in the evaluation can greatly exceed the reliability of the weakest signal. The authors call this "confidence inflation," and propose treating evaluation scores as claims possessing three properties. Formality (that human evaluation is stronger evidence than automated metrics), scope (that benchmark results apply only to the tested distribution), and validity period (that as contamination accumulates and the distribution shifts, benchmark results expire). It is the argument that evaluation results must carry tags: a formality grade, a scope declaration, and an expiration date. Across 54 models and ten scenarios on a public leaderboard, the top five selected by average score and the top five selected by the weakest signal did not overlap by even a single one (arXiv:2607.26191).

What Remains

There is one distinction that holds this article together. "Measuring things is difficult" and "Things cannot be measured" are different.

The former is what this entire body of literature is doing, and the latter is what no one is doing. The team that reviewed 445 benchmarks concluded with eight recommendations (arXiv:2511.04703). The team that discovered detection assumptions can perform at the level of random guessing read that as a possibility that the model is learning the distribution rather than memorizing individual cases (arXiv:2410.18966). The team that cross-examined five detection methods wrote that more robust and generalizable contamination assessment research is urgently needed (arXiv:2409.09927). The team that called the irreproducibility of human evaluation a "grim picture" wrote in the same passage that it is an opportunity to redesign (arXiv:2305.01633). Those who doubt the ruler are different from those who want to throw the ruler away. Rather, they are the ones building rulers most diligently.

There are three score-reading habits we can draw from this literature.

First, read scores as claims, not facts. A score without details on which distribution it was measured from, when, and who graded it is a claim that hides its scope and expiration date (arXiv:2607.26191).

Second, compare differences against fluctuation. Whether the difference between two numbers is meaningful cannot be judged without knowing the variance of that benchmark (arXiv:2406.10229). On small benchmarks, standard error bars are likely to underestimate uncertainty (arXiv:2503.01747).

Third, separate the name from what is being measured. The fact that the word "reasoning" is in a benchmark's name is no evidence that it measures reasoning. That connection must be argued separately, and the name for that argument is construct validity (arXiv:2510.23191).

There are also things that must be left as unresolved.

There is no oracle for contamination detection, meaning a baseline that tells you what the correct answer is. We usually do not know the ground truth of what a model has learned. This is partly because training corpora are not entirely public, and partly because if the interpretation that models are learning distributions is correct, the dichotomy of "seen/unseen" is not well-defined itself (arXiv:2410.18966). The problem of validating detectors without knowing the ground truth remains open (arXiv:2409.09927).

The benchmark of human evaluation also wavers. To measure the bias of model judges, human judgments must be used as the standard, but the alignment and reproducibility of human evaluations themselves are not established (arXiv:2104.05361, arXiv:2305.01633). How to break this loop has not been solved.

New rulers need rulers. There is no standard criterion to evaluate dynamic benchmarks (arXiv:2502.17521), and psychometric tools do not necessarily reduce the variance problem (arXiv:2406.10229).

Finally, we apply the same yardstick to this article itself. Except for one position paper confirmed to be published in a conference, the literature cited here consists of preprints published on arXiv, and this article has not checked which of the remaining ones passed peer review. The figures within them are values observed and self-reported by each study in its own place, not values that verified one another. The demand of this article—that scope, timing, and grade of evidence must be disclosed—applies equally to this article.

This does not mean we should not trust numbers. It means we must see what the numbers are asking us to trust. Next to a score, an invisible sentence is always attached: "When measured with this test paper, by this grader, at this time." The moment you read that sentence aloud, the same numbers look different in size.

Source Papers

• "One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling" (2013, preprint) — arXiv:1312.3005 — Proposes a shared corpus for measuring progress, showing perplexity reduction via 5-gram baselines and combination techniques.
• "Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks" (2021, preprint) — arXiv:2103.14749 — Finds an average of at least 3.3% label errors across test sets in 10 datasets, with model rankings reversing upon label correction.
• "Dynabench: Rethinking Benchmarking in NLP" (2021, preprint) — arXiv:2104.14337 — A dynamic dataset creation platform placing humans and models in a loop.
• "The Great Misalignment Problem in Human Evaluation of NLP Methods" (2021, preprint) — arXiv:2104.05361 — Examines the misalignment among problem definitions, methods, and human evaluation, with only 1 out of 10 sampled papers showing complete alignment.
• "Augmenting NLP data to counter Annotation Artifacts for NLI Tasks" (2023, preprint) — arXiv:2302.04700 — Explores how models rely on dataset artifacts instead of solving tasks, and augmentation-based correction.
• "NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark" (2023, position paper published in EMNLP Findings) — arXiv:2310.18018 — Defines contamination levels, highlights that the scale of the problem remains unknown, and stresses the need for community-wide measurement.
• "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples" (2023, preprint) — arXiv:2311.04850 — Explores the bypassability of n-gram-based contamination removal, benchmark overlap in large public corpora, and contamination in synthetic data.
• "Missing Information, Unresponsive Authors, Experimental Flaws" (2023, preprint) — arXiv:2305.01633 — Finds that only 13% of existing human evaluations are reproducible candidates, alongside flaws in most selected experiments.
• "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" (2024, preprint) — arXiv:2406.01574 — Addresses saturation diagnosis, choice expansion, noise question removal, and reduced prompt sensitivity.
• "Are We Done with MMLU?" (2024, preprint) — arXiv:2406.04127 — Re-annotates 5,700 questions, estimating an overall error rate of 6.49%, 57% in one subset, and inconsistencies with originally reported performance.
• "Quantifying Variance in Evaluation Benchmarks" (2024, preprint) — arXiv:2406.10229 — Measures seed variance and monotonicity, the effects of task format changes, and the limitations of item response theory.
• "PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection" (2024, preprint) — arXiv:2406.18326 — Uses identically distributed counterparts and confidence comparison tests, suspecting contamination in nearly all tested models and benchmarks.
• "A Critical Review of Causal Reasoning Benchmarks for Large Language Models" (2024, preprint) — arXiv:2407.08029 — Shows that many causal reasoning benchmarks can potentially be solved via domain knowledge retrieval.
• "Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges" (2024, preprint) — arXiv:2409.09927 — Cross-examines five detection methods, highlighting the difficulty of detecting instruction-finetuning contamination and limited consistency across methods.
• "Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptions" (2024, preprint) — arXiv:2410.18966 — Reviews 50 papers across eight category assumptions, showing membership inference can perform on par with random guessing.
• "MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark" (2024, preprint) — arXiv:2412.15194 — Separately addresses intentional and unintentional leakage through a design with public validation sets and private test sets.
• "Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering" (2025, preprint) — arXiv:2502.06193 — Identifies tasks where model judges show higher correlation with human scores than traditional automated metrics.
• "Recent Advances in Large Language Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation" (2025, preprint) — arXiv:2502.17521 — Discusses the inherent limits of static reinforcement methods and the absence of dynamic benchmark evaluation criteria.
• "Position: Don't Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints" (2025, preprint) — arXiv:2503.01747 — Highlights the underestimation of central-limit-theorem-based error bars in small-scale benchmarks and proposes alternatives.
• "Medical Large Language Model Benchmarks Should Prioritize Construct Validity" (2025, preprint) — arXiv:2503.10694 — Verifies construct validity gaps in licensing-exam-based medical benchmarks using real clinical data.
• "The Benchmarking Epistemology: Construct Validity for Evaluating Machine Learning Models" (2025, preprint) — arXiv:2510.23191 — Explores how benchmark scores are relative dataset measurements, formalizes construct validity conditions, and presents three case studies.
• "The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation" (2025, preprint) — arXiv:2511.01365 — Examines the cycle of saturation and replacement, raising the question of whether passing a benchmark truly equates to reasoning capability.
• "Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks" (2025, preprint) — arXiv:2511.04689 — Proposes item-response-theory-based adaptive testing to drastically reduce question counts while differentiating ability among targets with identical accuracy.
• "Measuring what Matters: Construct Validity in Large Language Model Benchmarks" (2025, preprint) — arXiv:2511.04703 — Features 29 reviewers systematically reviewing 445 benchmarks, identifying patterns that undermine validity and offering eight recommendations.
• "What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025" (2026, preprint) — arXiv:2601.07648 — Analyzes metric inertia, metric-reference correspondence issues, and verification gaps (less than 8% human verification).
• "When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation" (2026, preprint) — arXiv:2602.16763 — Analyzes 60 benchmarks, finding about half are saturated, with expert curation contributing to resilience.
• "Measuring Five-Nines Reliability: Sample-Efficient LLM Evaluation in Saturated Benchmarks" (2026, preprint) — arXiv:2605.11209 — Explores the reliability axis obscured by saturation, noting that models with identical accuracy can have vastly different failure rates in estimation.
• "Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models" (2026, preprint) — arXiv:2606.19544 — Analyzes roughly 540,000 judgments, examining kappa decay, rank shifting, and the consistency–bias paradox.
• "Position: Evaluation Scores Are Perishable Knowledge Claims" (2026, preprint) — arXiv:2607.26191 — Discusses confidence inflation, formality, scope, validity duration, and top-tier discrepancies between average aggregation and weakest-signal aggregation.