Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

Noosaga Placements

• Deep Learningframework95%
  The paper is situated within the deep learning framework, specifically focusing on transformer-based large language models and their internal representations.
  A parallel thread in mechanistic interpretability has by now built a sharp toolkit for that question, showing that many high-level behaviors... are concentrated along low-dimensional directions in a transformer's residual streamWe report three findings, across seven judges, seven bias types, and nine benchmarks
• Deep Learningsubfield95%
  The paper focuses on mechanistic interpretability of large language models (transformers), analyzing hidden states, activation geometry, and steering interventions in LLMs used as judges. This is core deep learning research on understanding and controlling LLM behavior.
  We argue that LLM-as-judge bias is best read as a representation-level phenomenon, an internal mapping from surface cues to score predictions that lives in the judge's hidden stateSteering hidden states along this subspace drives scoring in both directionsacross seven judges, seven bias types, and nine benchmarks
• Large Language Modelsframework95%
  The paper is entirely situated within the context of large language models used as judges, studying their internal representations and biases.
  Large language models are now routinely deployed as automatic judgesacross seven judges, seven bias types, and nine benchmarks
• Natural Language Processingsubfield90%
  The paper studies LLM-as-judge scoring bias, which is a core NLP evaluation problem. It uses NLP benchmarks and focuses on language model behavior in evaluation tasks.
  Large language models are now routinely deployed as automatic judges: they rate answers, compare candidates, and supply reward signals for alignment and preference learningWe report three findings, across seven judges, seven bias types, and nine benchmarks
• Representation Learningframework90%
  The paper uses representation learning concepts and methods to analyze and manipulate hidden state representations of LLMs, identifying bias subspaces and using them for causal control and prediction.
  We argue that LLM-as-judge bias is best read as a representation-level phenomenon, an internal mapping from surface cues to score predictions that lives in the judge's hidden statebaseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace
• Ai Safetysubfield85%
  The paper addresses bias in LLM judges, which is a fairness and safety concern. It proposes methods for detecting and causally controlling bias, contributing to AI safety and interpretability.
  Such scoring biases undermine benchmark validity and, because judges sit inside RLHF and evaluation pipelines, silently propagate into the models they are meant to audita simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks
• Interpretabilityframework90%
  The paper uses mechanistic interpretability methods (activation steering, representation analysis) to understand and control bias in LLM judges.
  Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge BiasSteering hidden states along this subspace drives scoring in both directions
• Machine Learningsubfield70%
  The paper uses machine learning methods (PCA, LDA, logistic regression, LightGBM) for analysis and prediction, but the primary contribution is not to general ML theory.
  We evaluate two predictors on them: a simple linear projection (logistic regression on the per-layer bias-direction projections) and a more expressive GBDT pipeline (LightGBM with recursive feature elimination and Optuna search)
• Transformer Architectureframework90%
  The paper studies transformer-based LLMs and their internal representations, specifically analyzing hidden states across layers.
  showing that many high-level behaviors... are concentrated along low-dimensional directions in a transformer's residual streamrecord the hidden state of the final token at the output of every decoder layer

Abstract

Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/

Paper Context