The same evidence is not equally usable everywhere. The measured curve depends on the model, prompt, and task.
Hide an answer inside a long context and models find it more easily near the edges than in the middle. That "lost in the middle" result was measured on transformers, but it now gets quoted as a fact about long context itself, including for architectures that work nothing like one.
So we tested it. Within each question, we slide one answer-bearing passage through every position of a fixed document set while preserving the distractors, question, and decoding setup. We repeat that paired sweep across attention, state-space, and hybrid models on identical exploratory questions.
The exploratory 2.8B family curves split at the beginning. Pythia favours the opening, while the two compared Mamba checkpoints do not, and all three favour the end. Depth, positional encoding, state size, checkpoint family, and capability remain confounded, so this is a measured family difference rather than an isolated architecture effect.
The unit of analysis is the question, not the generation. Every bootstrap resample draws complete bundles of all ten positions together, so the interval covers variation across questions rather than across independently sampled prompts.
Both numbers are one subtraction: an end of the context, minus the middle.
Each slot above has its own accuracy. is how often the model answers correctly when the answer sits in slot 5, and so on for every slot.
Switch between the exploratory family comparison, the tightly matched 8B checkpoint contrast, the corpus control, and the production systems. Colour encodes the mixer class: attention, state-space, hybrid.
If attention contributes to primacy, the released pure and hybrid 8B checkpoints provide the tightest comparison in this study. They come from the same lab, training data, and scale, but the hybrid changes both attention and MLP composition, so the contrast is not attention-only.
The hybrid does show a primacy effect on its own, and the pure model does not. But the honest comparison is the gap between them, and that gap is small with an interval that still includes zero. So the result points the same way as the main comparison without proving the case.
Every comparison on this page puts two models against each other on exactly the same questions. That matters: it means the difference is measured directly, rather than inferred from two separate scores.
Repeating the family comparison at five approximate size pairs finds a near-zero gap in the two weakest pairs and a positive gap in the three capable pairs. This descriptive pattern does not establish a causal size threshold.
A null result is only informative if the measurement could have detected a real effect. Before any model comparison, the harness was shown to recover a known position effect end to end.
The public author list and archival URL are not yet available. Until they are, please cite by title.
@misc{mixingmatters2026,
title = {Mixing Matters? Evidence and Its Limits for Position Bias
Across Sequence Mixers},
note = {New in ML workshop submission},
year = {2026}
}This work builds on the position-intervention protocol of Liu et al. (2024) and evaluates models introduced by Biderman et al. (2023), Gu and Dao (2024), Dao and Gu (2024), and Waleffe et al. (2024). Synthetic retrieval uses RULER; the sink analysis follows Xiao et al. (2024).