Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling
Abstract
Do larger models learn increasingly similar representations of the same world? The Platonic Representation Hypothesis (PRH) predicts that they should, but similar can mean native cosine geometry---nearest-neighbour structure induced by cosine similarity before any learned alignment---or linear recoverability: correspondence revealed by a supervised linear map. Using matched Legacy Survey and Hyper Suprime-Cam (HSC) embeddings across five model families, we find that Ridge alignment strengthens the relationship between model size and cross-dataset neighbourhood correspondence relative to native cosine geometry. The effect appears in all five families and persists when both representations are reduced to dimension, so it is not simply that wider embeddings give alignment more dimensions to work with. The learned maps remain far from rotations or uniform scaling, and Sparse Autoencoder (SAE) and Block-Sparse Featuriser (BSF) features give no advantage over the same alignment applied to dense embeddings. Model scale thus improves linear recoverability more clearly than native cosine-neighbourhood correspondence, without establishing convergence toward a shared native metric geometry.
Related
- Shared Geometry Is Not Shared Physics: A Layerwise Test of the Platonic Representation Hypothesis in Astronomy
- The Physical Origins of Scatter in the Dust Attenuation Scaling Relation of Star-Forming Galaxies: Star--Dust Geometry
- Winds Against Alignment: AGN Feedback and the Spin Evolution of Massive Black Hole Binaries
- Resistive alignment of a black-hole magnetosphere
- PERRY: A Human-in-the-Loop GUI for Precision Alignment of MUSE Data
BibTeX
@misc{hoy2026native,
title = {{Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling}},
author = {Angus Hoy and Austin Lutterbach and Jasper Chong and Aman Kumar and Michael J. Smith},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.22749},
eprint = {2610.22749},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22749},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling
AU - Angus Hoy
AU - Austin Lutterbach
AU - Jasper Chong
AU - Aman Kumar
AU - Michael J. Smith
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2610.22749
UR - https://arxiv.org/abs/2610.22749
AB - Do larger models learn increasingly similar representations of the same world? The Platonic Representation Hypothesis (PRH) predicts that they should, but similar can mean native cosine geometry---nearest-neighbour structure induced by cosine similarity before any learned alignment---or linear recoverability: correspondence revealed by a supervised linear map. Using matched Legacy Survey and Hyper Suprime-Cam (HSC) embeddings across five model families, we find that Ridge alignment strengthens the relationship between model size and cross-dataset neighbourhood correspondence relative to native cosine geometry. The effect appears in all five families and persists when both representations are reduced to dimension, so it is not simply that wider embeddings give alignment more dimensions to work with. The learned maps remain far from rotations or uniform scaling, and Sparse Autoencoder (SAE) and Block-Sparse Featuriser (BSF) features give no advantage over the same alignment applied to dense embeddings. Model scale thus improves linear recoverability more clearly than native cosine-neighbourhood correspondence, without establishing convergence toward a shared native metric geometry.
ER -