Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data
Abstract
Supervised contrastive learning and its balanced variants have demonstrated their effectiveness for representation learning under long-tailed distributions. However, although prior analyses suggest that their optimal representations may exhibit a regular simplex geometry, whether such a geometry is indeed exhibited remains insufficiently understood. In this work, we present a theoretical framework for these objectives that characterizes whether a regular simplex is formed at the optimum. By applying this to representative objectives, we show that these objectives fail to form a regular simplex at the optimum because they remain dependent on the number of samples per class. Motivated by this analysis, we introduce structural balancing that can be applied to contrastive objectives, and propose a fully balanced contrastive objective that provably forms a regular simplex geometry at its optimum. Both theoretical and empirical results demonstrate that the proposed method yields a more balanced representation space and improves performance on long-tailed recognition and transfer learning tasks.
Related
- Joint Contrastive-Adversarial Representation Learning for Unsupervised Domain Adaptation
- Anchor Divergence for Semantic Geometry in Contrastive Learning
- Dual-domain self-supervised feature alignment via spectral–spatial representation learning for deepfake anomaly detection
- From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model
- Geometry-Aware Latent Representation Learning for Drag Coefficient Prediction of Irregular Particles
BibTeX
@misc{jeong2026rethinking,
title = {{Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data}},
author = {Joonhyeok Jeong and Jy-yong Sohn and Kibok Lee},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.32039},
eprint = {2610.32039},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.32039},
}
RIS
TY - UNPB
TI - Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data
AU - Joonhyeok Jeong
AU - Jy-yong Sohn
AU - Kibok Lee
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2610.32039
UR - https://arxiv.org/abs/2610.32039
AB - Supervised contrastive learning and its balanced variants have demonstrated their effectiveness for representation learning under long-tailed distributions. However, although prior analyses suggest that their optimal representations may exhibit a regular simplex geometry, whether such a geometry is indeed exhibited remains insufficiently understood. In this work, we present a theoretical framework for these objectives that characterizes whether a regular simplex is formed at the optimum. By applying this to representative objectives, we show that these objectives fail to form a regular simplex at the optimum because they remain dependent on the number of samples per class. Motivated by this analysis, we introduce structural balancing that can be applied to contrastive objectives, and propose a fully balanced contrastive objective that provably forms a regular simplex geometry at its optimum. Both theoretical and empirical results demonstrate that the proposed method yields a more balanced representation space and improves performance on long-tailed recognition and transfer learning tasks.
ER -