{
  "abstract": "The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully conditioning, a new encoder. For jointly Gaussian data, we show that the IB solution factorizes as $A(β)=D(β)W$: the information-bearing directions $W$ do not depend on $β$, and $β$ acts only through scalar gains. We ask whether this separation of \\emph{representation} from \\emph{rate control} survives beyond Gaussian models, treating the budget as a given operating parameter rather than a quantity to be optimized. For a budget-free representation \\(F(X)\\), the \\emph{separation gap} is controlled by the \\emph{sufficiency deficit} $ε_F=I(X;Y\\mid F(X))$: We show there is at least $ε_F$ in the worst case for finite inputs, and it is bounded uniformly in $λ$ by a function of $ε_F$ for finite label alphabets. The same certificate extends to a new target distribution and to pooled multi-domain data, where a conditional-alignment term appears. On MNIST, SVHN, CIFAR-10 and STL-10 (10 seeds, eight prespecified budgets), a frozen CNN representation with one compressor--decoder pair per budget reduces estimated training multiply--accumulate operations (MACs) by $7.7\\times$ and wall-clock time by about $2\\times$ relative to end-to-end VIBs, and attains higher budget-averaged accuracy ($65.6\\%$ versus $57.3\\%$ per domain; $67.1\\%$ versus $57.1\\%$ pooled), with the advantage concentrated at strong compression, where the VIBs collapse. At weak compression ($λ\\le0.036$), VIB has higher cross-entropy and ECE; its surrogate Lagrangian is higher per domain, indistinguishable from the pooled VIB in the macro average, and lower on the pooled test set.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2610.36255",
  "author_total": 2,
  "authors": [
    {
      "name": "Hassan Tavakoli"
    },
    {
      "name": "Bella Bose"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/1p2f5m9fn0cqy.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2610.36255",
  "fulltext_url": null,
  "id": "1p2f5m9fn0cqy",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2610.36255",
  "language": null,
  "licence": "LicenseRef-arXiv-nonexclusive-1.0",
  "licence_label": "arXiv licence",
  "licence_source": "article",
  "licence_url": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.IT"
    ],
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "math.IT"
    ],
    [
      "openalex",
      "1707|Advanced Neural Network Applications"
    ]
  ],
  "openalex": "W7224125760",
  "pdf_url": "https://arxiv.org/pdf/2610.36255",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-26",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/1p2f5m9fn0cqy.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:35:53Z",
      "native_id": "2610.36255",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2610.36255"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:math.IT",
      "subfield": "general"
    }
  ],
  "title": "What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control",
  "title_text": "What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control",
  "type": "preprint",
  "updated_at": "2026-10-30T12:35:53Z",
  "url": "https://research.draconitic.com/paper/1p2f5m9fn0cqy",
  "venue": "arXiv",
  "versions": []
}