From Pixels to Privacy-Sensitive Features: An Information-Theoretic Framework for Reconstruction Bounds

Abstract

What can an adversary recover from a model trained with DP-SGD? Its (ε,δ) guarantee does not answer this question directly. We derive a distortion bound that can be computed before training and applies even to an adversary who observes the parameter trajectory during training, knows every other training record and the population distribution from which the training records are drawn. The analyst may choose the representation in which error is measured, allowing the bound to target the aspects of reconstruction that are privacy-sensitive in a given application. Our information-theoretic framework separates two contributions: the information about a record revealed by training, measured through conditional mutual information, and the uncertainty already present in the data distribution, measured through source entropy. Rate-distortion theory combines them into a lower bound on the expected error of every reconstruction attack. For DP-SGD, the leakage bound depends only on the sampling rates and noise multipliers, while the source entropy is lower-bounded from a sample with finite-sample confidence. An orthogonal block construction keeps the resulting bound informative for high-dimensional images. We instantiate the framework in pixel space and in LPIPS feature space, obtaining, to our knowledge, the first lower bound on reconstruction distortion measured directly by LPIPS. On a range of image datasets the bounds establish non-trivial unavoidable expected reconstruction error within heuristically recommended privacy-budget ranges and beyond. Experiments compare the bounds with strong reconstruction attacks and identify the principal sources of the remaining slack.

From Source Reconstruction to Predictive State Preservation: An Information-Theoretic Framework for AI-Native Communication
Yi Wang, Linglong Dai
arXiv1 Sep 2026PreprintarXiv licenceFull textGeneral mathematicsarXiv
A Joint Information-Theoretic and Graph Learning Framework for Sensor Placement and Macroscopic Traffic State Reconstruction
Ying Zhang, Fatemeh Fakhrmoosavi, Arash E. Zaghi
arXiv28 Sep 2026PreprintarXiv licenceFull textGeneral mathematicsarXiv
Reconstruction of Geometric and Structural Aspects of Undeciphered Messages by Information-theoretic Methods
Hector Zenil, Abicumaran Uthamacumaran
arXiv5 Oct 2026PreprintarXiv licenceFull textGeneral mathematicsarXiv
Intent-Hiding Jailbreaks: An Information-Theoretic Framework for Compositional Attacks
Fengwei Tian, Ravi Tandon
arXiv1 Oct 2026PreprintarXiv licenceFull textGeneral mathematicsarXiv
A Category-theoretic Reconstruction of Logical Expressivism
Kristopher Brown
arXiv11 Oct 2026PreprintCC BY 4.0Full textAlgebra & number theoryarXiv

Cite

BibTeX RIS

BibTeX

@misc{bourne2026from,
  title = {{From Pixels to Privacy-Sensitive Features: An Information-Theoretic Framework for Reconstruction Bounds}},
  author = {Henry Bourne and Robert F. Allison},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.13768},
  eprint = {2610.13768},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.13768},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - From Pixels to Privacy-Sensitive Features: An Information-Theoretic Framework for Reconstruction Bounds
AU  - Henry Bourne
AU  - Robert F. Allison
PY  - 2026
DA  - 2026/10/09
PB  - arXiv
DO  - 10.48550/arxiv.2610.13768
UR  - https://arxiv.org/abs/2610.13768
AB  - What can an adversary recover from a model trained with DP-SGD? Its (ε,δ) guarantee does not answer this question directly. We derive a distortion bound that can be computed before training and applies even to an adversary who observes the parameter trajectory during training, knows every other training record and the population distribution from which the training records are drawn. The analyst may choose the representation in which error is measured, allowing the bound to target the aspects of reconstruction that are privacy-sensitive in a given application. Our information-theoretic framework separates two contributions: the information about a record revealed by training, measured through conditional mutual information, and the uncertainty already present in the data distribution, measured through source entropy. Rate-distortion theory combines them into a lower bound on the expected error of every reconstruction attack. For DP-SGD, the leakage bound depends only on the sampling rates and noise multipliers, while the source entropy is lower-bounded from a sample with finite-sample confidence. An orthogonal block construction keeps the resulting bound informative for high-dimensional images. We instantiate the framework in pixel space and in LPIPS feature space, obtaining, to our knowledge, the first lower bound on reconstruction distortion measured directly by LPIPS. On a range of image datasets the bounds establish non-trivial unavoidable expected reconstruction error within heuristically recommended privacy-budget ranges and beyond. Experiments compare the bounds with strong reconstruction attacks and identify the principal sources of the remaining slack.
ER  -

Abstract

Related

Cite