Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data
Abstract
The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. However, generating high-utility synthetic data often carries the risk of memorizing and regurgitating private information from the training corpus. In this work, we present a customizable empirical auditing framework designed to detect and explain such data disclosures. Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures''-where the system incidentally generates a user's data. By partitioning input data into training and holdout sets and applying rigorous statistical hypothesis testing, we determine if observed disclosures are consistent with strict privacy baselines, such as zero-learning or specific Differential Privacy (DP) bounds. Crucially, this approach requires no model access, no canary insertion, and no reference model training -only the synthetic output and a held-out control set. We demonstrate that this framework effectively functions as a membership inference attack, providing empirical lower bounds on privacy leakage that are tighter than prior data-based auditing methods. Our approach is model-agnostic, applies to any synthetic data generation mechanism, and requires orders of magnitude fewer computational resources than shadow-model or canary-based alternatives.
Related
- Statistical Analysis of Synthetic Data with Verification: A Linear Regression Framework
- A Formal Graphical Inference Framework for Combining Effect Size with Statistical Significance: Application to Multivariate and Functional Linear Models
- A Constraint-Aware Generative Framework for Synthetic Origin-Destination Demand in Logistics Networks
- Information-Geometric Grammar Principle: A Statistical-Geometric Framework for Adaptive Systems
- A Hybrid Spatial Statistical Learning Framework for Individualized Probability Estimation: Application to Multisite Autism Neuroimaging
BibTeX
@misc{amin2026phantoms,
title = {{Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data}},
author = {Kareem Amin and Rudrajit Das and Alessandro Epasto and Adel Javanmard and Dennis Kraft and Mónica Ribero and Sergei Vassilvitskii},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2606.16952},
eprint = {2606.16952},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.16952},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data
AU - Kareem Amin
AU - Rudrajit Das
AU - Alessandro Epasto
AU - Adel Javanmard
AU - Dennis Kraft
AU - Mónica Ribero
AU - Sergei Vassilvitskii
PY - 2026
DA - 2026/11/04
PB - arXiv
DO - 10.48550/arxiv.2606.16952
UR - https://arxiv.org/abs/2606.16952
AB - The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. However, generating high-utility synthetic data often carries the risk of memorizing and regurgitating private information from the training corpus. In this work, we present a customizable empirical auditing framework designed to detect and explain such data disclosures. Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures''-where the system incidentally generates a user's data. By partitioning input data into training and holdout sets and applying rigorous statistical hypothesis testing, we determine if observed disclosures are consistent with strict privacy baselines, such as zero-learning or specific Differential Privacy (DP) bounds. Crucially, this approach requires no model access, no canary insertion, and no reference model training -only the synthetic output and a held-out control set. We demonstrate that this framework effectively functions as a membership inference attack, providing empirical lower bounds on privacy leakage that are tighter than prior data-based auditing methods. Our approach is model-agnostic, applies to any synthetic data generation mechanism, and requires orders of magnitude fewer computational resources than shadow-model or canary-based alternatives.
ER -