Continuous 3-D Latent Diffusion for Medical Image Generation and Reconstruction
Abstract
High-resolution three-dimensional (3-D) medical latent diffusion remains limited by the cost of decoding full volumes, which existing latent autoencoders (AEs) handle through overlapping sub-volume decoding. We propose a continuous 3-D latent diffusion model framework for computed tomography (CT) and magnetic resonance imaging (MRI) generation and measurement-guided reconstruction, built on a coordinate-rendering autoencoder (CoRe-AE). Its decoder applies its convolutional stage once on the compact latent grid and renders voxel intensities from spatial coordinates through a lightweight implicit head, evaluated in memory-bounded chunks. This design avoids overlapping sub-volumes and remains differentiable with respect to the latent, so that the decoder can be used inside measurement-guided reconstruction. The framework was evaluated on CT volumes of 512\textsuperscript{3} voxels and MRI volumes of 256\textsuperscript{3} voxels. At 512\textsuperscript{3}, the decoder was 4--18 times faster than the MAISI and 3D MedDiffusion AEs and required the lowest peak graphics processing unit (GPU) memory, including when its gradient was computed. CoRe-AE also achieved the highest peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) among the evaluated AEs on both modalities. The resulting frozen 3-D latent prior generated full volumes without patch seams and, without retraining, reconstructed sparse-view CT and accelerated MRI through hard data consistency, within 2.4--4.9 dB PSNR of the pixel-domain decomposed diffusion sampler (DDS). A single volumetric latent prior thus supports both generation and reconstruction of high-resolution medical volumes on one GPU. Our code will be made available at https://github.com/Mellak/continuous-3d-latent-diffusion/.
Related
- High-fidelity dual-domain photoacoustic data synthesis with latent diffusion models
- SymADiT: Symmetry-Aware Crystal Generation with Latent Flow Matching
- EPC-3D-Diff: Equivariant Physics Consistent Conditional 3D Latent Diffusion for CBCT to CT Synthesis
- Continuous Hasenbusch transport towards gauge diffusion with fermions
- Fully Quantum Latent Diffusion for Synthetic Malware Records: Effective Denoising, Limited Class Information, and Costly Post-Selection
BibTeX
@misc{mellak2026continuous,
title = {{Continuous 3-D Latent Diffusion for Medical Image Generation and Reconstruction}},
author = {Youness Mellak and Antoine De Paepe and Dimitris Visvikis and Alexandre Bousse},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.16491},
eprint = {2607.16491},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.16491},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Continuous 3-D Latent Diffusion for Medical Image Generation and Reconstruction
AU - Youness Mellak
AU - Antoine De Paepe
AU - Dimitris Visvikis
AU - Alexandre Bousse
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2607.16491
UR - https://arxiv.org/abs/2607.16491
AB - High-resolution three-dimensional (3-D) medical latent diffusion remains limited by the cost of decoding full volumes, which existing latent autoencoders (AEs) handle through overlapping sub-volume decoding. We propose a continuous 3-D latent diffusion model framework for computed tomography (CT) and magnetic resonance imaging (MRI) generation and measurement-guided reconstruction, built on a coordinate-rendering autoencoder (CoRe-AE). Its decoder applies its convolutional stage once on the compact latent grid and renders voxel intensities from spatial coordinates through a lightweight implicit head, evaluated in memory-bounded chunks. This design avoids overlapping sub-volumes and remains differentiable with respect to the latent, so that the decoder can be used inside measurement-guided reconstruction. The framework was evaluated on CT volumes of 512\textsuperscript{3} voxels and MRI volumes of 256\textsuperscript{3} voxels. At 512\textsuperscript{3}, the decoder was 4--18 times faster than the MAISI and 3D MedDiffusion AEs and required the lowest peak graphics processing unit (GPU) memory, including when its gradient was computed. CoRe-AE also achieved the highest peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) among the evaluated AEs on both modalities. The resulting frozen 3-D latent prior generated full volumes without patch seams and, without retraining, reconstructed sparse-view CT and accelerated MRI through hard data consistency, within 2.4--4.9 dB PSNR of the pixel-domain decomposed diffusion sampler (DDS). A single volumetric latent prior thus supports both generation and reconstruction of high-resolution medical volumes on one GPU. Our code will be made available at https://github.com/Mellak/continuous-3d-latent-diffusion/.
ER -