Semantic-Aware Neural Video Codec for Error-Resilient Low-Latency Transmission
Abstract
Emerging physical AI systems require low-latency, task-oriented video communication over unreliable channels. We propose a semantic-aware multi-level neural video coding method for robust low-latency video transmission over unreliable channels that are abstracted as multi-level packet erasure channels. Built upon the real-time DCVC-RT neural video codec, the proposed framework introduces a semantic- and feature-aware coding strategy that partitions encoded representations into packets carrying different levels of semantic and latent-feature importance and assigns these packets to different streams, each associated with a priority level when transmitted over unreliable communication channels. We also developed an error-resilient entropy model that removes inter-packet dependencies, allowing each packet to be decoded independently under packet losses. The complete system is trained end-to-end over the abstracted multi-level packet erasure channels, enabling learning of channel-aware representations together with importance-aware packet assignment while facilitating the network for differentiated packet prioritization. Experiments show that the proposed framework significantly improves robustness over baseline DCVC-RT under packet erasures, achieving graceful degradation in less important regions while better preserving task-relevant visual content.
Related
- Resilient Cell-Free MIMO ISAC Under Uncertainties: An Impairment-Aware Graph Neural Network Framework
- Resilient Monitoring of Social Dynamical Systems through Collaborative Multi-Agent Networks under Latency
- POST-MIMO: Partition-Oriented Semantic Transmission for MIMO Point Cloud Delivery
- Semantic Freshness Optimal Sampling and Transmission for Gossiping Receivers
- SkillComm: Skill-Driven Semantic Communication for Sequential Workflows via Incremental Token Transmission
BibTeX
@misc{mortaheb2026semanticaware,
title = {{Semantic-Aware Neural Video Codec for Error-Resilient Low-Latency Transmission}},
author = {Matin Mortaheb and Homa Esfahanizadeh and Jinfeng Du and Harish Viswanathan},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.16279},
eprint = {2609.16279},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.16279},
}
RIS
TY - UNPB
TI - Semantic-Aware Neural Video Codec for Error-Resilient Low-Latency Transmission
AU - Matin Mortaheb
AU - Homa Esfahanizadeh
AU - Jinfeng Du
AU - Harish Viswanathan
PY - 2026
DA - 2026/09/14
PB - arXiv
DO - 10.48550/arxiv.2609.16279
UR - https://arxiv.org/abs/2609.16279
AB - Emerging physical AI systems require low-latency, task-oriented video communication over unreliable channels. We propose a semantic-aware multi-level neural video coding method for robust low-latency video transmission over unreliable channels that are abstracted as multi-level packet erasure channels. Built upon the real-time DCVC-RT neural video codec, the proposed framework introduces a semantic- and feature-aware coding strategy that partitions encoded representations into packets carrying different levels of semantic and latent-feature importance and assigns these packets to different streams, each associated with a priority level when transmitted over unreliable communication channels. We also developed an error-resilient entropy model that removes inter-packet dependencies, allowing each packet to be decoded independently under packet losses. The complete system is trained end-to-end over the abstracted multi-level packet erasure channels, enabling learning of channel-aware representations together with importance-aware packet assignment while facilitating the network for differentiated packet prioritization. Experiments show that the proposed framework significantly improves robustness over baseline DCVC-RT under packet erasures, achieving graceful degradation in less important regions while better preserving task-relevant visual content.
ER -