Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance
Abstract
This paper presents an information-gain-guided reinforcement-learning sensor-selection framework for single-vessel tracking in heterogeneous maritime sensor networks. The proposed approach is motivated by information-theoretic sensor management: instead of activating all sensors or repeatedly performing computationally expensive online expected-information-gain evaluation, a learned policy selects one tracking-relevant sensor at each decision epoch. A Bayesian sequential Monte Carlo tracker estimates the vessel state from noisy measurements and provides a belief representation for scheduling under nonlinear and non-Gaussian conditions. A Proximal Policy Optimization agent selects one of five sensors in a georeferenced simulation of the CMMI Smart Marina testbed at Ayia Napa Marina, Cyprus. The policy is trained on the testbed's actual five-sensor configuration. The agent observes belief-state, detection-history, coverage, sensor-geometry, and realized-information-gain features. The reward is defined as a realized-information-gain term gated by an observability mask. Final-test simulations compare the proposed framework with random single-sensor selection, always-on sensing using all sensors simultaneously, and the expected-information-gain sensor-selection baseline proposed in our previous work. Results show that the learned policy achieves tracking performance close to always-on sensing while activating only one sensor per decision time step and avoiding the computationally expensive online entropy search required by expected-information-gain selection. Additional zero-shot evaluation without retraining on ten moderately perturbed versions of actual layout configuration showed broadly stable tracking, with any increase in positional tracking error remaining below 1 meter across all perturbations.
Related
- Continuous-Time Mean-Variance Portfolio Selection Reinforcement Learning Problems for a Markov Regime-Switching System with No-Shorting Constraints
- HALO: Hierarchical Evidence Selection and Reinforcement Learning for Allocating Budgeted Label Optimization from Longitudinal Clinical Records
- Feature Space Selection and Heterogeneous Effect Estimation for Blood-Brain Barrier Permeability: A Random Forest to the Generalized Random Forest Pipeline
- Reinforcement Learning Under Runtime Confounding: Policy Learning and Off-Policy Evaluation
- Deep Reinforcement Learning for Blind Passive Beamforming in RIS-Assisted MIMO Systems
BibTeX
@misc{starodubov2026reinforcement,
title = {{Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance}},
author = {Andrei Starodubov and Yaqub Aris Prabowo and Andreas Hadjipieris and Roberto Galeazzi and Ioannis Kyriakides},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.22667},
eprint = {2607.22667},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.22667},
}
RIS
TY - UNPB
TI - Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance
AU - Andrei Starodubov
AU - Yaqub Aris Prabowo
AU - Andreas Hadjipieris
AU - Roberto Galeazzi
AU - Ioannis Kyriakides
PY - 2026
DA - 2026/09/02
PB - arXiv
DO - 10.48550/arxiv.2607.22667
UR - https://arxiv.org/abs/2607.22667
AB - This paper presents an information-gain-guided reinforcement-learning sensor-selection framework for single-vessel tracking in heterogeneous maritime sensor networks. The proposed approach is motivated by information-theoretic sensor management: instead of activating all sensors or repeatedly performing computationally expensive online expected-information-gain evaluation, a learned policy selects one tracking-relevant sensor at each decision epoch. A Bayesian sequential Monte Carlo tracker estimates the vessel state from noisy measurements and provides a belief representation for scheduling under nonlinear and non-Gaussian conditions. A Proximal Policy Optimization agent selects one of five sensors in a georeferenced simulation of the CMMI Smart Marina testbed at Ayia Napa Marina, Cyprus. The policy is trained on the testbed's actual five-sensor configuration. The agent observes belief-state, detection-history, coverage, sensor-geometry, and realized-information-gain features. The reward is defined as a realized-information-gain term gated by an observability mask. Final-test simulations compare the proposed framework with random single-sensor selection, always-on sensing using all sensors simultaneously, and the expected-information-gain sensor-selection baseline proposed in our previous work. Results show that the learned policy achieves tracking performance close to always-on sensing while activating only one sensor per decision time step and avoiding the computationally expensive online entropy search required by expected-information-gain selection. Additional zero-shot evaluation without retraining on ten moderately perturbed versions of actual layout configuration showed broadly stable tracking, with any increase in positional tracking error remaining below 1 meter across all perturbations.
ER -