Reinforcement learning enables autonomous microrobot navigation and intervention in simulated blood capillaries
Abstract
Autonomous microrobots navigating biological vasculature could enable targeted drug delivery and thrombolysis, yet training control policies for physiologically complex environments remains an open challenge. Prior reinforcement learning (RL) studies of microrobotic navigation have largely used idealized geometries lacking realistic flow in confined branching, and dense cellular obstacles. Here, we develop a physically grounded, deliberately idealized two-dimensional capillary simulation, with resolved flow fields, explicit red blood cell dynamics, and schematic-derived branching geometry, training RL agents via an advection-dominated concentration field driven by physiological flow speeds. Mapping success across robot size and swimming speed, we show via zero-shot transfer that most training failures reflect optimization limits, not physical ones. A policy transferred from a more demanding condition succeeds far beyond where per-combination training converges. Its eventual collapse offers a tentative lower bound on what is physically achievable. Successful agents converge on Run-and-Rotate strategies. The transferred policy, a variant of this family, shows a distinctive profile, that we believe is responsible for its exceptional transfer success. Without retraining, these agents perform targeted blocking and unblocking of individual vessel branches, restoring red blood cell passage toward baseline levels and suggesting RL as a viable framework for microrobotic intervention in biological networks.
Related
- Autonomous Droplet Navigation via Model-Based Reinforcement Learning
- Multi-criteria prioritized reinforcement learning for autonomous underwater vehicle navigation in 3D multi-node environments
- A Fermatean fuzzy CRITIC WASPAS framework for conflict detection in autonomous drone navigation
- Hybrid Vision Perception for Low-Cost Autonomous Vehicle Navigation
- GIS and landmark-aided autonomous vehicle localization for navigation continuity in urban corridors
BibTeX
@article{drotleff2026reinforcement,
title = {{Reinforcement learning enables autonomous microrobot navigation and intervention in simulated blood capillaries}},
author = {Jannik Drotleff and Paul Hohenberger and Samuel Tovey and Christoph Lohrmann and Julian Hoßbach and Konstantin Nikolaou and Christian Holm},
journal = {Scientific Reports},
year = {2026},
month = nov,
doi = {10.1038/s41598-026-77959-8},
url = {https://www.nature.com/articles/s41598-026-77959-8},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Reinforcement learning enables autonomous microrobot navigation and intervention in simulated blood capillaries
AU - Jannik Drotleff
AU - Paul Hohenberger
AU - Samuel Tovey
AU - Christoph Lohrmann
AU - Julian Hoßbach
AU - Konstantin Nikolaou
AU - Christian Holm
PY - 2026
DA - 2026/11/03
JO - Scientific Reports
DO - 10.1038/s41598-026-77959-8
UR - https://www.nature.com/articles/s41598-026-77959-8
AB - Autonomous microrobots navigating biological vasculature could enable targeted drug delivery and thrombolysis, yet training control policies for physiologically complex environments remains an open challenge. Prior reinforcement learning (RL) studies of microrobotic navigation have largely used idealized geometries lacking realistic flow in confined branching, and dense cellular obstacles. Here, we develop a physically grounded, deliberately idealized two-dimensional capillary simulation, with resolved flow fields, explicit red blood cell dynamics, and schematic-derived branching geometry, training RL agents via an advection-dominated concentration field driven by physiological flow speeds. Mapping success across robot size and swimming speed, we show via zero-shot transfer that most training failures reflect optimization limits, not physical ones. A policy transferred from a more demanding condition succeeds far beyond where per-combination training converges. Its eventual collapse offers a tentative lower bound on what is physically achievable. Successful agents converge on Run-and-Rotate strategies. The transferred policy, a variant of this family, shows a distinctive profile, that we believe is responsible for its exceptional transfer success. Without retraining, these agents perform targeted blocking and unblocking of individual vessel branches, restoring red blood cell passage toward baseline levels and suggesting RL as a viable framework for microrobotic intervention in biological networks.
SN - 2045-2322
ER -