Understanding the Markov decision process for reinforcement learning in robotics perception
- 1Department of Mining, Industrial and ICT Engineering (EMIT), Manresa School of Engineering (EPSEM), Universitat Politècnica de Catalunya (UPC), BarcelonaTech
- 2Faculty of Computer Science, Electronics, and Telecommunications, AGH University of Krakow
Abstract
The application of robots has increased significantly in recent years in various sectors, including household use, surveillance, medical services, manufacturing, and logistics automation. To guarantee the efficient and secure functioning of such robots, it is essential to develop reliable control mechanisms that can adjust to evolving settings. Reinforcement learning (RL), placed between supervised and unsupervised learning, addresses learning in successive decision-making issues characterized by inadequate feedback. This paper presents a formal framework and comprehensive introduction of the Markov Decision Process (MDP), together with the description of value functions and policies. The principles and perceptions underlying MDPs are presented in this article, along with RL algorithms for calculating optimal behaviors using dynamic programming for a Q-learning-based approach. RL environments for robotics path planning are generally designed as MDPs, with the goal of learning a control strategy to maximize the total reward. Based on multiple interpretations of optimality with regard to the objective of learning sequencing decisions, the primary focus of the present paper is to provide the introduction of fundamental information of MDP-based algorithms to acquire optimal behaviors. Lastly, theoretical aspects are validated in this study with a simulation experiment using the Q-learning approach in a custom robot environment that has various obstacles at different locations. The initial and target positions are defined for the robot in the environment in the grid map. The simulation shows a rate of accuracy of 88.80% for the robot to reach the target point. Furthermore, a comparative performance analysis has been performed between Q-learning and DQN algorithms.
Related
- The correlation between risk perception and glycemic management decision-making behavior in pregnant women with gestational diabetes: a cross-sectional study
- Urgent but isolated: Public perception and understanding of One Health in daily life
- Understanding digital diffusion and online decision-making in public health promotion: a sequential mixed-methods study of the violence prevention charter in Canada
- “It’s not necessarily science based”: understanding organ utilisation decision-making in cardiothoracic transplantation in the UK through qualitative interviews
- Preoperative information burden among parents of children with congenital heart disease: a qualitative study of information access, understanding, and decision-making
BibTeX
@article{raj2026understanding,
title = {{Understanding the Markov decision process for reinforcement learning in robotics perception}},
author = {Ravi Raj and Andrzej Kos},
journal = {Scientific Reports},
year = {2026},
month = jul,
doi = {10.1038/s41598-026-64605-6},
url = {https://doi.org/10.1038/s41598-026-64605-6},
note = {CC BY},
}
RIS
TY - JOUR
TI - Understanding the Markov decision process for reinforcement learning in robotics perception
AU - Ravi Raj
AU - Andrzej Kos
PY - 2026
DA - 2026/07/01
JO - Scientific Reports
DO - 10.1038/s41598-026-64605-6
UR - https://doi.org/10.1038/s41598-026-64605-6
AB - The application of robots has increased significantly in recent years in various sectors, including household use, surveillance, medical services, manufacturing, and logistics automation. To guarantee the efficient and secure functioning of such robots, it is essential to develop reliable control mechanisms that can adjust to evolving settings. Reinforcement learning (RL), placed between supervised and unsupervised learning, addresses learning in successive decision-making issues characterized by inadequate feedback. This paper presents a formal framework and comprehensive introduction of the Markov Decision Process (MDP), together with the description of value functions and policies. The principles and perceptions underlying MDPs are presented in this article, along with RL algorithms for calculating optimal behaviors using dynamic programming for a Q-learning-based approach. RL environments for robotics path planning are generally designed as MDPs, with the goal of learning a control strategy to maximize the total reward. Based on multiple interpretations of optimality with regard to the objective of learning sequencing decisions, the primary focus of the present paper is to provide the introduction of fundamental information of MDP-based algorithms to acquire optimal behaviors. Lastly, theoretical aspects are validated in this study with a simulation experiment using the Q-learning approach in a custom robot environment that has various obstacles at different locations. The initial and target positions are defined for the robot in the environment in the grid map. The simulation shows a rate of accuracy of 88.80% for the robot to reach the target point. Furthermore, a comparative performance analysis has been performed between Q-learning and DQN algorithms.
SN - 2045-2322
ER -