Abstract :

Cyber-Physical Systems (CPS) bring together physical processes with computing, communication, and control. They often operate in environments full of uncertainty, noise, and constant change, which makes traditional deterministic models struggle to capture how these systems really behave. This work introduces a more flexible framework based on Markov processes that helps model, predict, and control CPS in a more realistic way. By viewing system behaviour as probabilistic transitions between states, it becomes easier to analyze uncertainty and understand how the system evolves over time. The study looks at discrete-time Markov chains and expands the discussion to Hidden Markov Models (HMMs) and Markov Decision Processes (MDPs), allowing both visible and hidden aspects of system dynamics to be represented. It outlines a well-defined process for defining states, calculating transition probabilities, and making forecasts. The paper explores, in addition to that, the use of control techniques based on the use of probability theory and shows that these methods have a greater level of robustness compared to traditional control techniques. An example is given to show how this model improves performance and flexibility. All in all, Markov modelling is a good starting point for dealing with the challenges in CPSs, paving the way for integration with other tools.

---

Keywords :

Cyber physical systems, Markov Chains, Markov Decision Processes, Probabilistic Prediction, Stochastic Control, System reliability

---

References :

1. Gao, M., Li, Z., Pang, T., Xu, H., & Chen, S. (2023). Event-based security control for Markov jump cyber–physical systems under denial-of-service attacks. Applied Sciences, 13(21), 11815. https://doi.org/10.3390/app132111815
2. Zaytseva, T., Kravtsova, L., & Kaminskaya, N. (2023). Markov processes in researching the probability of cyber attacks on marine vessels. Journal of Information Technologies in Education, 52, 20–32. https://doi.org/10.14308/ite000763
3. Kovtun, V., Izonin, I., & Gregus, M. (2022). The functional safety assessment of cyber-physical system operation process described by Markov chain. Scientific Reports, 12, 7089. https://doi.org/10.1038/s41598-022-11193-w
4. Lalropuia, K. C., & others. (2019). Modeling cyber-physical attacks based on stochastic game and Markov processes. Reliability Engineering & System Safety, 181, 28–37. https://doi.org/10.1016/j.ress.2018.08.014
5. Gore, R., Padilla, J., & Diallo, S. (2017). Markov chain modeling of cyber threats. SIMULATION, 93(3). https://doi.org/10.1177/1548512916683451
6. Karageorgiou, S., & Karyotis, V. (2022). Markov-based malware propagation modeling and analysis. Network, 2(3), 456–478. https://doi.org/10.3390/network2030028
7. Li, Q. L., Ma, J. Y., Chang, Y. X., & Yu, H. B. (2019). Markov processes in blockchain systems. Computational Social Networks, 6(5). https://doi.org/10.1186/s40649-019-0066-1
8. Nikolov, N. (2024a). Smart cities as a tool for environmental sustainability: opportunities and challenges. Environment. Technology. Resources. Rezekne, Latvia,Proceedings of the International Scientific andPractical Conference, 1, 261-266., DOI: 10.17770/etr2024vol1.7948
9. Nikolov, N. (2024b). Development of an Intelligent System for Managing Energy Consumption in the Home. ComputerScience and Technologies (Technical University of Varna), ISSN 1312-3335, pp. 11-20
10. Nikolov, N. (2024c). ntegration of renewable energy systems in the infrastructure of smart cities. Annual of NaturalSciences Department, 9, 44–52, https://doi.org/10.33919/ansd.24.9.5
11. Nikolov, N. (2024d). Strategies for Optimizing Business Processes in Public Services in Smart Cities. Annual of the University of Telecommunications and Post, 8, 17-21
12. Nikolov, N. (2024e). Algorithm for building a concept for the development of Sofia as a smart city. Annual of the University of Telecommunications and Post, 8, 99-103
13. Nikolov, N. (2025a). The intelligent school: the heart of the smart city. Annual of Natural Sciences Department, ISSN 2367-6302, 2025, 12-24, DOI: 10.33919/ansd.25.10.2
14. Nikolov, N. (2025b). Assessment of the ecological footprint of smart cities through sustainable development indicators. Annual of Natural Sciences Department, ISSN 2367-6302, 40-50, DOI: 10.33919/ansd.25.10.5
15. Peneva, G. (2024). System for Managing Technological Obsolescence in Decisions for the Development of the Production Capacity of an Industrial Enterprise. Sofia.
16. Zhu, C., Wei, J., & Lu, C. (2025). Asynchronous model predictive control of Markov jump CPS under cyber-attacks. International Journal of Systems Science. https://doi.org/10.1080/00207721.2025.2552270
17. Reed, S., & Ziadé, E. (2023). On transient analysis of Markov chains in CPS security contexts. Methodology and Computing in Applied Probability. https://doi.org/10.1007/s11009-023-10002-9
18. Xu, S., Lu, W., & Li, H. (2016). A stochastic model of active cyber defense dynamics. arXiv.
19. Lanotte, R., Merro, M., & Tini, S. (2017). A probabilistic calculus of cyber-physical systems. arXiv.
20. Leong, A. S., Ramaswamy, A., Quevedo, D. E., Karl, H., & Shi, L. (2018). Deep reinforcement learning for CPS scheduling. arXiv.
21. Gu, Y. K., Zhang, J., Shen, Y. J., & Fan, C. J. (2019). Fault tree analysis using Markov chains. Journal of Industrial and Production Engineering. https://doi.org/10.1080/21681015.2019.1645050
22. Alanen, J., et al. (2022). Dependability and risk assessment in industrial CPS. Reliability Engineering & System Safety. https://doi.org/10.1016/j.ress.2021.108270
23. Santos, J. C. S., et al. (2019). Tactical vulnerabilities in complex systems. Journal of Systems and Software, 149, 263–284. https://doi.org/10.1016/j.jss.2018.10.030