Abstract :

The rapidly evolving of digital environment prompts advanced network security solutions with essential defend against complex cyber threats. However, network security receives a promising boost from the combination of Software-Defined Networking (SDN) and Artificial Intelligence (AI) because which enables real-time control and intelligent decision-making. Real-time management of network resources through SDN allows flexible control while AI boosts the detection of anomalies in large datasets. In this paper we proposed a Forest Tree AI-SDN Firewall with an innovative hierarchical framework that combines these two powerful technologies to provide adaptive network security solutions with scalable and resilient capabilities. The framework draws its design principles from SDN infrastructure based on three separate layers, Root Layer, Trunk Layer and Canopy Layer. Real-time traffic filtering at the Root Layer uses lightweight edge sensors to achieve 98.2% accuracy while its FPGA-accelerated TLS 1.3 inspection system handles 40 Gbps of data. The Trunk Layer uses reinforcement learning algorithms with a federated SDN control plane to achieve dynamic policy optimization through 12ms response times. The Canopy Layer uses deep learning ensemble technology that combines CNN, LSTM and GNN architectures to detect zero-day threats effectively with 99.4% recall and 92% coverage of encrypted traffic analysis. The system achieves 99.2% threat detection precision during benchmark tests while generating 0.8% incorrect alerts and allowing policy updates at speeds 5.2 times faster than conventional security systems. The proposed system evaluating encrypted information and strengthening adversarial resistance together with cross-domain coordination and achieving 38 Gbps/W energy efficiency.

---

Keywords :

Adaptive Network Security, AI-SDN Firewall, Deep learning, Deep Learning. AI-SDN Firewall, Encrypted Traffic Analysis, Encrypted Traffic Inspection, Federated SDN, Forest Tree Architecture, Hierarchical Security, Threat Detection., Zero-Day Detection

---

References :

1. H. Abdi, L. Audah, A. Salh, M. A. Alhartomi, H. Rasheed, S. Ahmed, and A. Tahir, “Security Control and Data Planes of SDN: A Comprehensive Review of Traditional, AI, and MTD Approaches to Security Solutions,” IEEE Access, vol. 12, pp. 47912–47944, Apr. 2024.
2. Shabana, S. Mohmmad, K. Shankar, and Y. Chanti, “AI Based SDN Technology Integration with their Challenges and Opportunities,” Asian Journal of Computer Science and Technology, vol. 8, no. S3, pp. 165–169, 2019.
3. Sina Ahmadi. Next Generation AI-Based Firewalls: A Comparative Study. International Journal of Computer (IJC), 2023.
4. Holik and P. Dolezel, “Industrial Network Protection by SDN-Based IPS with AI,” in Proc. 12th Asian Conference on Intelligent Information and Database Systems (ACIIDS 2020), Phuket, Thailand, Mar. 23–26, 2020.
5. Krishnan, K. Jain, A. Aldweesh, P. Prabu, and R. Buyya, “OpenStackDP: A scalable network security framework for SDN-based OpenStack cloud infrastructure,” Journal of Cloud Computing: Advances, Systems and Applications, vol. 12, no. 26, 2023.
6. Prabakaran, R. Ramar, I. Hussain, B. P. Kavin, S. S. Alshamrani, A. S. AlGhamdi, and A. Alshehri, “Predicting Attack Pattern via Machine Learning by Exploiting Stateful Firewall as Virtual Network Function in an SDN Network,” Sensors, vol. 22, no. 3, p. 709, Jan. 2022.
7. Cheng, C. Wu, H. Zhou, Y. Zhang, R. Wang, and W. Ruan, “Guarding the Perimeter of Cloud-based Enterprise Networks: An Intelligent SDN Firewall,” in Proc. 2018 IEEE 20th Int. Conf. on High Performance Computing and Communications; IEEE 16th Int. Conf. on Smart City; IEEE 4th Int. Conf. on Data Science and Systems, Exeter, UK, 2018Forman, G. 2003. An extensive empirical study of feature selection metrics for text classification. J. Mach. Learn. Res. 3 (Mar. 2003), 1289-1305.
8. Abdulqadder, S. Zhou, D. Zou, I. T. Aziz, and S. M. A. Akber, “Multi-layered Intrusion Detection and Prevention in the SDN/NFV Enabled Cloud of 5G Networks using AI-based Defense Mechanisms,” Computer Networks, vol. 180, p. 107364, 2020.