DISCOVERING BLUE TEAM SOLUTIONS FOR AN AUTONOMOUS CYBER OPERATIONS CHALLENGE USING AN EVOLUTIONARY HEURISTIC SEARCH

Abstract

In this thesis, a novel machine learning-based approach to autonomous network defence is introduced. The approach utilises an evolutionary strategy to optimise heuristic blue team agents. One approach typically assumed for approaching this problem would deploy a (complex) neural network to discover an appropriate blue agent policy through reinforcement learning against a ‘red’ team on a simulated network environment. Conversely, in this work, we use blue team knowledge regarding network topology and possible attack vectors to define a default defensive heuristic. In common with neural solutions, a preprocessed observation space is assumed in which ‘host scan state’ is expressed. However, we categorised actions in the action space to impose a structured action selection strategy, enabling a defensive efficiency to be maximized using an evolutionary strategy, i.e. a form of Steepest Assent Hill Climbing. Our approach was benchmarked using a simulated network environment with three subnets and diverse adversaries called TTCP CAGE Challenge 2. The CAGE Challenge 2 task defines two types of attacking agents: b_line and meander. We demonstrate that the red b_line agent was countered through a strategy that prioritized the defence of critical hosts. Defending against the adaptive red meander agent required a tiered strategy treating hosts with varying importance levels. Our model achieved second place on the official ranking board (consisting of 16 solutions based on different deep learning frameworks) and surpassed the champion team while performing testing on an updated simulation engine. These results show the potential of evolutionary strategies for advancing AI-driven cyber defence. Specifically, we develop valuable insights into how researchers in the field can utilize knowledge about task representation for discovering efficient solutions for cyber-defence.