Developing and Evaluating Metrics for Automated Performance Grading in Capture the Flag Games

Capture the Flag (CTF) games are used both in the practical training of cybersecurity skills and to identify talents (e.g., in competitions). These use cases require a rigorous metric for assessing the performance of participants. The goal of this work is to develop and evaluate techniques for automated performance grading (i.e., scoring), which are based on analyzing CTF game data. Existing research focuses on autograding in the domain of teaching programming, but similar research in the cybersecurity domain is almost non-existent. Addressing this area would bring two major contributions. One is supporting instructors and game organizers in a post-game evaluation of participants' skills. The other is building predictive models that will employ the metrics to identify various groups of students, e.g., those at the risk of dropping out. 

Active Learning in ICS/SCADA Environment 

Industrial control systems play a critical role in nations’ critical information infrastructure. Any security breach can lead to a disaster including loss of life. Although there are first attempts to incorporate operational technology in well-established games, exercises and trainings primarily focused on information technology, a systematic approach is missing. The goal of this PhD is to propose methods and new formats of efficient active learning focused on ICS/SCADA. The research will be conducted at a new ICS/SCADA testbed purchased in 2019.  

Interactive Visualizations in Cybersecurity 

Interactive visualizations can help in better understanding and handling of security incidents. The goal of the PhD is to explore different ways to visualize and interact with knowledge extracted automatically from very large heterogeneous datasets from the network monitoring tools. The student will be working within the team of researchers and incident handlers of the CSIRT-MU. Since the underlying infrastructure for collecting the data already exists, the student will have immediate access to data and users. User evaluation sessions (workshops, interviews, tests) with the incident handlers will be part of the work, in order to validate the visualization design.  

Computer-Supported Cooperative Work in Cybersecurity Teams

Incident handlers usually work in co-located teams and regularly interact with other stakeholders (e.g., security managers, superior CSIRT, etc.). The need for collaboration support between them is crucial, especially when dealing with time-critical incidents. The goal of the PhD is to explore workflows that can be beneficial for the incident handlers of the CSIRTs. The topic includes an analysis of the current workflows and defining the new ways to improve them from the user-perspective. Additionally, the design and evaluation of novel tools that helps to communicate the incident among multiple people both co-located and in distance. User evaluation sessions (workshops, interviews, tests) with the incident handlers will be part of the work, in order to validate the proposed methods. The focus of the work is laid more on co-located collaboration in multi-display environments and high-resolution display walls to improve situation awareness of the CSIRT members. 

Monitoring and Measurement of Hosts’ Compliance with Network Security Policies

Active and passive network measurements, probing, and monitoring are used to collect information on the computer network. Enumeration of assets, the discovery of open ports and services, device fingerprinting, and vulnerability discovery are examples of tasks that could be performed using the methods above in order to build and maintain cyber situational awareness. However, isolated pieces of information might be misleading, and particular measurement tasks require different scope and level of details. 

The goal of the Ph.D. is to assess security policies that could be checked using network measurements. The preliminary list would include patching and update policies, setting proper firewall rules, not opening unnecessary ports and services, running updated antivirus software, using strong crypto, having valid certificates, etc. Subsequently, the student would design and evaluate techniques to check for compliance with the security policies using the methods of active network probing and passive network traffic monitoring. 

Automated Security Operations and Integrated Threat Management

Security operations represent a complex system of people, processes, and technologies with the goal of managing organizational threats. Security operations are typically supported by several core capabilities, including data & information management, asset & risk management, vulnerability management, security event management, incident case management, and feedback management. The contemporary hypothesis is simple - the more these capabilities become automated and integrated (via loose coupling), the more streamlined the security operations become. The candidates will study, analyze, and map the existing security operations landscape and select well-defined tasks within the core capabilities that have a potential for improvement. The candidates will propose novel approaches and methods in order to improve these tasks measurably (quantitatively), e.g., in terms of accuracy, computational complexity, latency, number of manual steps, incident detection time, or incident response time. The examples of such tasks include asset discovery, impact assessment, anomaly detection, incident triage, threat detection, or mitigation strategy selection. 

Training Environment for Autonomous Cybersecurity Agents

A key requirement for training of autonomous agents is the availability of a training environment. Even though many such environments were created in recent years, they are mostly unusable for the cybersecurity domain. They are either too general, too simple, or too slow to be useful for learning.

The goal of this work is to overcome the deficiencies of the current environments and to prepare one, which can be used for rapid prototyping of attack strategies. To achieve this goal, a number of research problems have to be tackled, such as

1) How to model a general network communication with respect to the attacker-defender interaction? 
2) How to enable environment updates, while not invalidating agents' learned behavior?
3) How to transition from simulation to real-world deployment?
4) How to incorporate real-world offensive and defensive tools into the environment?

Advancing the state of the training environments for cybersecurity agents will strongly benefit the research of cyberdefense - from emulation of sophisticated attackers, over generation of up-to-date security datasets, to setting a new baseline for cyberdefense research.