Security Analytics Use Cases and Data
COMP90073 Security Analytics
Dr. , CIS Semester 2, 2021
COMP90073 Security Analytics © University of Melbourne 2021
Outline
• Security Analytics Use Cases
• Security Data
• Research Benchmark Datasets Overview
COMP90073 Security Analytics © University of Melbourne 2021
Security Analytics Use Cases
• Incident Investigation and Forensics • Security Monitoring
• Advanced Threat Detection
• Incident Response
• Compliance
• Fraud Analytics and Detection • Insider Threat Detection
COMP90073 Security Analytics © University of Melbourne 2021
Incident Investigation and Forensics
Image source: www.splunk.com
• Security incidents can occur without warning and can often go undetected long enough to pose a serious threat to an organization. Usually by the time security teams are aware of an issue, there’s a good chance the damage has been done. [1]
COMP90073 Security Analytics © University of Melbourne 2021
Security Monitoring
• Security monitoring enables you to analyse a continuous stream of near-real-time data for threats and other potential security issues. Data sources for monitoring include network and endpoint systems–as well as cloud devices, data centre systems and applications. [1]
https://digitalguardian.com/blog/how-build-security-operations- center-soc-peoples-processes-and-technologies
COMP90073 Security Analytics © University of Melbourne 2021
Advanced Threat Detection
• An advanced persistent threat (APT) is a set of stealthy and continuous computer-hacking processes, often orchestrated by a person or persons targeting a specific entity. APTs usually target private organizations and/or states for business or political motives. [1]
Image source: www.splunk.com
COMP90073 Security Analytics © University of Melbourne 2021
Incident Response
• Incident Response (IR) involves the monitoring and detection of security events on IT systems, and the execution of response plans to those events. IR Teams are sometimes called blue teams. Blue teams defend an organization’s infrastructure when threats are detected, whereas red teams attempt to discover weaknesses in the existing configuration of those same systems. [1]
COMP90073 Security Analytics © University of Melbourne 2021
Compliance
• In nearly all environments, there are regulatory requirements in one form or another–especially when dealing with the likes of General Data Protection Regulation (GDPR), Health Insurance Portability and Accountability Act (HIPAA), Payment Card Industry Data Security Standard (PCI DSS), (SOX) and even common guidelines that aren’t considered true compliance. [1]
COMP90073 Security Analytics © University of Melbourne 2021
Fraud Analytics and Detection
• Machine data plays a pivotal role in and is at the heart of detecting fraudulent activities in the digital age. [1]
Image source: www.splunk.com
COMP90073 Security Analytics © University of Melbourne 2021
Insider Threat Detection
Image source: www.splunk.com
• Insider threats come from current or former employees, contractors or partners who have access to the corporate network and intentionally or accidentally exfiltrate, misuse or destroy sensitive data. They often have legitimate access to networks and permission to download sensitive material, easily evading traditional security products. [1]
COMP90073 Security Analytics © University of Melbourne 2021
Outline
• Security Analytics Use Cases
• Security Data
• Research Benchmark Datasets Overview
COMP90073 Security Analytics © University of Melbourne 2021
Security Data
• Common Attributes • Network
• Endpoint
• Authentication
• Web Activity
COMP90073 Security Analytics © University of Melbourne 2021
Common Attributes
• Real-world data – Unlabelled
– A lot of attributes • Generic attributes
– Who
• e.g., user/machine/network/domain identification
– What
• e.g., process/application/file/action
– When
• e.g., time zone, timestamp
– Where
• e.g., source, destination
COMP90073 Security Analytics © University of Melbourne 2021
Common Attributes
• TCP/IP five-tuple
– Source IP address
– Source port
– Destination IP address – Destination port
– Protocol
• 1: ICMP
• 6: TCP • 17: UDP
COMP90073 Security Analytics © University of Melbourne 2021
Network
“Visibility into network traffic is critical for any security team. The priority is to see what types of traffic are entering and exiting your network. It’s critical to see the traffic that’s permitted as well as communication attempts that have been blocked.” [1]
Sample source
• Firewall traffic logs
COMP90073 Security Analytics © University of Melbourne 2021
Example: Firewall Traffic Logs
Data source: Splunk Boss of the SOC 2.0 Dataset
COMP90073 Security Analytics © University of Melbourne 2021
Endpoint
“Endpoint logs complement network visibility to give insight into malicious activities such as malware execution, an insider performing unauthorized activity or an attacker dwelling in your network.” [1]
Sample source
• Windows Event Logs
• Linux System Logs
• Linux Auditing System (Linux AuditD) • MacOS System Logs
COMP90073 Security Analytics © University of Melbourne 2021
Example: Windows Event Logs
Data source: Splunk Boss of the SOC 2.0 Dataset
COMP90073 Security Analytics © University of Melbourne 2021
Authentication
“Authentication logs can tell you when and from where users are accessing systems and applications. Since most successful attacks eventually include the use of valid credentials, this data is critical in helping to tell the difference between a valid login and an account takeover.” [1]
Sample source
• Windows Active Directory
• Local Authentication
• Identity & Access Management (IAM)
COMP90073 Security Analytics © University of Melbourne 2021
Example: Windows Active Directory Logs
Data source: Splunk Boss of the SOC 2.0 Dataset
COMP90073 Security Analytics © University of Melbourne 2021
Web Activity
“Many attacks start with a user visiting a malicious website or end with valuable data being exfiltrated to a site that the attacker controls. Visibility into who’s accessing what sites and when is critical for investigation.” [1]
Sample source
• Next generation firewall (NGFW) traffic filters logs • Web proxy logs
COMP90073 Security Analytics © University of Melbourne 2021
Example: HTTP Traffic Logs
Data source: Splunk Boss of the SOC 2.0 Dataset
COMP90073 Security Analytics © University of Melbourne 2021
Outline
• Security Analytics Use Cases
• Security Data
• Research Benchmark Datasets Overview
COMP90073 Security Analytics © University of Melbourne 2021
Research Benchmark Datasets Overview
• KDDcup99 Dataset
• NSL-KDD Dataset
• DARPA 2000 Dataset • CAIDA Dataset
COMP90073 Security Analytics © University of Melbourne 2021
KDDcup99 Dataset
• Most widely used dataset to evaluate Network based Anomaly Detection methods & systems. Attack scenarios include:
– Denial of service (DoS): An attacker attempts to prevent valid users from using a service provided by a system
– Remote to local (r2l): Attackers try to gain entrance to a victim machine without having an account on it, e.g., guessing password
– User to root (u2r): Attackers have access to a local victim machine and attempt to gain privilege of a superuser (root)
– Probing: Attackers attempt to acquire information about the target host, e.g., port scanning.
COMP90073 Security Analytics © University of Melbourne 2021
KDDcup99 Dataset
Table – Distribution of normal and attack traffic instances [2]
COMP90073 Security Analytics © University of Melbourne 2021
KDDcup99 Dataset
• Download: http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html – Snippet
– Field description http://kdd.ics.uci.edu/databases/kddcup99/kddcup.names
COMP90073 Security Analytics © University of Melbourne 2021
NSL-KDD Dataset
• Problem with KDDcup99: redundant records [3]
– 78% and 75% of the records are duplicated in the train and test set
• A new dataset consisting of selected records of KDDcup99 dataset which improves the evaluation performance
– Description: https://www.unb.ca/cic/datasets/nsl.html
– Download: https://github.com/jmnwong/NSL-KDD-Dataset
Table – Distribution of normal and attack traffic instances [2]
COMP90073 Security Analytics © University of Melbourne 2021
DARPA 2000 Dataset
• This dataset targets evaluating detection of complex attacks that contains multiple steps.
– Description&Download:https://www.ll.mit.edu/r-d/datasets/2000-darpa- intrusion-detection-scenario-specific-datasets
– Itincludesfiveattackphases: • IPSweep
• Probing
• Breaking into the system by exploiting vulnerability
• Installing DDoS software for the compromised system • Launching DDoS attack against another target
COMP90073 Security Analytics © University of Melbourne 2021
CAIDA Dataset
• CAIDA collects many different types of data and makes them available to the research community. CAIDA datasets are very specific to particular events or attacks, such as the DDoS 2007 dataset. Most of its longer traces are anonymized backbone traces without their payload.
• Description & Download: https://www.caida.org/catalog/datasets/overview/#H2279
COMP90073 Security Analytics © University of Melbourne 2021
Other Datasets
• MarkusRing,SarahWunderlich,DenizScheuring,DieterLandes, , “A Survey of Network-based Intrusion Detection Data Sets”, arXiv:1903.02460, https://arxiv.org/abs/1903.02460
COMP90073 Security Analytics © University of Melbourne 2021
Summary
• Security analytics use cases
– Explain seven common use cases
• Security data
– Explain four primary categories of data sources
– Select common attributes
– Understand the role of each data source in detecting cyber threats
• Research benchmark datasets
– Understand the primary use case for each dataset
COMP90073 Security Analytics © University of Melbourne 2021
Reference
• [1] Splunk Inc., 2021, The Essential Guide to Security 2021
• [2] M.H.Bhuyan, et al., 2017, Network Traffic Anomaly Detection and
Prevention, Springer
• [3] M. Tavallaee, E. Bagheri, W. Lu and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1-6, doi: 10.1109/CISDA.2009.5356528.
COMP90073 Security Analytics © University of Melbourne 2021