Network Traffic Data Collection for Machine Learning Analysis

Network traffic has increased substantially due to the introduction of advanced network-enabled applications and devices. The introduction of software defined networks (SDNs) and machine learning (ML) has empowered optimizing network operations and network traffic monitoring, resulting in improved complex traffic operations and security with faster malicious intention detections. This paper focuses on network traffic data collection systems, and the data is evaluated using a survey of ML algorithms, depending on the data type (tabular or image). Adhering to system architecture best practices including a decoupled design to integrate with existing network monitoring infrastructures and cybersecurity standards; and online and offline data collection via packet capture (PCAP) standards. For packet based network traffic data analysis, we convert captured data into images and feed into a convolutional neural network to classify the data based on requirements. For statistical based network traffic data analysis, we apply feature engineering on tabular data and feed into various ML systems to classify based on requirements. Finally, We show that the same ML algorithm outperforms publicly available datasets using our collection method.