SC21 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Detecting Network Intrusion Anomalies through Egonet-Based Data Mining with Apache Spark


Authors: Nathan Paik and Se Ho Kwak (Pomona College) and Enyue Lu (Salisbury University)

Abstract: Network intrusions often contain dangerous breaches to network security systems and their data. We design an anomaly detection system to identify network intrusions. Our proposed detection method is inspired by the use of egonets in the oddball algorithm but differs by the extracted features and the anomaly classification procedure. The detection process follows the generalized design: create a k-nearest-neighbors graph from a network dataset; extract each node’s egonet’s edge weights, number of edges, and total eigenvector centrality sum; compare each node’s egonet’s features through pairwise comparisons; and define a median “truth” line from the comparison and label nodes as anomalous based on their distance from the line. We have achieved an anomaly detection accuracy score of up to 92.9% with the eigenvector centrality score vs. edge weight feature comparison. We parallelize our algorithm by implementing Resilient Distributed Datasets in Apache Spark.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF


Back to Poster Archive Listing