This paper successfully tackles the problem of processing a vast amount of security related data for the task of network intrusion detection. It employs Apache Spark, as a big data processing tool, for processing a large size of network traffic data. Also, we propose a hybrid scheme that combines the advantages of deep network and machine learning methods. Initially, stacked autoencoder network is used for latent feature extraction, which is followed by several classification-based intrusion detection methods, such as support vector machine, random forest, decision trees, and naive Bayes which are used for fast and efficient detection of intrusion in massive network traffic data. A real time UNB ISCX 2012 dataset is used to validate our proposed method and the performance is evaluated in terms of accuracy, f-measure, sensitivity, precision and time.

Keywords