Apart from using traditional security solutions in software systems such as firewalls and access control mechanisms, utilizing intrusion detection systems are also necessary. Intrusion detection is a process in which a set of methods are used to detect malicious activities against the victims. Many techniques for detecting potential intrusions in software systems have already been introduced. One of the most important techniques for intrusion detection based on machine learning is using Hidden Markov Models -HMM-. In recent decades, many research communities have been working towards HMM-based intrusion detection. Therefore, a large volume of research works has been published and hence, various research areas have emerged in this field. However, until now, there has been no systematic and up-to-date review of research works within the field. This paper aims to survey the research in this field and provide open problems and challenges based on the analysis of advantages, limitations, types of architectural models, and applications of current techniques. Six various architecture models for intrusion detection purposes are proposed in the literature. We compare these models based on performance criteria in order to select an appropriate type for a specific application. The results show that HMM-based intrusion detection techniques have six main advantages- precise intrusion detection, ability to detect new and unknown intrusions, prediction of the intruder’s potential next steps, usage in real-time applications by processing data streams on-the-fly, usage of heterogeneous data sources as input, and visual representation of acquired knowledge relative to the other techniques of machine learning.

Keywords