Edinburgh Napier University

  There are a multitude of threats now faced in computer networks such as viruses, worms, trojans, attempted user privilege gain, data stealing and denial of service. As a first line of defence, firewalls can be used to prevent threats from breaching a network. Although effective, threats can inevitably find loopholes to overcome a firewall. As a second line of defence,security systems,such as malicious software scanners may be put in place to detect a threat inside the network. However, such security systems cannot necessary detect all known threats, therefore a last line of defence comes in the form of logging a threat using Intrusion Detection Systems (Buchanan, 2009, p. 43).Being the last line of defence, it is vital that IDSs are up to an efficient standard in detecting threats. Researchers have proposed methodologies for the evaluation of IDSs but, currently, no widely agreed upon standard exists (Mell, Hu, Lippmann, Haines, & Zissman, 2003, p. 1).Many different categories of IDSs are available, including host-based IDS(HIDS), network-based IDS(NIDS)and distributed-based IDS(DIDS).Attempting to evaluate these different categories of IDSs using a standard accepted methodology allows for accurate benchmarking of results. This thesis reviews four existing methodologies and concludes that the most important aspects in an effective evaluation of IDSs must include realistic attack and background traffic, ease of automation and meaningful metrics of evaluation.A prototype framework is proposed which is capable of generating realistic attacks including surveillance/probing, user privilege gain, malicious software and denial of service. The framework also has the capability of background traffic generation using static network data sets. The detection metrics of efficiency, effectiveness and packet loss are defined along with resource utilisation metrics in the form of CPU utilisation and memory usage. A GUI developed in Microsoft .NETC# achieves automation of sending attack and background traffic,along with the generation of detection metrics from the datalogged by the IDS under evaluation.Using a virtual networking environment, the framework is evaluated against the NIDS Snort to show the capabilities of the implementation. Mono was used to run the .NET application in a Linux environment. The results showed that, whilst the NIDS is highly effective in the detection of attacks(true-positives), its main weakness is the dropping of network packets at higher CPU utilisations due to high traffic volume. At around 80Mbps playback volumes of background traffic and above, it was found that Snort would begin to drop packets. Furthermore, it was also found that the NIDS is not very efficient as it tends to raise a lot of alerts even when there are no attacks (false-positives).The conclusion drawn in this thesis is that the framework is capable of carrying out an evaluation of an NIDS. However, several limitations to the current framework are also identified. One of the key limitations is that there is a need for controlled aggregation of network traffic in this framework so that attack and background traffic can be more realistically mixed together. Furthermore, the thesis shows that more research is required in the area of background traffic generation. Although the framework is capable of generating traffic using state data sets, a more ideal solution would be an implementation in which allows the user to select certain “profiles” of network traffic. This would serve the purpose of better reflecting the network environment in which the IDS will be deployed on.