Home > Security > Full Story

Intrusion Detection Systems: Beyond the first line of defense
By Rakesh Raghudharan

Firewalls form the first line of defence against malicious hackers. But what if a hacker is able to get past your network firewall. This is where Intrusion Detection Systems come into the picture

Most corporate networks are protected from malicious hackers by firewalls. Firewalls form the first line of defense against anyone having a go at your precious data, but firewalls can rarely identify types of attacks, or attacks on allowed services. This calls for something more than firewalls.

Intrusion Detection Systems (IDS) allow administrators to detect and respond to these attacks. An Intrusion Detection System tries to detect and alert on attempted intrusions into a system or network, where an intrusion is considered to be any unauthorized or unwanted activity on that system or network.

The full efficiency of IDS can be exploited only if it is monitored. Monitored Intrusion Detection Systems offer real time detection and response to attacks including dynamic blocking, complaints to ISPs, report generation etc.

Isn't that what a firewall does?
A better way to understand IDS would be to take your house as an analogy. The locks on your windows and doors stop strangers from gaining access to your house. These are your firewalls. Intrusion detection systems are a combination of early warning and alarm systems. Your intrusion detection system is provided by your alarm system. When someone attempts to force entry into your house, your alarm will sound to scare off the intruder (a "reactive" IDS), or it might make a silent telephone call to the local police station (a "passive" IDS).

A more advanced form of domestic intrusion detection system is your alert neighbor, who, knowing the normal pattern of people walking through your street, spots someone loitering outside, gathering information about the normal activity in your street. This would be an example of detection of "anomalous" activity.

The primary role of a firewall is to limit the access between networks. Firewalls are designed to filter "normal" network traffic, based on attributes as source and destination addresses, port numbers etc. Even the modern "Stateful Inspection Firewalls" often do not correctly handle maliciously crafted "incorrect" network traffic. There is normally the option for the firewall to raise an alert when some prohibited traffic tries to pass through. In comparison, a network based IDS (NIDS) is aware of what constitutes legal and illegal network packets and can raise alerts when such traffic is detected.

Types of IDS

Intrusion detection can be categorized in 3 different ways:

Network- and host-based systems: Network-based systems examine the individual packets flowing through a network. They are able to understand all the different flags and options that exit within a network packet unlike firewalls, which typically looks at the IP addresses, ports and ICMP types. A NIDS can therefore detect maliciously crafted packets that are designed to be overlooked by a firewall's relatively simplistic filtering rules. NIDS are also able to look at the "payload" within a packet, ie see which particular Web server program is being accessed and with what options, and to raise alerts when an attacker tries to exploit a bug in such a code. Most of the firewalls are unable to do this.

Host-based Intrusion Detection Systems are concerned with what is happening on each individual computer or "host". They are able to detect such things as repeated failed access attempts or changes to critical system files.

Misuse and anomaly detection systems: Misuse detection within network-based IDS involves checking for illegal types of network traffic, eg combining options within a network packet that should never legitimately occur. Misuse detection by host-based IDS would include attempts by a user to execute programs for which they have no legitimate need.

Detection of anomalous activity relies on the system knowing what is "regular" network traffic, and thus what isn't. Anomalous traffic to a host based IDS might be interactive accesses outside of normal office hours. An example of anomalous traffic on network-based IDS is repeated attempted access by one remote machine to many diverse services on one or more internal systems, all in quick succession, something very similar to a port scan of these systems. Many modern systems use a combination of both misuse and anomalous detection engines.

Passive vs reactive: A third way of categorizing Intrusion Detection Systems is by their passive or reactive nature. Passive systems simply detect the potential security breach, log the information and raise an alert. Reactive systems on the other hand are designed to respond to the illegal activity, for example by logging off a user or by reprogramming the firewall to disallow network traffic from a suspected hostile source.

The drawback of using a reactive IDS may be scenario wherein an attacker crafts rouge network traffic aimed at a company's Internet mail system. The traffic is crafted so that it appears to come from the Internet Service Provider's mail system. The network based reactive IDS detects this anomalous traffic and reprograms company's firewall to disallow all traffic from that system. The company is now unable to receive any email via the ISP.

So skilled staff plays a crucial role for the success of any IDS. A properly trained Intrusion detection analyst should be able to identify "faked" traffic or at least he or she should liaise with the ISP to determine the source of the problem.

Simple approach for setting up an Intrusion Detection System
There are a variety of different probes hackers will attempt. The first type, port scans, is one of the most common one. The scan can be on a specific target, or used to scan entire IP ranges, often chosen at random. This is the most popular information gathering method used by hackers today as it identifies what ports and services are open.

To detect these scans, a system can be build that e-mails an alert to the system administrator whenever someone connects to a predetermined port. This can be done by identifying three to five of the most commonly scanned ports. Then select two to three systems to listen on these ports. When an intruder scans the network, he will most likely hit the systems listening on these ports. When these ports are scanned, the systems logs the attempt, executes various predetermined actions and then email an alert to a point of contact. The end result is that the administrator receives an e-mail for each of the port scanned. For example if you have 3 systems, each listening on 4 ports, then you may get up to 12 emails from a single network port scan.

The commonly scanned ports are IMAP (port 143), SMB (port 139), Login port 513), and HTTP (port 80).

How does IDS fit into a company's security strategy?

Intrusion detection systems should be seen as an important layer in the company's "defense in depth" strategy. However, before deploying an intrusion detection system certain prerequisites need to be satisfied, they are:

l A well-defined high-level security policy covering what is and isn't permitted on company's systems and networks. This would include such things as the password policy, which Internet facilities staff may access, etc.

• Low level platform-specific policies detailing how the high-level strategy is to be implemented. For instance, how to configure password management subsystems on your NT and UNIX servers, the configuration details for company's Internet firewalls, etc.
• Documented (and tested) procedures for staff to follow, should a security incident be detected. For instance, the Help Desk receives numerous calls one morning from staff complaining that their accounts have been disabled, and the system logs show repeated failed login attempts to all these systems during the previous night.
• Regular audits to confirm that the policies have been enacted, and that the defenses are adequate for the level of risk you are exposed to. For instance, performing regular network scans from outside, and inside, the organization's firewalls to determine what ports are open and how much information the firewall and routers leak.
• Available staff skilled in the operation and monitoring of both the built-in and 3rd party security tools installed on the servers and network devices. For instance, if the staff currently doesn't have the time to check the firewall and router logs, IDS alerts are unlikely to be acted upon in a timely manner.

The above should be considered pre-requisites, and are essential for getting the best possible results from the IDS.

Deployment of Network Intrusion Detection System

Care should be taken when the deployment of the NIDS is done. Consider the example network in Figure 1.

Placing a NIDS on the outside of the external firewall will give an early warning advantage, as it should enable the administrator to detect the port scans that typically indicate the start of hacker activity. However, not all scans will be followed by an actual attack, as the hacker may determine that the network currently has no weaknesses that they can exploit. This could lead to large number of alerts that do not require attention. One common yet dangerous effect of this is that the staff may lose faith in the IDS and start ignoring alerts. External firewall can be used to provide alerts for the traffic that it has denied. By placing NIDS inside the DMZ (De-Militarized Zone, a part of the network that is neither "inside" nor "outside" the corporate entity) the advantage that could be taken is that the tailoring of NIDS attack signature database can be done to consider only those attacks that are applicable to the systems in the DMZ; at the same time the firewall will have blocked all other traffic.

A NIDS located within the HR network would be able to monitor all the traffic to and from and within that network as unauthorized traffic has to be allowed to the HR department. Such a system would not have to be as powerful as, for example, a NIDS located outside the external firewall, as both the volume and type of traffic it needs to monitor are greatly reduced.

NIDS on a Switched network

NIDS works by looking at all the network traffic flowing past. Simply connecting NIDS to a normal port on a switched network will be of little use. The very nature of switched networks means that only traffic destined for a particular device is sent to that device. There are two ways to approach this situation:

• To connect the NIDS to a "spanning" port on an appropriate switch. A spanning port is one, which is programmed to receive copies of all the traffic flowing through the switch, or a selected subset of it. A problem with this approach is that the aggregate bandwidth presented to the NIDS could be overwhelming.
• To use a network tap device. This allows the one-way pick-up of packets from a connection, but is only suitable for intercepting traffic to a single network device.

Host-based IDS

As with all security measures for individual PCs, evaluate each system for risk and deploy host based IDS software appropriately. If the main concern is attack from the Internet, concentrate host-based defenses in the DMZ.

Conclusion

Intrusion detection systems add an early warning capability to your defenses, alerting you to any type of suspicious activity that typically occurs before and during an attack. Since most cannot stop an attack, intrusion detection systems should not be considered an alternative to traditional good security practices. There is no substitute for a carefully thought out corporate security policy, backed up by effective security procedures which are carried out by skilled staff using the necessary tools. Instead, intrusion detection systems should be viewed as an additional tool in the continuing battle against hackers and crackers.

Rakesh Raghudharan is a 2nd yr PGDTM student at Symbiosis Institute of Telecom Management (SITM), Pune and can be reached at

rakeshraghudharan@rediffmail.com

Network configuration

The diagram below shows a common network configuration deploying the Network IDS

1. Network IDS 1 would monitor the traffic that has been permitted through the Internet-connected firewall. It can thus look for rogue Web and e-mail messages.

2. Network IDS 2 would monitor all traffic passing into and out of the internal network.

3. The thirds NIDS system would specifically monitor all traffic into and out of the Human Resources network.

4. DMZ and other individual systems would be running host-based intrusion detection software based on their risk profile.

IDS tools

• Axent "Intruder Alert" and "Net Prowler"
• Cisco "NetRanger"
• Computer Associates "E-Trust" (formerly SessionWall-3)
• ISS "RealSecure"
• Martin Roesch's "SNORT"
• US Navy Surface Warfare Centre "SHADOW"

What Intrusion detection systems can and can't do

They can….

1. Increase the overall security of a company network environment.

2. Monitor the network traffic inside your firewalls.

3. Examine the contents of network messages, thus detecting for example "buffer overflow" types of attacks.

4. Recognize and report changes to files and directories.

5. Detect irregular access times.

They cannot…..

1. Replace skilled staff.

2. Cannot act as a full proof protection to cure the entire company security concerns.

3. Cannot compensate foe weakness in network protocols, feeble identification and authentication mechanisms.