Every OS comes with some security vulnerabilities. We look at how Windows, Unix and other operating systems can be hardened to reduce vulnerabilities

Unless you are specifically running a DNS server, in.named can return DnS information for intruders to make use in launching DNS-type attacks

If you are a MIS manager looking after corporate servers, firewalls, VPNs and databases, it is critically important that you know the fundamentals of OS hardening, especially in the light of recent exploits that turn even the smallest loopholes into open craters.

OS hardening is the black art of ensuring that all known OS vulnerabilities are plugged, and that the OS is monitored continuously.

Cleaning up Windows

Windows NT 4 is still widely used in many corporate servers where migration to Windows 2000 is often slow, compared to smaller businesses which tend to adopt the latest Windows versions as they roll out. Therefore, with the many vulnerabilities well-documented in NT 4, it is important that MIS managers ensure that all known problems are fixed quickly.

Even Windows 2000 is not immune, and it is even more complex than NT 4. When XP comes along, you can imagine that it will get even more complex, and therefore potentially vulnerable as well.

Windows 2000 users should install the OS on NTFS (NT File System) partitions only, never FAT (File Allocation Table). Also, if you are setting up a server, ensure that you start to partition it for NTFS, and not install to a FAT partition and then convert to NTFS. This is because Windows 2000 will not apply the default ACLs (access control lists), and therefore will not be able to help you with additional controls.

If this server only runs your firewall, VPN, demilitarised zone (DMZ), ensure that terminal, remote install, file, print and other unnecessary network services are not installed during the initial installation process. Since you would not generally offer certificate enrolment to other Internet users, you should not load certificate services either.

If you should set up certificate services, be sure to isolate that and run such a service from an isolated internal network within a physically and digitally secured parameter environment.

If your machine is used for SMTP or Web services, disable Client for Microsoft Networks by unchecking it, but do not uninstall. This is because the RPC (Remote Procedure Call) Locator Service for authentication will work only when the Microsoft Networking Client is installed, and if it is uninstalled, SMTP or IIS will not start.

Since Windows 2000 Server is really meant as a server only application, you may also want to manually configure DHCP and DNS, rather than allow DHCP to automatically configure IP addresses and DNS information. You can do this under the IP Protocol Properties. Under the Options of IP Protocol Properties, you can configure TCP/IP filtering as required.

For example, you may want to add inbound ports you want this Windows 2000 server to accept, as well as configure allow access (rights for specific users to the server from the network), assign individual (separate) administrator accounts, and rename the Administrator account to another (non-default) name.

Once you renamed the Administrator account to another name, set up a "dummy" Administrator account with zero privileges. This will log intruder entries as they try to sniff your network.

Your Windows 2000 Server, running specific Web, mail or DNS services, should also be configured so that it does not participate in a domain, that is, it should be installed into a non-existent workgroup.

Think of all Internet servers as DMZ devices, and they should not participate in your active directory or workgroups for your Intranet.

The sad thing about DNS

DNS running on a Windows 2000 server is also somewhat limited, and may not offer what full-strength DNS software can deliver. For example, under Windows 2000, even though you can check "Only allow access from secondaries included on Notify List", there is not much control over the restriction of query requests, a fairly common tactic to sniff out DNS information.

You may want to manually use BIND (Berkeley Internet Name Daemon)

from ISC (www.isc.org/products/BIND), through a command line interface. It is not pretty, but it offers more DNS protection than Windows 2000 can. The current version is 9.1.3, a maintenance release, but there is also a 9.2 b1 version for testing. ISC advises that you should not run BIND prior to version 8.2.3, because prior versions are susceptible to DNS attacks.

The road well-traveled

Unix, in its many variants, such as Solaris, AIX, HP-UX, IRIX, BSDi, OpenBSD, FreeBSD, and even Mac OS X, are all derived from BSD (Berkeley Software Distribution) and AT&T System V.

Linux is interesting because it is not derived from any Unix version, but has been developed by Linus Torvalds to act like Unix.

Unix and related variants are more secure than Windows servers, due in part to their ancestry (read "old"). Therefore, many bugs have been ironed out. But that is not to say that Unix servers are secure out of the box. They are definitely not. In fact, there is a fair bit of work to be done prior to offering Internet services should you run Unix.

First, you want to set up separate partitions for SWAP and /tmp, and install the OS against "out of disk space" attacks, which is similar to denial of service (DoS). This is where intruders create large volumes of logs or upload large files through FTP or mail into your file system.

That is why you should partition your file system with physical partitions. In the case of running a mail server, you can designate /var/spool/mail on a separate disk partition or even a disk array. Other directories such as /usr/local can be mounted as read-only.

For Unix systems, inetd (or Internet daemon) is a process which invokes at boot time and accesses the /etc/inetd.conf file to activate specific services. If you are running a DMZ or firewall server, you should disable most of the unnecessary services (similar to what you should do with a Windows server), including Bind (in.named), Finger (in.fingerd), Echo and others.

Unless you are specifically running a DNS server (which you should not do on the same machine you run e-mail from), in.named can return DNS information for intruders to make use of in launching DNS-type attacks.

If you are running Bind, remember never to use Bind 4, which is so full of holes that it almost guarantees problems from the start. When running BIND, make sure that you restrict zone transfers (one of the most common intrusion attempts) to specific secondaries in your primaries, by using ACLs to allow or deny transfers.

Likewise, in.fingerd can return user information, or validate the existence of specific users, and cause mail servers to be bombarded with spam mail or mail relay attacks.

Tight wraps around TCP

Wietse Venema-a well-known Unix programmer, author and software tools developer-created tcp_wrappers, which allow you to define access control to various services based on criteria such as username, IP address or DNS domain. Tcp_wrappers, has two main files, /etc/hosts.allow, and /etc/hosts.deny.

Since access is granted or denied on the first matching rule (hosts.allow first, then hosts.deny), you must check the rules properly for known and unknown wildcards. Otherwise, the rules will break. Most GNU/Linux and BSD distributions have tcp_wrappers installed and configured by default. For other distributions, download at ftp://ftp.porcupine.org/pub/security/index.html.

Put the fix on sendmail

Sendmail is the default MTA (mail transfer agent) for most Unix or Unix-like distributions. As with most Unix applications, there have been versions that were found to have exploits. So with sendmail, run versions greater than 8.9.3. Sendmail 8.11.4 is the current stable release.

To improve the security of your sendmail agent, you can turn off SMTP VRFY (verify) and EXPN (expand alias) commands so that intruders will find it harder to retrieve information about your mail server. The example command would be:

define('confPRIVACY_FLAGS', 'novrfy,noexpn')dn1

You can also set authwarnings (which adds X-Authentication-Warning headers to warn possible mail spoofs), needmailhelo (requiring sender daemon to issue SMTP HELO before sending email), needexpnhelo and needvrfyhelo (requiring sender daemon to issue SMTP HELO before allowing EXPN and VRFY usage), noreceipts (disabling notification of delivery and read receipts, which spam senders often use), and restrictmailq (preventing users to view contents of a mail queue), and many others.

Sendmail today also has standard features which allow you to reject unauthorized mail relays, where spammers make use of your mail server to relay mail to millions of users downstream (which is bad for your reputation obviously). Sendmail also allows you to deny access from known spam sites (you can manually update specific ones). The example from Sendmail.org is:

Kspammers hash /etc/spammers

Scheck_relay

R$+ $| $+ $: $(spammers $1 $: OK $)

ROK $@ OK

R$+ $#error $: 521 $1

If you prefer an alternative mail agent, you can consider PostFix (www.postfix.org), which is also written by Venema. As with sendmail, you can also use POP before SMTP authentication to allow remote users to send mail through your mail server with a limited time POP authentication.

It also features patches and add-ons for various additional functionality, including content filtering, antivirus scanning (that works in conjunction with third-party or commercial antivirus applications), Web configuration, and so on.

Currently, PostFix is available in source, which you can compile for your Unix environment. There are also ported binaries for OpenBSD, RedHat Linux, Debian Linux and HP-UX.

Whither Mac?

If you are a Mac user, you have to ask yourself one question: OS 9 or OS X? If you are still running OS 9 for your server, you may find that the obscurity of OS 9 is enough security for many. Since the majority of problems found out there target specifically Windows and Unix distributions, OS 9, being a unique and proprietary OS, does not fall prey to these attacks.

Nonetheless, some basic advice is necessary. For example, you should turn off Personal Web Sharing (PWS), since PWS is a poor cousin to a full-fledged Web server product such as Apache, or better still, WebStar.

You should turn off all unnecessary TCP/IP related extensions within your server other than those necessary for running your server, whether it be a mail server, a Web server, or a QuickTime streaming server. Because OS 9 does not offer true multi-tasking like Unix can, you should create a redundant array of inexpensive Macs (RAIM) offering round-robin or mainstream load-balancing and failover.

If you happen to run Tenon's (www.tenon.com) MachTen Unix running on OS 9, be aware that since MachTen runs as an application on top of OS 9, it acts a bit differently from a hardware platform Unix. In this instance, the full gamut of Unix security and OS 9 security should be catered for.

If you are moving over to OS X, the game is similar to Unix. This is because OS X is really a dressed-up BSD Unix with an Aqua GUI.

However, OS X hides many of the administrator-level functions from users, and if not configured correctly, cannot run effectively as an Internet server. For OS X, you would have to apply the same kind of hardening techniques as you would to any Unix operating system. However, Apple did a good job at hiding the SU account, since the "administrator" account is really not quite the same as the real SU or root account, which has to be set from the terminal, or by "resetting the password" during a second-run installation.

From the medieval to the space odyssey

From the ancient times where knights and soldiers wore different levels of armour (and had to contend with different performance and speed constraints as you move up in protection), likewise, we face the same hassle and considerations as we move into the end of 2001.

From the point where we unpack the hardware from the boxes and cartons, we have to decide how best to secure that hardware, even before we set up services for our users and customers.

Operating systems may shift-shape as we go along, but with every new version upgrade, you can be sure new vulnerabilities and exploits will be unveiled often with the "help" of hackers and the game of MIS professionals scurrying to close every exploit will be afoot again.

Seamus Phan is research director at KnowledgeLabs News Center (www.knowledgelabs.net), an independent technology news bureau. He can be reached at seamus_phan@mfasia.com.sg

<<