New Approaches to Making Solaris More Secure

When it comes to system (or network) security, a little paranoia is a Good Thing. Some people like to secure their networks on a system by system basis, while others are content to install their OSes with the default settings, trusting on their firewall to keep the Bad Guys at bay. Being the paranoid type, I prefer to use a multi-tiered approach to security. Judicious firewall and/or router settings help keep the Bad Guys out of the network (or at the very least, restrict what type of traffic they can employ), and tightening up the security on the individual hosts in the network will limit what damage they can do should they get past the outer defences.

Setting up firewalls and routers is beyond the scope of this article; what I do intend to do is to show you how to tighten up the security of your Solaris boxes, using a couple of scripts I wrote to help automate the process. But before we go into the process of tightening up our systems, we need to get the foundation right.

Install The OS

The first step is to install the latest possible release of Solaris your hardware will support. Later releases of Solaris are generally more secure, and better performing—not to mention, more feature rich—than the earlier ones. For this article, I’ll be using Solaris 7.

Until your system is secure, you should isolate it from any untrusted network. I.e., it should not be on any publicly accessible network until we’re finished. If this precaution isn’t taken, there’s a chance that someone could break into it, possibly installing trojan horses, thereby rendering the whole exercise futile! To copy the scripts and their associated files to the target machine, you should attach a tape drive to the new system, and restore the files from a tape you created on a trusted machine. Either that, or use a trusted LAN to transfer the files.

A golden rule of system security is “the fewer services your system is running, the fewer potential vulnerabilities it has” (a process known as “reducing the attack surface”). Therefore, it is desirable to install only those OS components you actually need. I actually install all my servers headless (the console is on TTY A): servers have no business running X in my opinion. Security risks notwithstanding, X uses up precious resources that can be put to better use. For example, on a Creator 3D equipped machine, it is not unusually to have an Xsun process using about 100 MB of virtual memory! To these ends, I select a Core OS Software Group installation (unless the machine is to be a desktop, in which case I select the Developer Software Group), although there’s a few packages I add (System Accounting, Terminal Information, and Static Utilities to name a few); the exact list of additional packages will depend on the intended use of the system. Is it destined to be a desktop system, or a server? If it’s a server, what services will it offer, for example, NFS exported filesystems, email, FTP, or some combination of these?

For ease of administration, I like to keep my OS and data on separate disks. For the same reason, I usually divide my OS disk into only two partitions: / and swap. This is the partition map for the 2 GB OS disk from one of my servers:

s0: / Rest of disk s1: swap 512 megabytes s2: overlap s3: s4: s5: s6: s7:

The root partition is large to allow for extra logging and auditing information. Swap should be sized according to your hardware, but extra swap helps prevent “denial of service” type attacks.

Make sure that you don’t select the automatic reboot option, as the first of my scripts must be run before the initial reboot.

Phase One

Phase one is immediately after Solaris has been installed, but before the initial reboot. Copy the script, called phase1, (and its associated directory of data files, phase1_files) from a trusted location to /tmp on your new machine, and run it. The first thing the script does is ask you what your domain name is, and puts your answer into /etc/defaultdomain (actually, as we have not yet rebooted, the domain name gets stored in /a/etc/defaultdomain).

The next thing the script does is set up a number of scripts that set the umask to 022 for each init state. When Solaris boots, the umask is set to its default of 777, so each file that gets created as part of the boot process has to have its permissions explicitly set if they are not to be too permissive. By setting umask to 022, we ensure that no files get accidentally created with write permission for anyone but root.

Copying Files

In one or two places in the main article, I’ve mentioned that various files get copied. Although the mechanism is very simple, I thought that I’d explain how it works here.

In the installation scripts, the following lines of code perform the copying:

FLAG=false echo " Copying files: \c" cat directory/INDEX | while read SRC DEST OWNER GROUP PERMS; do if [ $FLAG = "false" ]; then echo "$SRC\c" FLAG=true else echo ", $SRC\c" fi cp phase1_files/$SRC $DEST chown $OWNER:$GROUP $DEST chmod $PERMS $DEST done echo "."

I use a variable, FLAG, to indicate if this is the first iteration of the loop; this lets me keep the formatting nice! Each line (which represents one file) of the INDEX file is read in, and is broken in to five tokens. The tokens are the source file name, the destination file name, the file’s owner and group, and finally, the permissions for the file.

The name of the file is printed out, then it is copied to its destination, and its ownership and permissions are set as required.

To illustrate the format of the directory/INDEX file, here are a few lines from one of the INDEX files I use:

dig /opt/local/bin/dig root other 511 ssh /opt/local/bin/ssh root other 4511 top /opt/local/bin/top root sys 2511

Notice that ssh is SUID root, and top is SGID sys.

Finally, after setting a few things in /etc/system (most notably, noexec_user_stack and noexec_user_stack_log get set to 1. The former ensures that code can’t be executed from the stack (a common technique used in buffer overrun attacks), while the latter logs any attempts to do so. Unfortunately, this technique is only effective on sun4m class machines and newer.). The root user’s home directory is changed to /root, with a very restrictive set of permissions. On Solaris machine, / must be world readable; by making /root root’s home directory (and making it accessible only to root), various files can be moved out of temptation’s way.

When the script has finished running, reboot your machine, and install the latest Recommend and Security patch cluster from SunSolve. Note that because the patch installation process runs as the user nobody, the patches must be installed from a world-readable directory, so trying to install them from /root won’t work (I usually use /tmp, which has the added benefit of tidying up after itself). Once the patches have been installed, you should probably reboot your machine again, so that they can take effect, and to ensure that installing them didn’t break anything.

Phase Two

The phase2 script starts off by asking you the name of your mail server and the IP address of your default router. The script assumes that email is handled by a central mail server, so it arranges for all email to be sent to it. Don’t worry if the system you are working in is intended to be a mail server, as the configuration that gets put in place at this stage can easily be superseded by a more elaborate one later. The script then adds a couple of users (access, which in intended to be the only account that can log on to the machine across the network if the machine is destined to be a server, and backup, which is the user that tape backups are performed as. It is neither necessary nor desirable for backups to run as root, especially when multiple systems are backed up to a central server over the network, as allowing arbitrary processes to be executed by a remote root user has serious security concerns. The disk partition special files are all readable by the group sys, so the backup user is a member of that group.), and sets TCP_STRONG_ISS in /etc/default/inetinit to 2. This makes the TCP initial sequence number less predictable, by following the guidelines in RFC 1948.

Next, the default router is set up, and the loginlog is enabled. If an account has more than five consecutive failures, and /var/adm/loginlog exists, all of the failures are logged in this file. All of the files in the /etc directory hierarchy have their group write permission bit cleared (among other things, this keeps sendmail happy), and /etc/syslog.conf is modified to log syslog messages at the auth.notice level and above.

Even though most add-on programs will probably be shared to our new system using NFS, there are a number of files that although they’re not part of the OS, need them to be available even if the NFS mounted directories aren’t. For example, the ssh suite of programs, and the TCP Wrappers ones. With this in mind, the next thing the script does is create a /opt/local directory hierarchy, and copy a number of files to it. I selected /opt/local as the base directory (rather than the more traditional /usr/local), because /opt is the SVR4 place to put optional (or add-on) packages, and /opt/local emphasises the fact that the files are local to this machine, rather than /opt/somethingelse, which could be mounted from another server. I use an NFS mounted /usr/local for packages that insist on residing there.

To guard against unauthorised use of potentially privileged accounts, all the system users (a system user is defined by the script as being any user in /etc/passwd when the phase2 script starts running), with the exception of root, have their shell changed to /opt/local/bin/noshell, a do nothing and exit script. I use this path, rather than /bin/false, as it allows me to drop in a replacement program at some point in the future with additional functionality, like logging the time and the IP address from which the login attempt took place.

Next, Wietse Venema’s TCP Wrapper and Tatu Ylonen’s ssh utilities are installed. For the security conscious, these programs are a must! Part of the TCP Wrapper set up comments out every line in /etc/inet/inetd.conf. This ensures that only the services (that are controlled by inetd) you specifically enable will be able to run, preferably under the control of TCP Wrapper. It’s worth pointing out that connection attempts that are denied by the rules in /etc/hosts.deny result in an email with the host and service names, as well as the name or IP address of the remote system, getting emailed to the local security administrator.

The last part of configuring ssh involves generating the system’s private host key; on slower machines, this can take a while!

The final thing the phase2 script does is build a null client /etc/mail/sendmail.cf. A null client is one that simply forwards all mail to the named mail host. The script does this by creating /usr/lib/mail/cf/nullclient.mc, which gets processed by m4. The resultant file is copied to /etc/mail/sendmail.cf.

Wrapping Up

Once the phase2 script has finished running, you still have a number of tasks to perform manually. These are:

Disable the services you’re not planning to offer in /etc/rc?.d.
Set a password for the access user.
Enable the services in /etc/inet/inetd.conf you wish to support.

If the current machine is destined to become a desktop workstation, you’re pretty much done! However, setting up a server will require the installation and configuration of the appropriate packages, e.g., Apache, BIND, Sendmail, etc. On my own systems, I use a number scripts similar to the ones I’ve presented here to perform these installations.

Author’s Bio

Rich Teer has more than 10 years of industry experience with UNIX systems and C programming. He lives in Kelowna, BC, where he runs his own Solaris consultancy and web hosting company, Rite Online Inc. In what little spare time he has between consulting assignments, running a business, teaching himself Java, and writing, Rich enjoys spending time with his wife, Jenny, and their dog, Judge. He is currently writing a book called Solaris Systems Programming, which will be published by Addison Wesley in 2002.

Rich Teer’s Home Page