BSD DevCenter
oreilly.comSafari Books Online.Conferences.


Securing Small Networks With OpenBSD, Part 3

by Jacek Artymiak

I planned to write about pf log file analysis tools and techniques, but the mail I received after the last installment of Securing Small Networks With OpenBSD made me change my mind, because many readers want to know more about the way pf works.

So I'll leave pf log analysis until the next installment, and answer two questions: one about pf rules and one about problems with sendmail.

Why are "in" rules used to filter outbound traffic?

That question pops up quite frequently in mail I receive and in conversations with my students. The answer lies in the difference between how pf looks at packets and interfaces and how we view them. Unlike the administrator writing the rules, pf knows nothing about the routes that packets have to follow. Its only job is to check packets arriving or leaving network interfaces on the machine it runs on; all it does is check their type, flags, shape, state, origin, and destination. Inbound packets are matched against in rules and outbound packets are matched against out rules. That's the whole magic.

In order to better understand the whole process, let's have a closer look at the main steps on the route that packets sent from a machine on the internal private network have to follow in order to reach a host located somewhere on the Internet (we'll borrow the host and network addresses, and interface names from the first installment of Securing Small Networks With OpenBSD:

Figure 1. The path that an outbound packet follows to reach a host located outside the local network.

  • Machine sends a series of packets to the gateway interface located on the firewall machine. The address of that interface ( is stored in the local routing table.
  • The packets arrive at the ne1 gateway interface on the firewall machine that is running pf.
  • pf matches the inbound packets against the NAT rules and performs address translation and redirection.
  • pf matches the inbound against the in rules for the ne1 interface.
  • Packets that pass the pf tests are let into the firewall and their destination is checked against the routing table stored on the firewall. The routing table specifies which interface a packet should be sent to in order to reach its destination.
  • Packets bound for hosts outside our network arrive at the tun0 external interface. They are ready to leave, but before the firewall machine sends them on, pf kicks in once again.
  • pf matches the outbound packets against the out rules for the tun0 interface.
  • pf matches the outbound packets against the NAT rules and performs address translation and redirection.
  • Packets that pass all pf tests are sent to the network the tun0 is connected to. How they reach the destination host is no longer of interest to the firewall; the routers managing traffic on other networks will take care of it.

Figure 2. The path that an inbound packet follows to reach a host located inside the local network.

Packets arriving from the hosts located on the Internet follow a similar route, but in the opposite direction, and with other pf table sections applied to them.

Why is my /var/spool/clientmqueue directory filled with strange files?

Why is sendmail being run at regular intervals after I turn it off in rc.conf?

What are these strange noises my firewall is making at regular intervals?

Why is root mail filled with messages from cron?

Why is the disk on the firewall filling up very quickly?

Believe it or not, all of these questions often have the same answer: you are probably having problems with scripts run by the cron daemon (man cron, man 1 crontab, and man 5 crontab explain how cron works in detail).

There are three such scripts installed by default in OpenBSD: /etc/daily, /etc/weekly, and /etc/monthly. They perform daily, weekly, and monthly security checks and basic system housekeeping, and send the results to user root.

When cron runs a script, it will send the script's output to the user specified in the MAILTO variable in the crontab from which the script was run, or the owner of that crontab. That's why the root mail account is being filled with "strange" mail.

The cron daemon uses sendmail to deliver messages. It doesn't matter that you set the sendmail_flags=NO flag in /etc/rc.conf, such settings only tell the system not to start the sendmail daemon -- the sendmail itself can still be run from the command line, a script, or another program. That is why top shows sendmail on the list of active processes from time to time.

As long as sendmail is being run by cron after running one of the default scripts, or one of your own scripts, you have nothing to worry about. The strange noises your machine makes at regular intervals are the sounds of the systems plowing through the filesystem.

When the firewall machine is set up in the way described in Part 1 or 2 of Securing Small Networks With OpenBSD, sendmail cannot deliver messages sent by cron and fills /var/spool/clientmqueue with strangely named files. These are undelivered messages and their headers. You can read them using the less command, followed by the name of the file you want to read. Each message/header pair has a different prefix followed by an identical number. But since that way of reading mail is not very convenient, you can make your life easier in two ways: either relax the firewall rules, which will be the subject of a future article, or modify the daily script and the root crontab in the following way:

  • In /etc/daily change:

    sh /etc/security 2>&1 > $OUT
    if [ -s $OUT ]; then
    mail -s "$host daily insecurity output" root < $OUT


    sh /etc/security 2>&1 > $OUT.TMP
    if [ -s $OUT.TMP ]; then
    echo "$host daily insecurity output" > $OUT
    cat $OUT.TMP >> $OUT
    rm $OUT.TMP
    /usr/libexec/mail.local root > $OUT
  • Edit the crontab for user root with:

    $ crontab u root e

    and change

    30 1 * * * /bin/sh /etc/daily 2>&1 | tee /var/log/daily.out | mail -s "`/bin/hostname` daily output" root

    30 3 * * 6 /bin/sh /etc/weekly 2>&1 | tee /var/log/weekly.out | mail -s "`/bin/hostname` weekly output" root

    30 5 1 * * /bin/sh /etc/monthly 2>&1 | tee /var/log/monthly.out | mail -s "`/bin/hostname` monthly output" root


    30 1 * * * /bin/sh /etc/daily 2>&1 | tee /var/log/daily.out | /usr/libexec/mail.local root

    30 3 * * 6 /bin/sh /etc/weekly 2>&1 | tee /var/log/weekly.out | /usr/libexec/mail.local root

    30 5 1 * * /bin/sh /etc/monthly 2>&1 | tee /var/log/monthly.out | /usr/libexec/mail.local root

These changes replace the mail command (which calls sendmail) with mail.local, which delivers mail straight to the local user's mailbox, bypassing sendmail.

As for the undelivered files from /var/spool/clientmqueue, you can remove them with:

$ rm /var/spool/clientmqueue/*

If the system complains about the length of the argument list, as can happen when this directory is filed with thousands of files, you should use ls, and wait for it to display all files. When the command prompt is visible again, look at filenames and try to spot identical prefixes you can use to construct a wildcard, such as:



Start with qfg3MM and you can use:

$ rm qfg3MM*

to remove them.

After removing several smaller groups of files, you can try to shorten the wildcard to make it encompass more files:

$ rm qfg*

If you still get errors, use ls, construct a longer wildcard, and repeat the process as many times as necessary to clean the /var/spool/clientmqueue directory. After you remove all files, make sure to monitor this directory for new, undelivered messages.

Alternately, you can look into xargs to help with the long command lines.

Until next time.

Jacek Artymiak started his adventure with computers in 1986 with Sinclair ZX Spectrum. He's been using various commercial and Open Source Unix systems since 1991. Today, Jacek runs, writes and teaches about Open Source software and security, and tries to make things happen.

Return to the BSD DevCenter.

Sponsored by: