Linux DevCenter    
 Published on Linux DevCenter (http://www.linuxdevcenter.com/)
 See this if you're having trouble printing code examples


Tools of the Trade: Part 2

by Carl Constantine
06/29/2001

Welcome back to our continuing discussion of the various tools of the Linux trade.

This article is the second of a three-part series that takes a look at some of the common tools you can use on your own systems to spot holes, look for potential problems, and then take steps to tighten your grip on the system.

Last time, I took you through a brief introduction to "honey pots," Ethereal, and the venerable nmap. This time, we'll take a look at a few more common tools, namely tcpdump and Tripwire.

tcpdump

Tcpdump is a network traffic analysis tool originally created by the Network Research Group at Lawrence Berkley National Lab. As the name implies, tcpdump allows you to "dump" TCP traffic to screen or file for later analysis. Actually, tcpdump also serves as a back-end program to many other network analysis tools such as snort and shadow. The underlying traffic capture library, libcap, is also used in other tools such as Ethereal (which we discussed last time), tcptrace, and many others. You can find out more details on these tools from the tcpdump web site. Tcpdump comes with most Linux distributions by default so you don't have to grab it yourself.

Like many other tools, tcpdump can only be used by the root user. There are many other tools, including some commercial tools, that provide slightly different or more elegant output than tcpdump. However, tcpdump is a good raw tool that can help you understand other tools and your network.

By default, tcpdump reads all the traffic from the default network interface (usually eth0M) and spews all the output to the console. For many reasons, primarily the data whips up the screen at a rather uncontrollable rate on a busy network; this is probably not always the behavior you want or need. Thus, tcpdump includes many command options to change the behavior into something more manageable.

Let's take a look at a typical packet you might capture using tcpdump. This output was captured without any command-line options given to tcpdump.

13:37:11.950966 Mallard.36872 > archive.progeny.com.www: . ack 1259760 win 376
48 <nop,nop,timestamp 249582195 600468459,nop,nop,sack sack 1 {1261208:1280032
} > (DF)

This packet is a download session from a web server. How do I know that? Well a little experience for one thing, and I set it up that way. But let me break down the packet into more detail for you.


Diagram.

Notice the destination port on archive.progeny.com is www, port 80. Therefore, this is a web session. Notice that the source and destination addresses are resolved. "Mallard" is the name of my machine. You can restrict the output to show IP addresses and numbers instead of the resolved host name (use the -n option) or you can not show some things such as the time stamp (use the -t option).

TCP flags

There are several TCP flags you might encounter when using tcpdump. They are s, ack, f, r, p, urg, and . (period). I'll describe them briefly here.

TCP Flag Flag in tcpdump Flag Meaning
SYN s Syn packet, a session establishment request. The first part of any TCP connection.
ACK ack Ack packet, used to acknowledge the receipt of data from the sender. May appear in conjunction with other flags.
FIN f Finish flag, used to indicate the sender's intention to terminate the connection to the receiving host.
RESET r Indicates the sender's intention to immediately abort the existing connection.
PUSH p Signals the immediate push of data from the sending host to the receiving host. For interactive applications such as telnet, the main issue is the quickest response time, which this "push" flag signals.
URGENT urg Urgent data should take precedence over other data. For example, a Ctrl-C to terminate a FTP download.
Placeholder . If the connection does not have a syn, finish, reset, or push flag set, this placefolder flag will be found after the destination port. Note that it also appears in conjunction with the ack flag.

Understanding the information provided by tcpdump takes a bit of time and practice. It does help to have a good TCP reference book such as TCP/IP Illustrated, Volume 1 by Dr. Richard Stevens.

Another session in tcpdump

Comment on this articleDo you have any additional tips concerning how to use tcpdump and Tripwire to protect your Linux server.
Post your comments

Let's look at something with more meat. Remember last time I showed a TCP session using Ethereal. Well, let's look at a similar session in tcpdump. For the purposes of this illustration, I've cut out a bunch of extraneous data.

For this particular packet dump, I used the following command: tcpdump -l -vvv -x -X > tcpdat & tail -f tcpdat. This command makes tcpdump give both Hex and ASCII output in a very verbose mode.

15:55:37.842404 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 107:110(3) ack 88 win 5840 <nop,nop,timestamp 250412784 526505810>
Telnet:

0x0000  fffd 01                 DO ECHO
(DF) [tos 0x10]  (ttl 64, id 45160, len 55)
0x0000  4510 0037 b068 4000 4006 05e1 c0a8 0119  E..7.h@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4eec 744a ed10  .....3..u$N.tJ..
0x0020  8018 16d0 2035 0000 0101 080a 0eec fef0  .....5..........
0x0030  1f61 d752 fffd 01                        .a.R...
15:55:37.843792 192.168.1.254.telnet > Mallard.36915: P [tcp sum ok] 88:107(19) ack 110 win 32120  <nop,nop,timestamp 526505810 250412784> (DF) (ttl 64, id 6922, len 71)
0x0000  4500 0047 1b0a 4000 4006 9b3f c0a8 01fe  E..G..@.@..?....
0x0010  c0a8 0119 0017 9033 744a ed10 7524 4eef  .......3tJ..u$N.
0x0020  8018 7d78 2b8d 0000 0101 080a 1f61 d752  ..}x+........a.R
0x0030  0eec fef0 0d0a 6c6f 6361 6c68 6f73 7420  ......localhost.
0x0040  6c6f 6769 6e3a 20                        login:.

OK. We've connected to the remote system. Notice the last line. The fact that we're waiting for a login ID is clear. Now, understand that with telnet, every keystroke is sent to the remote system as a separate packet. With that in mind, look at the very last character of the next few packets. Note: some packets have been removed as they are echo packets sent from the server back to the client system.

15:55:37.879456 Mallard.36915 > 192.168.1.254.telnet: . [tcp sum ok] 110:110(0) ack 107 win 5840  <nop,nop,timestamp 250412788 526505810> (DF) [tos 0x10]  (ttl 64, id 45161, len 52)
0x0000   4510 0034 b069 4000 4006 05e3 c0a8 0119  E..4.i@.@.......
0x0010   c0a8 01fe 9033 0017 7524 4eef 744a ed23  .....3..u$N.tJ.#
0x0020   8010 16d0 2124 0000 0101 080a 0eec fef4  ....!$..........
0x0030   1f61 d752                                .a.R
15:55:38.672320 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 111:112(1) ack 108 win 5840  <nop,nop,timestamp 250412867 526505873> (DF) [tos 0x10]  (ttl 64, id 45164, len 53)
0x0000   4510 0035 b06c 4000 4006 05df c0a8 0119  E..5.l@.@.......
0x0010   c0a8 01fe 9033 0017 7524 4ef0 744a ed24  .....3..u$N.tJ.$
0x0020   8018 16d0 b18a 0000 0101 080a 0eec ff43  ...............C
0x0030   1f61 d791 6f                             .a..o
15:55:38.798130 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 112:113(1) ack 109 win 5840  <nop,nop,timestamp 250412879 526505893> (DF) [tos 0x10]  (ttl 64, id 45166, len 53)
0x0000   4510 0035 b06e 4000 4006 05dd c0a8 0119  E..5.n@.@.......
0x0010   c0a8 01fe 9033 0017 7524 4ef1 744a ed25  .....3..u$N.tJ.%
0x0020   8018 16d0 b168 0000 0101 080a 0eec ff4f  .....h.........O
0x0030   1f61 d7a5 6f                             .a..o
15:55:38.922395 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 113:114(1) ack 110 win 5840 <nop,nop,timestamp 250412892 526505906> (DF) [tos 0x10] (ttl 64, id 45168, len 53)
0x0000   4510 0035 b070 4000 4006 05db c0a8 0119  E..5.p@.@.......
0x0010   c0a8 01fe 9033 0017 7524 4ef2 744a ed26  .....3..u$N.tJ.&
0x0020   8018 16d0 ac4c 0000 0101 080a 0eec ff5c  .....L.........\
0x0030   1f61 d7b2 74   .a..t

Oh my! someone is trying to log in as root. First of all, you should never log in as root directly. It's safer to log in as yourself and then issue the su command to become root. Let's see what else we have.

15:55:39.175941 192.168.1.254.telnet > Mallard.36915: P [tcp sum ok] 113:123(10) ack 116 win 32120  <nop,nop,timestamp 526505943 250412916> (DF) (
ttl 64, id 6928, len 62)
0x0000   4500 003e 1b10 4000 4006 9b42 c0a8 01fe  E..>..@.@..B....
0x0010   c0a8 0119 0017 9033 744a ed29 7524 4ef5  .......3tJ.)u$N.
0x0020   8018 7d78 d18f 0000 0101 080a 1f61 d7d7  ..}x.........a..
0x0030   0eec ff74 5061 7373 776f 7264 3a20       ...tPassword:.

Ok, waiting for the password now. Again, pay attention to the last character of each of the following packets.

15:55:40.290932 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 116:117(1) ack 123 win 5840  <nop,nop,timestamp 250413029 526505943> (DF) [tos 0x10]  (ttl 64, id 45173, len 53)
0x0000  4510 0035 b075 4000 4006 05d6 c0a8 0119  E..5.u@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4ef5 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 b78e 0000 0101 080a 0eec ffe5  ................
0x0030  1f61 d7d7 68                             .a..h
15:55:40.382285 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 117:118(1) ack 123 win 5840  <nop,nop,timestamp 250413038 526506057> (DF) [tos 0x10]  (ttl 64, id 45174, len 53)
0x0000  4510 0035 b076 4000 4006 05d5 c0a8 0119  E..5.v@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4ef6 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 b012 0000 0101 080a 0eec ffee  ................
0x0030  1f61 d849 6f                             .a.Io
15:55:40.485134 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 118:119(1) ack 123 win 5840 <nop,nop,timestamp 250413048 526506066> (DF) [tos 0x10]  (ttl 64, id 45175, len 53)
0x0000  4510 0035 b077 4000 4006 05d4 c0a8 0119  E..5.w@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4ef7 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 b1fe 0000 0101 080a 0eec fff8  ................
0x0030  1f61 d852 6d                             .a.Rm
15:55:40.611048 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 119:120(1) ack 123 win 5840  <nop,nop,timestamp 250413061 526506076> (DF) [tos 0x10]  (ttl 64, id 45176, len 53)
0x0000  4510 0035 b078 4000 4006 05d3 c0a8 0119  E..5.x@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4ef8 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 b9e6 0000 0101 080a 0eed 0005  ................
0x0030  1f61 d85c 65                             .a.\e
15:55:40.918409 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 120:121(1) ack 123 win 5840  <nop,nop,timestamp 250413091 526506089> (DF) [tos 0x10]  (ttl 64, id 45177, len 53)
0x0000  4510 0035 b079 4000 4006 05d2 c0a8 0119  E..5.y@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4ef9 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 bcba 0000 0101 080a 0eed 0023  ...............#
0x0030  1f61 d869 62                             .a.ib
15:55:41.080038 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 121:122(1) ack 123 win 5840  <nop,nop,timestamp 250413108 526506120> (DF) [tos 0x10]  (ttl 64, id 45178, len 53)
0x0000  4510 0035 b07a 4000 4006 05d1 c0a8 0119  E..5.z@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4efa 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 bd89 0000 0101 080a 0eed 0034  ...............4
0x0030  1f61 d888 61                             .a..a
15:55:41.221580 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 122:123(1) ack 123 win 5840  <nop,nop,timestamp 250413122 526506136> (DF) [tos 0x10]  (ttl 64, id 45179, len 53)
0x0000  4510 0035 b07b 4000 4006 05d0 c0a8 0119  E..5.{@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4efb 744a ed33  .....3..u$N.tJ.3
0x0020  8018 16d0 ab6a 0000 0101 080a 0eed 0042  .....j.........B
0x0030  1f61 d898 73                             .a..s
15:55:41.312177 Mallard.36915 > 192.168.1.254.telnet: P [tcp sum ok] 123:124(1) ack 123 win 5840  <nop,nop,timestamp 250413131 526506150> (DF) [tos 0x10]  (ttl 64, id 45180, len 53)
0x0000  4510 0035 b07c 4000 4006 05cf c0a8 0119 E..5.|@.@.......
0x0010  c0a8 01fe 9033 0017 7524 4efc 744a ed33  .....3..u$N.tJ.3
0x0020   8018 16d0 b952 0000 0101 080a 0eed 004b  .....R.........K
0x0030   1f61 d8a6 65                             .a..e

We now know the password for the root account to this box is "homebase." We know that's the last character because of the next few packets which show the remote system information confirming a successful login.

15:55:41.876090 192.168.1.254.telnet > Mallard.36915: P 125:195(70) ack 126 win 32120  <nop,nop,timestamp 526506213 250413178> (DF) (ttl 64, id
6938, len 122)
0x0000   4500 007a 1b1a 4000 4006 9afc c0a8 01fe  E..z..@.@.......
0x0010   c0a8 0119 0017 9033 744a ed35 7524 4eff  .......3tJ.5u$N.
0x0020   8018 7d78 2034 0000 0101 080a 1f61 d8e5  ..}x.4.......a..
0x0030   0eed 007a 5468 696e 4c69 6e75 7820 6c6f  ...zThinLinux.lo
0x0040   6361 6c68 6f73 7420 322e 322e 3134 2d74  calhost.2.2.14-t
0x0050   6869                                     hi
15:55:41.964197 192.168.1.254.telnet > Mallard.36915: P [tcp sum ok] 195:206(11) ack 126 win 32120  <nop,nop,timestamp 526506222 250413187> (DF) (ttl 64, id 6939, len 63)
0x0000   4500 003f 1b1b 4000 4006 9b36 c0a8 01fe  E..?..@.@..6....
0x0010   c0a8 0119 0017 9033 744a ed7b 7524 4eff  .......3tJ.{u$N.
0x0020   8018 7d78 7705 0000 0101 080a 1f61 d8ee  ..}xw........a..
0x0030   0eed 0083 6c6f 6361 6c68 6f73 7423 20    ....localhost#.

There you have it, easy as pie.

With a little more experimentation and familiarity with tcpdump, you can do much more. Once again, if this same session used Secure Shell (SSh) instead, the attacker would not be able to capture any of this information, it would be garbage and of no use at all.

I hope I've showed you just how easy it is to look at packets on a network. There is much much more to it than what I've showed here, but this should give you a starting point for further learning.

Tripwire

Tripwire is a unique tool in that it doesn't really stop an attacker from compromising your system. However, Tripwire can tell you which files have been changed or replaced since the last run.

Tools such as Tripwire are often used on a honey pot system. You can see and log what the intruder changes this way very easily. Tripwire uses a database that includes all the information about your system that you want to log. This database is created the first time you run Tripwire. Then, it's used in successive audits to compare what has changed and what has not.

In many configurations, there are files that change all the time, such as those found in user's home directories so you don't want to include those in the audit. However, files that rarely change such as the programs in /usr, /sbin, /bin, and so forth, are prime targets for inclusion in the database. It depends on your needs and the programs you run.

Installing Tripwire

You can install Tripwire in one of two ways. You can either download the source code and compile it yourself (recommended for any security tool) or you can download the RPM or Debian packages and install that way. As I write this, the latest version of Tripwire is 2.3.1-2.

Place the source in an accessible location such as /usr/local/src. Then uncompress it in the normal way:

$ cd /usr/local/src
$ tar -xzvf tripwire-2.3.1-2.tar.gz
$ cd tripwire/src

Compiling Tripwire from source is different from most Linux applications, there's no config script for example. Before compiling Tripwire, look over the makefile and make any changes you need. The makefile is well documented, so take your time reading. You'll learn a lot in the process. Be sure you also have gtcc 2.95.2 or greater as well or Tripwire will not compile properly. The makefile makes many references to gmake, which on Linux systems at least is the same as make. In fact you may find both commands with one sym-linked to the other.

Tripwire can be compiled in one of two configurations, debug or release. Debug is useful if you are a developer and want to know more about the internals of Tripwire and such or if you're testing out a new version before putting on a production system. Release is the mode you'll use on a production system and the one we'll concentrate on here. Once you've made the modifications you want to the makefile, you compile tripwire with the following command:

$ make release

Once finished, you actually install Tripwire using the supplied install script. You must be root to run it:

$ su
password:

# cd /usr/local/src/tripwire-2.3.1-2/install
# ./install.sh

The install script copies all the Tripwire programs, libraries, and man pages to their proper places on the system. It also places the default policy and configuration files in /etc/tripwire/. The install script will also prompt you for site key passwords and local key passwords, compile the default policy and configuration files, and finally sign these files with the site key. Remember these passphrases as they cannot be recovered. You have been warned!

Tripwire's files

Tripwire uses four files in the working of its magic: the policy file, the configuration file, the database file, and the report files. Some of these files exist in two formats: binary and text.

The text version is what you edit to configure how you want Tripwire to work. Then, you run twadmin to compile the text files into the binary files it uses. You should then move the text files to a floppy or CD media and delete them from the system. This way, an intruder cannot change what Tripwire reports to cover his tracks.

Screenshot.

The policy file /etc/tripwire/tw.pol is used by you as the administrator to specify how Tripwire checks the system. The policy file consists of various rules, each of which specify a system object that Tripwire monitors, and describe what changes to that object should be reported or ignored.

The configuration file /etc/tripwire/tw.cfg stores system-specific information such as the location of the Tripwire data files and the settings used for notification via email when violations occur.

A good sample policy and configuration file comes with the Tripwire source code. In the beginning, you can use them as a starting point instead of trying to create your own until you become more familiar with how Tripwire works.

The database file /var/lib/tripwire/$(HOSTNAME).twd is central to the integrity assessment strategy. Tripwire uses the rules specified in the policy file to create a "snapshot" of the system. Ideally, this snapshot should be of a clean install so you are guaranteed of being in a known secure state. The database file is then used as a "baseline" during integrity checks. The current state of the system is compared against this database to determine if any changes have been made since the database was created.

The report files /var/lib/tripwire/$(HOSTNAME)-$(DATE).twr are produced every time the integrity check is run. Usually, you run an integrity check at a set interval using the cron facility. The results of that check, including any changes (additions, deletions, or modifications) that violate the policy file rules, are stored in a report file. Summary results of the integrity check are stored in the report file which can be viewed in a variety of formats at varying levels of detail.

Resources

Web sites:

HoneyNet Project

The SANS Institute

Security Portal

Linux Administrator's Security Guide

Root Prompt

Linux System Administrator's Guide

Books:

Network Intrusion Detection, 2nd Ed. (New Riders) ISBN: 0-7357-1008-2

Intrusion Signatures & Analysis (New Riders) ISBN: 0-7357-1063-5

Maximum Linux Security (SAMS) ISBN: 0-672-31670-6

Linux Network Administrator's Guide, 2nd Edition (O'Reilly) ISBN: 1-56592-400-2

Practical Unix and Internet Security (O'Reilly) ISBN: 1-56592-148-8

Running Linux, 3rd Edition (O'Reilly) ISBN: 1-56592-469-X

Linux System Security (Prentice Hall) ISBN: 0-13-015897-0

Using Tripwire

Once everything is set up, you're ready to begin using Tripwire. Tripwire consists of a few different applications: twadmin compiles the text files into binary files used by Tripwire while twprint allows you to take a binary config file and get the ASCII text out of it. Because these programs are administrative, you'll want to remove them from your system once everything is set up. Removing them will prevent an intruder from using the admin programs to change what Tripwire looks for and so forth.

The main application is called tripwire. Use tripwire to create your initial database like this:

# tripwire -m i

The -m tells Tripwire what mode to use, in this case, initialization mode. Now that the database is created, you run comparisons on your system with the following command:

# tripwire -m c

You should put this into a crontab or shell script file. How often you run Tripwire depends on your level of paranoia. If you are really paranoid, you might run it every couple hours. If you're not that paranoid, once a day may be sufficient. Remember, you have to look over the reports generated by Tripwire, so if you do it too often, the task may become too daunting. But again, that's entirely up to you. What do the reports look like you ask? Like this.

There is more to it than this, but the files can be large depending on your system. If you find that you are changing some files that Tripwire compares (new versions of applications for example) you can always update Tripwire's database with the command:

# tripwire -m u

Conclusions

There are many things you can do to avoid use of some tools like tcpdump and Ethereal. You can use a switched network for a start to avoid "broadcasting" your packets to every workstation on your network. Note: This doesn't stop somone from sniffing traffic going from your network to the Internet, or from capturing data from the Internet (such as a remote user) destined for your network. But it does narrow down the ability to use such tools internally, unless they are being run on the source or destination machine (such as one of your servers).

Another thing to do, is get rid of any network protocol that transmits text in the clear. This means telnet and FTP to start. You should also look at getting rid of the Berkley "R commands" such as rdist, rlogin, et al. You should be using SSh for all remote access and server to server synchronization. If you really need the Berkley "R commands," consider running them over an encrypted SSh session instead.

Tripwire is a great tool for tracking changes to your system. Depending on how "paranoid" you are and how often you run it, you can find a change very quickly, hopefully before the intruder has a chance to stop Tripwire from running and trash any logs. But hey, that's where tools like syslog come in. We'll take at look at this tool next week along with the venerable snort.

Carl Constantine works for Open Source Solutions, Inc. (www.os-s.com) as a Linux Trainer and Programmer.


Return to the Linux DevCenter.

Copyright © 2009 O'Reilly Media, Inc.