oreilly.comSafari Books Online.Conferences.


Big Scary Daemons

Controlling Bandwidth


A buddy over at my favorite small ISP called me the other day: "Our T1 is running really, really slow!" I tried a traceroute from the outside world. While I hit the backbone's border router in 50 milliseconds, my packets took over 2 seconds to cross the T1 and hit the ISP's router.

The network monitor showed that his T1 was actually doing just fine. After all, it was pushing about 1.544 megabits out and about 20 megabits in -- excellent throughput for a circuit that was running at capacity. The traffic had been averaging about half a T1 just the day before -- something had obviously changed.

Because every web server on the network is monitored, it was trivial to track down the offender. One of the web servers accounted for all that traffic. Five minutes with WebTrends gave him everything we needed, and we checked the site in question.

Nope, not porn. No MP3s. None of the usual suspects, just a small company with the typical half-dozen page site, a guestbook CGI, and a banner ad.

The banner ad was consuming a full megabit a second. The ultra-high-end FreeBSD server -- Celeron 433-MHz processor, 128 Mbytes of RAM, 20-Gbyte IDE hard disk -- wasn't even noticing the strain, but the T1 was buckling and every other site was slower than Microsoft admitting wrongdoing in the antitrust suit.

These days, a small Web company stays in business by maintaining a high level of customer service. My friend called the user. The client appreciated advance warning of the outrageous bandwidth charge he was going to get, and asked that we help trim the bandwidth used to a reasonable level.

And herein lies the problem. How do you control bandwidth usage? One thought rattled at the back of my brain, but I pushed it aside in favor of something that wouldn't touch the kernel.

We first checked in Apache itself. Apache has a module to control bandwidth consumption, mod_throttle (/usr/ports/www/mod_throttle). You can easily install it just like any other port.

The mod_throttle configuration is very simple. First, edit httpd.conf to include the mod_throttle configuration file.

Include /usr/local/etc/apache/throttle.conf

The throttle.conf file starts off by telling Apache to load the throttle module, and to add it to the system. Reading the mod_throttle documentation, we quickly arrive at a basic setup.

LoadModule throttle_module    libexec/apache/
AddModule mod_throttle.c

ThrottleUser websiteusername speed 300 60 m

This worked, but was rather jerky. Pages would sometimes load quickly, sometimes slowly, and frequently be flat-out denied. It would be better to enable a smooth bandwidth restriction -- say, hook up this web site with a 128-Kbps bottleneck and let Apache behave normally. I've done this before by connecting an entire server to the network via PPP over a serial cable, but you can't disconnect one virtual domain from the rest of the server.

Also in Big Scary Daemons:

Running Commercial Linux Software on FreeBSD

Building Detailed Network Reports with Netflow

Visualizing Network Traffic with Netflow and FlowScan

Monitoring Network Traffic with Netflow

Information Security with Colin Percival

It's quite possible that mod_throttle could have been tweaked to perform better. I also suspect that on a large installation, the law of large numbers would have taken over. Circuit bandwidth would not have been irregular, because every throttled site would be behaving similarly.

Also, this Apache daemon was already heavily patched -- mod_ssl, FrontPage, and php4. I'm no Apache guru, and adding yet another module into the mix isn't in my comfort zone. The simpler we keep the systems, the better.

The next thought was to try shaping the traffic. The ISP has a Cisco router, after all. Ciscos can do this sort of thing. That's what you buy that expensive service contract for.

It turned out that we could have done this exactly as we liked on the Cisco -- if we upgraded to a more recent IOS. And added memory. Oh, you can't add more memory to a Cisco 2501? Would you like to speak to a salesman?

This is when that niggling thought in the back of my head stood up and screamed, "Dummynet!"

Pages: 1, 2

Next Pagearrow

Sponsored by: