Bennomarks: measuring latency in the Linux kernel
Benno Senoner has designed a series of stress tests to measure latency at critical points in Linux performance, including the impact of the X window system, calls to the
/proc filesystem, and read/write/copy disk activity. A test sound is played, the program measures the time taken by the
write() call to
/dev/dsp for each iteration of the sound, and the resulting data is represented in graphs with the ideal latency (the time it takes to play a single audio fragment) and the real measured latency superimposed on each other. The test results ("bennomarks") are also directed to the console, but the graphs are an impressive visual display of the efficiency gained by the low-latency patches.
Follow these links to view the performance graphs for the following kernels:
- 2.2.10 patched, tuned
- 2.2.17 patched, tuned
The graphs tell the tale: Average latencies under 2 msec with Ingo Molnar's original patch for the 2.2.10 kernel, and less than 4 msec with Andrew Morton's patch for kernel version 2.4.0-test9. The comparative views of the 2.4.0-test9 graphs are most instructive: Latencies decreased from almost 400 msec (unpatched kernel, untuned drives) to less than 4 msec (patched kernel, tuned drives).
Note: In all cases the test platform was a 550 MHz Pentium III with 256 MB RAM and two 15-GB Maxtor EIDE hard disks.
I urge you to run Benno's latency test suite. Two versions of the software are available (one for creating GIF-format graphs and one for PNG graphs); the build is uncomplicated, and it is easy to run. Simply become root on your system, tune your hard drives with Benno's tunedisk script, turn off any screensaver you have installed (you must run latencytest in X), and then start the program with this command sequence:
./do_tests none 3 256 0 350000000
do_tests script will perform five successive latency tests (stress_x11, stress_proc, stress_diskwrite, stress_diskcopy, stress_diskread) on a simple cyclic waveform (the
none in the command sequence; a WAV file may be named instead), using three audio fragments of 256 bytes each, with 0 syncing, and with a test file size of 350 MB (for the disk I/O tests). Console output should resemble this report from a run on the 2.4.0-test9 kernel.
Although Benno's program is generally safe and easy to use, you will be running it as root, and I do have a warning for certain users: If you have a SBLive soundcard and are using the ALSA driver, do_tests may freeze your system completely, requiring a hard system reset. Version 3.9.3q of the OSS/Linux driver for the SBLive gave me no troubles, and I assume the kernel modules will also work without problems. I must emphasize that the problem with ALSA occurs only with the SBLive: I ran the tests with the ALSA 0.5.9d driver for my SB PCI128 and experienced no problems.