TCP Traffic Analyzer

You probably have questions like these about traffic on a TCP (Transmission Control Protocol) server (or client):

  • How many connections lasted more (or less) than X milliseconds?
  • How many connections needed more than N attempts to succeed?
  • What is the distribution of connection duration or connection throughput?
  • What is the distribution of connection duration or throughput for connections in which the server or client sent more than N bytes?
  • What specific IP addresses and ports had connections that lasted between 50 and 100 milliseconds long?

You can get answers to these questions (and more) using Yahoo!'s TCP Traffic Analyzer (yconalyzer), available as an open-source project via free download.

At Yahoo! we use yconalyzer to analyze traffic coming into our servers from all over the world via HTTP, POP, IMAP, and SMTP. We use yconalyzer to gather information as a TCP client when our internal servers connect to other back-end servers, and get information that is eventually used in responding to the end user. Yconalyzer is frequently used for troubleshooting, as well as at regular intervals, to gather data.

Performance and compatibility

Yconalyzer works by gathering data from your network driver using libpcap. It uses a restrictive filter to capture a minimal set of TCP packets off the network interface and gather as much data as possible from these packets to derive the information as described earlier.

By restricting the filter to capture the packets, yconalyzer uses minimal CPU resources to monitor your system. You really don't want your monitoring software to skew your application behavior!

All pcap capture files generated by yconalyzer are readable by tcpdump. Yconalyzer can analyze pcap files captured by tcpdump as well. Yconalyzer takes all filters accepted by tcpdump.

Yconalyzer works for any protocol over TCP, so it can be used for HTTP, SMTP, POP, IMAP, and so on. It can be run on the servers speaking these protocols, or on clients as well.

Examples

Yconalyzer is launched via a command-line interface. It is easiest to run it in a capture mode, as in the following example:

yconalyzer -p 80 -w http.pcap -t 300

This command asks yconalyzer to monitor port 80 for 300 seconds, and write the packet stream onto the file http.pcap. The -t is optional (default being 60 seconds). You may also specify filters at the and of the command, as you may with tcpdump. An example would be something like src host proxy.foo.net. If such a filter is specified, yconalyzer would only capture packets to/from proxy.foo.net.

Once the pcap file is captured (either using tcpdump or yconalyzer), yconalyzer can be run again to analyze the captured file. For example:

yconalyzer -p 80 -r http.pcap

This command prints out overall statistics of all connections to/from port 80 — including some general statistics — and then a table that groups the connections as per their duration.

Yconalyzer accepts several command-line options to slice data differently and get a bird's eye view of the performance of service on a TCP port. You can also drill down on a specific set of connections on any bucket, and see what happened on a specific connection using the tcpdump filters that yconalyzer prints out.

Yconalyzer also has an option to output data in raw format, so that you can build web pages with the data, save data for consolidation across hosts, and so on.

In conclusion

You don't have to be intimidated by all the options that yconalyzer offers. Not only does it give a nice summary of options when the command is typed, it is distributed with a man page that explains the options in detail, with examples.

Try yconalyzer today! Download the sources.