Bandwidth monitoring with iptables

Gerard Beekmans
minor modifications By Jalal Hajigholamali|

Most of the time we use iptables to set up a firewall on a machine, but iptables also provides packet and
byte counters. Every time an iptables rule is matched by incoming or outgoing data streams, the software
tracks the number of packets and the amount of data that passes through the rules.
It is easy to make use of this feature and create a number of "pass-through rules" in the firewall. These
rules do not block or reroute any data, but rather keep track of the amount of data passing through the
machine. By using this feature, we can build a simple, effective bandwidth monitoring system that does
not require additional software.
Depending on how the firewall rules are set up, the setup for bandwidth monitoring may be very simple
or very complex. For a desktop computer, you may need to create only two rules to log the total input and
output. A system acting as a router could be set up with additional rules to show the totals for one or more
subnets, right down to the individual IP address within each subnet. In addition to knowing exactly how
much bandwidth each host and subnet on the network is using, this system could be used for billing or
chargeback purposes as well.
Rules setup
The rules setup itself is quick and straightforward, and takes only a few minutes. Obviously, you need to
be root or use sudo to insert iptables rules.
The examples in this article are based on a router that provides Internet service to various towns. The
iptables rules keep track of how much bandwidth each town uses and how much bandwidth each
customer in that town uses. At the end of each month, an administrator checks the counters. Individuals
who use more than they were supposed to get billed for over usage, the counters are reset to zero, and the
process is repeated at the beginning of the next month.
The IP addresses in this article are modified from the real addresses. We'll use the private IP space
192.168.0.0/16, subnetted into smaller blocks.
First, we will create two custom chains for the two towns and put town-specific rules in them. This will
keep the built-in FORWARD chain relatively clean and easy to read. In this example, the FORWARD
chain will only provide the global counters (all customers combined on a per-town basis).

iptables -N town-a
iptables -N town-b
The next data element is the total bandwidth counter. Because this machine is a router only, the INPUT
and OUTPUT chains are of little interest. This machine will not be generating a significant amount of
bandwidth (i.e., it is not serving as a mail or Web server), nor will it be receiving significant uploads from
other hosts.
Total bandwidth downloaded by and uploaded to the two towns combined:
iptables -A FORWARD
This is the easiest of rules. The rule will match any source and any destination. Everything that is being
passed through this router matches this rule and will provide the total of combined downloaded and
uploaded data.
We also want to see how much each town downloads and uploads separately:
# Town A Downloads
iptables -A FORWARD -d 192.168.1.0/26 -j town-a
# Town A Uploads
iptables -A FORWARD -s 192.168.1.0/26 -j town-a
# Town B Downloads
iptables -A FORWARD -d 192.168.1.64/27 -j town-b
# Town B Uploads
iptables -A FORWARD -s 192.168.1.64/27 -j town-b
The use of source and destination in the above rules may be a source of confusion. Destinations are often
equated with uploads, and sources are downloads. This would be true whether the data was destined for
the router or originated from the router itself.
In this application, however, we reverse the perspective. This router is forwarding (uploading) data to a
destination, but from a customer perspective, data is being received. In other words, the customer is
downloading that data. When dealing with customers, the terminology is data they downloaded, not what
the router uploaded to them. This is why in the FORWARD chain, the terms destination and source
typically have reversed meanings.
The rules created above give us separate totals for all downloads to and uploads from each individual
town. This is accomplished by matching the source and destination of all traffic through the router for a
town's specific subnet. After a rule is matched, the -j option invokes a jump to one of the custom chains.
These custom chains can then be used to add additional rules pertaining to the subnet. For instance, rules
can be created for each individual IP address in that subnet to track bandwidth on a per-host basis:
# Town A, Host 192.168.1.10 Download
iptables -A town-a -d 192.168.1.10
# Town A, Host 192.168.1.10 Upload
iptables -A town-a -s 192.168.1.10
You could repeat this process for every IP address for all towns within the subnet.

Bandwidth statistics
Viewing the current bandwidth usage is a matter of running iptables with the -L and -v options. The -L
outputs the statistics for a chain (or all chains if none is provided). The -v option provides verbose
output, including the packet and byte counters that we are interested in. I recommend using the -n option
as well to prevent DNS lookups, meaning iptables will show the IP addresses without attempting to
resolve the hostnames for the IP addresses, which would put additional and unnecessary load on the
router.
The output below is modified from the full output for brevity:
root@raptor:~# iptables -L -v -n
Chain FORWARD (policy ACCEPT 7936M packets, 3647G bytes)
bytes target source
destination
338G
0.0.0.0/0
0.0.0.0/0
104G town-a 0.0.0.0/0
192.168.1.0/26
40G town-a 192.168.1.0/26 0.0.0.0/0
20G town-b 0.0.0.0/0
192.168.1.64/27
12G town-b 192.168.1.64/27 0.0.0.0/0
This snippet shows that towns A and B combined have downloaded and uploaded a total of 338GB. Town
A is responsible for 104GB downloaded and 40GB uploaded. In the first line of output of the chain itself
is a "more" total number -- 3,647GB. This is the total amount of data routed through since the last time
this router was restarted, or more accurately, since the last time the iptables modules were inserted into
the kernel.
When a chain is "zeroed" (resetting all counters in a chain to zero) with the -Z option, this number is not
reset. For this reason, I recommend creating a real total rule to make it easier to reset the total counter. It
then takes one command to reset the counters, and you do not need to remove modules, restart the server,
or work with the iptables-save and iptables-restore commands to reset the counter.
Scrolling further down the output shows the individual IP addresses. Example for Town A:
Chain town-a (2 references)
bytes source
destination
32G
0.0.0.0/0
192.168.1.10
282M 192.168.1.10 0.0.0.0/0
1521M 0.0.0.0/0
192.168.1.11
656M 192.168.1.11 0.0.0.0/0
This output further breaks down the total bandwidth of Town A down to the individual customers.
The "2 references" shown in the iptables output refer to the two rules in the FORWARD chain that jump
to this chain.

Saving data across reboots

If you reboot the machine or remove the iptables kernel modules, you'll lose all of your packet and byte
counters. if these counters are to be used for billing purposes, you will want to make backups of the
running counters, and in the event of a reboot, restore the counters rather than starting from zero.
The iptables package comes with two programs that aid in this: iptables-save and iptablesrestore. Both programs need to be told to explicitly use the packet and byte counters during backup
and restore using the -c command line option.
The backup and restore process is fairly straightforward. To back up your iptables data, use
iptables-save -c > iptables-backup.txt. To restore the data, after reboot, use
iptables-restore -c < iptables-backup.txt.
Conclusion
Iptables provides a quick and easy way to track bandwidth usage without having to install additional
software. You have, and probably already use, the tools needed to accomplish this monitoring.
The flexibility and power of iptables allows for more complex monitoring scenarios. You can create rules
to not only track different subnets, but also to track specific ports and protocols, which lets you track
exactly how much of each customer's traffic is Web, email, file sharing, etc.
In addition, these bandwidth monitoring rules can also become blocking rules. If a host has used too
much bandwidth, its rule in a town's specific chain can be modified by adding -j DROP to both the
download and upload rules. This effectively stops traffic being routed to and from that host.