Rate Limiting the FCC with Linux IPtables

Inspired by NeoCities, and their Ferengi Plan, Blackhole-Networks has decided to shape the FCC's traffic to this entire server until they pay us the equivalent of $1000 in either Bitcoin, Litecoin, or Vertcoin.

A different approach was taken here, rather than use the rate-limiting features of an application like Nginx, IPtables (Netfilter) was used in conjunction with Linux Traffic Control to rate limit all of the FCC's assigned address space to a paltry 10Kpbs - the equivalent of an early 1990's dialup connection. This was done on a Debian based system using iptables and tc.

Topics

---

1. Methodology


2. Installation


3. Setup the Queues


4. Identify IP blocks


5. Mark FCC's Packets


6. Testing the Rate Limiting


7. Home

---

Methodology

Linux Traffic Control will be used to setup two class based queues. The first queue will be used for normal traffic and be allowed to use all of the available bandwidth, while the second queue will be shaped with a token bucket filter at a measly rate of 10 Kbps. Netfilter will be used to mark all traffic to or from any of the FCC's IP blocks. Traffic with the mark of the FCC will be sent to the shaper, while all other traffic will use the big fat juicy queue.

Installation

Install iptables and iproute . Odds are that they are already installed, but if they are missing for some reason, on Debian based systems:

root@system:~# apt-get install iptables iproute

Setup the queues

This assumes that the default queuing is in use, and nothing will be "hurt" by erasing any custom traffic control on the ISP bound interface and starting anew. The bandwith of our interface, and presumabley our upstream ISP all the way to their IP transit provider is 1 Gbps. All of these commands need to be done with root access.

The first step is to clear out our queing:

root@system:~# tc qdisc del dev eth0 root

Next, we create a root Class Based Queue (CBQ) on our ISP bound interface, eth0. The bandwidth is our bandwidth to our ISP and avpkt is the average packet size. For bandwidth, Mbit and Kbit can also be used.

root@system:~# tc qdisc add dev eth0 root handle 1: cbq bandwidth 1Gbit avpkt 1000

Next we create a class for our aggregate bandwidth of our interface.

root@system:~# tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 1Gbit rate 1Gbit allot 1514 weight 1Gbit prio 8 maxburst 1000 avpkt 1000

Now we create two subordinate classes, first a placeholder class for our normal traffic which we allow to use all of our bandwidth unrestricted.

root@system:~# tc class add dev eth0 parent 1:1 classid 1:100 cbq bandwidth 1Gbit rate 1Gbit allot 1514 weight 1Mbit prio 5 maxburst 1000 avpkt 1000 

Next, we create a shitty class just for the FCC bound packets.

root@system:~# tc class add dev eth0 parent 1:1 classid 1:200 cbq bandwidth 1Gbit rate 10kbit allot 1514 weight 10kbit prio 3 maxburst 5 avpkt 1000 bounded

Now we setup the queuing strategies. We use a Stochastic Fair Queue for the normal traffic, class 1:100.

root@system:~# tc qdisc add dev eth0 parent 1:100 sfq quantum 1514b perturb 15

And we assign a Token Bucket Filter limited to 10 Kbps for the traffic in class 1:200 which will be the FCC's packets. We'll keep the packets around for 100ms before discarding them.

root@system:~# tc qdisc add dev eth0 parent 1:200 tbf rate 10kbit latency 100ms burst 1540

Finally, we send everything that has a netfilter mark of 6 into the shitty queue!

root@system:~# tc filter add dev eth0 protocol ip parent 1:0 prio 8 handle 6 fw flowid 1:200

We can inspect our results, and see what we have so far.

root@system:~# tc qdisc
qdisc cbq 1: dev eth0 root refcnt 2 rate 1000Mbit (bounded,isolated) prio no-transmit
qdisc sfq 8015: dev eth0 parent 1:100 limit 127p quantum 1514b divisor 1024 perturb 15sec 
qdisc tbf 8017: dev eth0 parent 1:200 rate 10000bit burst 1540b lat 100.0ms 

Identify the IP blocks to be thottled

After scouring various sources, and chat from the Ferengi Plan on Github, the followig address blocks have thus far been identified as belonging to the FCC:

192.133.125.0/24
165.135.0.0/16
192.104.54.0/24
4.21.126.0/24
65.125.25.64/26
208.23.64.0/25
63.109.101.0/24
208.245.40.192/29

And IPv6:

2620:0:610::/48
2600:803:230::/48

Marking the FCC's Packets

Netfilter is used to do this with the iptables, and ip6tables commands in the form:

root@system:~# iptables -A PREROUTING -t mangle -j MARK --set-mark 6 -s <FCC IPv4 prefix>
root@system:~# iptables -A POSTROUTING -t mangle -j MARK --set-mark 6 -d <FCC IPv4 prefix>

And for IPv6:

root@system:~# ip6tables -A PREROUTING -t mangle -j MARK --set-mark 6 -s <FCC IPv6 prefix>
root@system:~# ip6tables -A POSTROUTING -t mangle -j MARK --set-mark 6 -d <FCC IPv6 prefix>

Since I'm lazy, I just wrote a couple of quick shell scripts to iterate through lists of the FCC's prefixes, and I'm just throttling outbound traffic to the FCC from my server. The file /etc/fcc.IPv4.ips contains the IPv4 addresses above, and likewise /etc/fcc.IPv6.ips contains the IPv6 address blocks.

#!/bin/sh
TABLE="POSTROUTING"
IPLIST="/etc/fcc.IPv4.ips"
IPT=/sbin/iptables
for IP in `cat $IPLIST | grep -v ^# | sort | uniq` ; do
	$IPT -A $TABLE -t mangle -j MARK --set-mark 6 -d $IP
done 

And for IPv6:

#!/bin/sh
TABLE="POSTROUTING"
IPLIST="/etc/fcc.IPv6.ips"
IPT=/sbin/ip6tables
for IP in `cat $IPLIST | grep -v ^# | sort | uniq` ; do
	$IPT -A $TABLE -t mangle -j MARK --set-mark 6 -d $IP
done 

And inspecting our work:

root@system:~# iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       all  --  192.133.125.0/24     0.0.0.0/0            MARK set 0x6

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       all  --  0.0.0.0/0            165.135.0.0/16       MARK set 0x6
MARK       all  --  0.0.0.0/0            192.104.54.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            192.133.125.0/24     MARK set 0x6
MARK       all  --  0.0.0.0/0            208.23.64.0/25       MARK set 0x6
MARK       all  --  0.0.0.0/0            208.245.40.192/29    MARK set 0x6
MARK       all  --  0.0.0.0/0            4.21.126.0/24        MARK set 0x6
MARK       all  --  0.0.0.0/0            63.109.101.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            65.125.25.64/26      MARK set 0x6
MARK       all  --  0.0.0.0/0            165.135.0.0/16       MARK set 0x6
MARK       all  --  0.0.0.0/0            192.104.54.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            192.133.125.0/24     MARK set 0x6
MARK       all  --  0.0.0.0/0            208.23.64.0/25       MARK set 0x6
MARK       all  --  0.0.0.0/0            208.245.40.192/29    MARK set 0x6
MARK       all  --  0.0.0.0/0            4.21.126.0/24        MARK set 0x6
MARK       all  --  0.0.0.0/0            63.109.101.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            65.125.25.64/26      MARK set 0x6
root@system:~#

Testing the Rate Limiting

Since I don't work for the FCC, how can we tell if this is working properly or not? Simple, I'll test from another server I own with and without the rate limiting in place for it's IP address by fetching a 1 Megabyte file.

[user@hobbiton ~]$ wget www.blackhole-networks.com/1meg.file
--2014-05-10 00:49:21--  http://www.blackhole-networks.com/1meg.file
Resolving www.blackhole-networks.com... 148.251.27.98, 2a01:4f8:201:73f0:2:2:56b:d808
Connecting to www.blackhole-networks.com|148.251.27.98|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576 (1,0M) [application/octet-stream]
Saving to: “1meg.file”

100%[======================================>] 1.048.576    912K/s   in 1,1s    

2014-05-10 00:49:23 (912 KB/s) - “1meg.file” saved [1048576/1048576]

[user@hobbiton ~]$ 

Now we add the firewall mark to the test servers IP address, first verifying the IP on our test box:

[user@hobbiton ~]$ ip addr show dev venet0
3: venet0: <BROADCAST,POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/void 
    inet 127.0.0.1/32 scope host venet0
    inet 192.3.130.135/32 brd 192.3.130.135 scope global venet0:0
[user@hobbiton ~]$ 

And we mark 192.3.130.135 with the mark of the FCC:

root@pippacott:~# iptables -A POSTROUTING -t mangle -j MARK --set-mark 6 -d 192.3.130.135
root@pippacott:~# iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       all  --  192.133.125.0/24     0.0.0.0/0            MARK set 0x6

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       all  --  0.0.0.0/0            165.135.0.0/16       MARK set 0x6
MARK       all  --  0.0.0.0/0            192.104.54.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            192.133.125.0/24     MARK set 0x6
MARK       all  --  0.0.0.0/0            208.23.64.0/25       MARK set 0x6
MARK       all  --  0.0.0.0/0            208.245.40.192/29    MARK set 0x6
MARK       all  --  0.0.0.0/0            4.21.126.0/24        MARK set 0x6
MARK       all  --  0.0.0.0/0            63.109.101.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            65.125.25.64/26      MARK set 0x6
MARK       all  --  0.0.0.0/0            165.135.0.0/16       MARK set 0x6
MARK       all  --  0.0.0.0/0            192.104.54.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            192.133.125.0/24     MARK set 0x6
MARK       all  --  0.0.0.0/0            208.23.64.0/25       MARK set 0x6
MARK       all  --  0.0.0.0/0            208.245.40.192/29    MARK set 0x6
MARK       all  --  0.0.0.0/0            4.21.126.0/24        MARK set 0x6
MARK       all  --  0.0.0.0/0            63.109.101.0/24      MARK set 0x6
MARK       all  --  0.0.0.0/0            65.125.25.64/26      MARK set 0x6
MARK       all  --  0.0.0.0/0            192.3.130.135        MARK set 0x6
root@pippacott:~# 

And we try to repeat our test, but give up after a few minutes because it is so SLOW!

[user@hobbiton ~]$ wget www.blackhole-networks.com/1meg.file
--2014-05-10 00:54:47--  http://www.blackhole-networks.com/1meg.file
Resolving www.blackhole-networks.com... 148.251.27.98, 2a01:4f8:201:73f0:2:2:56b:d808
Connecting to www.blackhole-networks.com|148.251.27.98|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576 (1,0M) [application/octet-stream]
Saving to: “1meg.file.1”

 7% [==>                                    ] 83.724       706B/s  eta 20m 18s 

We've successfully anti-Net-Nuetralized the FCC! Welcome to a taste of the future Tom Wheeler!

Note to the FCC: Inspired by Neocities, this site has also implemented "The Ferengi Plan". The FCC's internal IP blocks have all been throttled to 10kbps on this server. The rate limiting will be lifted once the FCC pays a yearly fee of $1000 in either Bitcoin, Litecoin or Vertcoin at the exchange rate of the day. Send payments to bitcoin:1Kpgfj4Ma8MTku6UCpYTmZf65cqrfcdUYa litecoin:La9BC8ni532K4hw1d3tRzCrrUGt2NFUR9o, or vertcoin:VxJGp6igThzv4MhAeVmQcf9UzWJohCXbR3 . Once the appropriate amount has been recieved, the shaping will cease.

Last page update 9 May 2014

Cheat Sheet: Using an Encrypted Filesystem for Mail Directories by Blackhole Networks is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Concerning comments and/or errors, please contact The BlackHole