Connection rate limiting with Apache2::ConnectionUtil pnotes
am 13.05.2009 03:50:34 von Fred MoyerGreetings,
I ran into an issue today where I had a load surge in my application
server, so I looked around for a suitable throttling module and didn't
see anything simple.
So I put together this snippet (very rough) which uses the connection
pnotes data structure to track how many requests are made over a
connection, and return a 503 if the client requests too much. I
thought about using something that shared the data across processes,
but was in a jam and needed to slow down some egregious user agents
which were making multiple rapid requests in succession. I had to
turn on KeepAlives for this to work, but it has been working pretty
well.
Thought I would share this (the code was written under some duress so
isn't all that pretty). MIN_COUNT is the minimum number of requests
to trigger a violation, and MAX_RATE is the maximum allowed request
rate over one connection.
my $c = $r->connection;
if (my $attempts = $c->pnotes($c->remote_ip)) {
my $count = $attempts->{count};
my @times = @{$attempts->{times}};
my $idx;
if ($#times > 9) {
# take the last 10 points
$count = 10;
$idx=$#times-$count;
} else {
$idx=0;
}
my $total_time = $times[$#times] - $times[$idx];
push @{$attempts->{times}}, time();
$attempts->{count}++;
$c->pnotes($c->remote_ip => $attempts);
if ($total_time != 0) {
my $rate = ($count / $total_time);
$r->log->debug("throttle check ip $ip, count $count, time
$total_time, rate $rate") if DEBUG;
if (($count > MIN_COUNT) && ($rate > MAX_RATE)) {
$r->log->error("rate violation ip $ip, total time $total_time,
count $count, rate $rate");
# make 'em wait
sleep 5;
return Apache2::Const::HTTP_SERVICE_UNAVAILABLE;
}
}
} else {
# start tracking this client
my %attempts = ( 'count' => 1, 'times' => [ time() ]);
$r->log->debug("setting new limit check for ip $ip, count 1, time "
.. time()) if DEBUG;
$c->pnotes($c->remote_ip => \%attempts);
}