Discussion:
Can ipfw be used to limit concurrent requests from an IP?
(too old to reply)
Will Squire
2016-05-27 19:34:56 UTC
Permalink
Can ipfw limit the number requests in a given amount of time from a specific IP?

To contextualise, if an IP sends requests in high concurrency (let's say 50 a second) can ipfw either block requests the exceed a threshold for that second (lets say the threshold is 20, 30 would be blocked), or ban/deny the given IP for exceeding a threshold?

The aim is to lessen strain under DoS attacks, specifically for HTTP. The system is using Apache and mod_evasive has been added and tested, but it is not functioning correctly.

(P.S. The freebsd-ipfw list seems to be for development of the technology only, so asking this here. Please let me know if this isn’t the case)


Thanks

Kind regards,

Will Squire
Ian Smith
2016-05-28 14:27:58 UTC
Permalink
In freebsd-questions Digest, Vol 625, Issue 7, Message: 3
On Fri, 27 May 2016 20:34:56 +0100 Will Squire <***@hotmail.co.uk> wrote:

(please wrap lines < 80 columns if possible)
Post by Will Squire
Can ipfw limit the number requests in a given amount of time from a
specific IP?
To contextualise, if an IP sends requests in high concurrency (let's
say 50 a second) can ipfw either block requests the exceed a
threshold for that second (lets say the threshold is 20, 30 would be
blocked), or ban/deny the given IP for exceeding a threshold?
Not as such. If you know the specific IP address (or range, or subnet)
you can use stateful rules with 'limit' instead of 'keep-state' to limit
the maximum number of concurrent connections to the port/s configured in
a given rule; see ipfw(8). You cauld use a table of addresses to block
or limit rather than hard-coding them into rule/s.

While this is very useful for avoiding DoS of any particular service, it
does not allow you to specify a rate, nor time limit, nor (directly) to
block an IP address that's exceeding the given number of connections.
Post by Will Squire
The aim is to lessen strain under DoS attacks, specifically for HTTP.
The system is using Apache and mod_evasive has been added and tested,
but it is not functioning correctly.
I haven't used (nor heard of) mod_evasive so can't comment on that, but
limiting the total number of connections open to a given service can
certainly mitigate the effect of such DoS attacks.

You could of course use /etc/inetd.conf (aka TCPwrappers) to limit
connections in just the ways you want, though I'm not sure starting HTTP
connections in that way is recommended these days. I use if for FTP and
POP3 connections, which works very well, thus:

sola# grep -v '#' /etc/inetd.conf
ftp stream tcp nowait/7/3 root /usr/libexec/ftpd ftpd -dll -S
pop3 stream tcp nowait/7/4 root /usr/local/libexec/qpopper qpopper -s -T 120

See inetd(1), particularly re the inetd.conf setting:
{wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]]

The above example limits pop3 connections to 7 children and 4
connections per IP per minute. Excess connections are logged to
/var/log/messages (and console.log if enabled) thus:

May 21 12:31:59 sola inetd[9671]: pop3 from 182.118.103.211 exceeded counts/min (limit 4/min)
May 21 14:21:51 sola inetd[9671]: pop3 from 182.118.99.168 exceeded counts/min (limit 4/min)
May 21 14:21:52 sola inetd[9671]: pop3 from 182.118.99.168 exceeded counts/min (limit 4/min)
May 21 14:26:40 sola inetd[9671]: pop3 from 182.117.230.117 exceeded counts/min (limit 4/min)
May 21 15:34:53 sola inetd[9671]: pop3 from 182.117.207.48 exceeded counts/min (limit 4/min)
May 21 16:26:56 sola inetd[9671]: pop3 from 182.117.226.184 exceeded counts/min (limit 4/min)

You could run a script to tail messages hunting for such lines, then add
the IP to a table if you want; for example I run a script that instantly
bans GET requests for certain strings to any of a number of webservers.
I also tend to check logs and hand-add naughty nets such as the above to
a block table, never to be seen again ..

I also use not dissimilar connection limits to sendmail's MTA, but
that's done in sendmail's own configuration.

Others may know better ways to deal specifically with HTTP connections?
Post by Will Squire
(P.S. The freebsd-ipfw list seems to be for development of the
technology only, so asking this here. Please let me know if this
isn?t the case)
It's usually fairly low volume and noone seems to mind usage questions,
though the developers usually tend to let these go by.

cheers, Ian
Shane Ambler
2016-05-29 13:20:11 UTC
Permalink
Post by Will Squire
Can ipfw limit the number requests in a given amount of time from a
specific IP?
To contextualise, if an IP sends requests in high concurrency (let's
say 50 a second) can ipfw either block requests the exceed a
threshold for that second (lets say the threshold is 20, 30 would be
blocked), or ban/deny the given IP for exceeding a threshold?
The aim is to lessen strain under DoS attacks, specifically for HTTP.
The system is using Apache and mod_evasive has been added and tested,
but it is not functioning correctly.
(P.S. The freebsd-ipfw list seems to be for development of the
technology only, so asking this here. Please let me know if this
isn’t the case)
You might want to look at sshguard

http://www.freshports.org/security/sshguard-ipfw/

http://www.sshguard.net/
--
FreeBSD - the place to B...Software Developing

Shane Ambler
Will Squire
2016-05-31 18:28:59 UTC
Permalink
Post by Ian Smith
In freebsd-questions Digest, Vol 625, Issue 7, Message: 3
(please wrap lines < 80 columns if possible)
Thanks, will do.
Post by Ian Smith
Post by Will Squire
Can ipfw limit the number requests in a given amount of time from a
specific IP?
To contextualise, if an IP sends requests in high concurrency (let's
say 50 a second) can ipfw either block requests the exceed a
threshold for that second (lets say the threshold is 20, 30 would be
blocked), or ban/deny the given IP for exceeding a threshold?
Not as such. If you know the specific IP address (or range, or subnet)
you can use stateful rules with 'limit' instead of 'keep-state' to limit
the maximum number of concurrent connections to the port/s configured in
a given rule; see ipfw(8). You cauld use a table of addresses to block
or limit rather than hard-coding them into rule/s.
Thanks for the reply Ian. I don’t think limit would work due to HTTP’s
“keep-alive” feature. I believe this means a connection would be kept open
(counting as one connection) and still open to heavy polling by the client.
Post by Ian Smith
While this is very useful for avoiding DoS of any particular service, it
does not allow you to specify a rate, nor time limit, nor (directly) to
block an IP address that's exceeding the given number of connections.
Post by Will Squire
The aim is to lessen strain under DoS attacks, specifically for HTTP.
The system is using Apache and mod_evasive has been added and tested,
but it is not functioning correctly.
I haven't used (nor heard of) mod_evasive so can't comment on that, but
limiting the total number of connections open to a given service can
certainly mitigate the effect of such DoS attacks.
Again, I think keep-alive might cause issues here (but please do correct me if
wrong). Limiting connection to the HTTP service might also worsen the DoS to
users.
Post by Ian Smith
You could of course use /etc/inetd.conf (aka TCPwrappers) to limit
connections in just the ways you want, though I'm not sure starting HTTP
connections in that way is recommended these days. I use if for FTP and
sola# grep -v '#' /etc/inetd.conf
ftp stream tcp nowait/7/3 root /usr/libexec/ftpd ftpd -dll -S
pop3 stream tcp nowait/7/4 root /usr/local/libexec/qpopper qpopper -s -T 120
{wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]]
The above example limits pop3 connections to 7 children and 4
connections per IP per minute. Excess connections are logged to
May 21 12:31:59 sola inetd[9671]: pop3 from 182.118.103.211 exceeded counts/min (limit 4/min)
May 21 14:21:51 sola inetd[9671]: pop3 from 182.118.99.168 exceeded counts/min (limit 4/min)
May 21 14:21:52 sola inetd[9671]: pop3 from 182.118.99.168 exceeded counts/min (limit 4/min)
May 21 14:26:40 sola inetd[9671]: pop3 from 182.117.230.117 exceeded counts/min (limit 4/min)
May 21 15:34:53 sola inetd[9671]: pop3 from 182.117.207.48 exceeded counts/min (limit 4/min)
May 21 16:26:56 sola inetd[9671]: pop3 from 182.117.226.184 exceeded counts/min (limit 4/min)
You could run a script to tail messages hunting for such lines, then add
the IP to a table if you want; for example I run a script that instantly
bans GET requests for certain strings to any of a number of webservers.
I also tend to check logs and hand-add naughty nets such as the above to
a block table, never to be seen again ..
I’m not familiar with using TCPwrappers, Have seen another recommend
SSHGuard though (which I am using already). Can I do something similar
with that, or does/should it do this (add to ban table) automatically? Unsure
if SSHGuard needs any additional rules written for Apache.
Post by Ian Smith
I also use not dissimilar connection limits to sendmail's MTA, but
that's done in sendmail's own configuration.
Others may know better ways to deal specifically with HTTP connections?
Post by Will Squire
(P.S. The freebsd-ipfw list seems to be for development of the
technology only, so asking this here. Please let me know if this
isn?t the case)
It's usually fairly low volume and noone seems to mind usage questions,
though the developers usually tend to let these go by.
cheers, Ian
Thanks

Kind regards,
Will Squire
Will Squire
2016-05-31 19:30:15 UTC
Permalink
Post by Shane Ambler
Post by Will Squire
Can ipfw limit the number requests in a given amount of time from a
specific IP?
To contextualise, if an IP sends requests in high concurrency (let's
say 50 a second) can ipfw either block requests the exceed a
threshold for that second (lets say the threshold is 20, 30 would be
blocked), or ban/deny the given IP for exceeding a threshold?
The aim is to lessen strain under DoS attacks, specifically for HTTP.
The system is using Apache and mod_evasive has been added and tested,
but it is not functioning correctly.
(P.S. The freebsd-ipfw list seems to be for development of the
technology only, so asking this here. Please let me know if this
isn’t the case)
You might want to look at sshguard
http://www.freshports.org/security/sshguard-ipfw/
http://www.sshguard.net/
Thanks Shane. Do you have any examples of how to implement
this with Apache? Already have SSHGuard installed, so would
be ideal if I can extend the functionality to the Apache service.

Did a bit of googling on this before, but didn't find any
standard instructions (simple ones for simple minds) between
different sources and think perhaps I’ve missed something? I didn’t
add the rule `ipfw add deny all from 'table(22)' to any` that the
port’s description specified because I read somewhere else
it wasn’t necessary…

And it also differed to this example documentation for ipfw (which I
found confusing):
http://www.sshguard.net/docs/setup/ <http://www.sshguard.net/docs/setup/>

So, I’m thinking I either need to add the following to `/etc/rc.conf`:
`sshguard_watch_logs=“/var/log/auth.log:/var/log/maillog:/var/log/httpd-access.log”`
to get SSHGuard to poll the Apache log file, or I can pipe the
Apache logs directly into SSHGaurd like the syslogd example
(but I’m not sure how to do this one, or which one is best)? And do
I need to go back and setup some ipfw rules for SSHGuard to
work properly?

Thank you
Post by Shane Ambler
--
FreeBSD - the place to B...Software Developing
Shane Ambler
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
Kind regards,
Will Squire
Ian Smith
2016-06-01 16:20:48 UTC
Permalink
[..]
Post by Ian Smith
Post by Will Squire
Can ipfw limit the number requests in a given amount of time from a
specific IP?
To contextualise, if an IP sends requests in high concurrency (let's
say 50 a second) can ipfw either block requests the exceed a
threshold for that second (lets say the threshold is 20, 30 would be
blocked), or ban/deny the given IP for exceeding a threshold?
you can use stateful rules with 'limit' instead of 'keep-state' to limit
Thanks for the reply Ian. I donÿÿt think limit would work due to HTTPÿÿs
ÿÿkeep-aliveÿÿ feature. I believe this means a connection would be kept open
(counting as one connection) and still open to heavy polling by the client.
That's probably right, and your desire to define connection rate limits
rates should have steered me away from suggesting 'limit'. inetd.conf
is careful not to mention http, so that was a dumb sugdestion too :(
Post by Ian Smith
Post by Will Squire
The aim is to lessen strain under DoS attacks, specifically for HTTP.
The system is using Apache and mod_evasive has been added and tested,
but it is not functioning correctly.
From what I've just dug up on a quick hunt, I think that's what you
should get working. It looks very useful if it works as advertised.

Running a command, more likely script, that invokes ipfw to add notified
addresses (with a timestamp, possibly set to some future time ($now +
$interval)?) to an ipfw table is easy; I have several scripts that deal
with this. A cron script to periodically find and delete the expired
entries is also straightforward.

However ipfw must be run as root, so you'll need some signalling so a
script run by www-user (ono) can inform a root shell process to run ipfw
commands and return status to your script. Good fun .. with some care.
Post by Ian Smith
I haven't used (nor heard of) mod_evasive so can't comment on that, but
limiting the total number of connections open to a given service can
certainly mitigate the effect of such DoS attacks.
Again, I think keep-alive might cause issues here (but please do correct me if
wrong). Limiting connection to the HTTP service might also worsen the DoS to
users.
Yes, I believe you're right. Even with the webserver returning 403s,
it's still an access.log DoS .. so the firewall is the tool to block
those deemed baddies, and mod_evasive looks like a good tool for the
deeming job .. just some hits searching 'mod_evasive apache freebsd':

https://project.altservice.com/issues/562
https://www.unixmen.com/protecting-apache-server-denial-service-dos-attack/
https://www.digitalocean.com/community/tutorials/how-to-protect-against-dos-and-ddos-with-mod_evasive-for-apache-on-centos-7
and of course
https://www.freshports.org/www/mod_evasive/

[.. chomping tcpwrappers stuff ..]
Iÿÿm not familiar with using TCPwrappers, Have seen another recommend
SSHGuard though (which I am using already). Can I do something similar
with that, or does/should it do this (add to ban table) automatically? Unsure
if SSHGuard needs any additional rules written for Apache.
I haven't used SSHGuard.

Perhaps if you share your mod_evasive config and a problem description,
someone here may be able to help? If so, I can offer ipfw script ideas.

cheers, Ian
Michael Sierchio
2016-05-31 22:00:32 UTC
Permalink
You can use dummynet pipes (and optionally queues) with a mask for this.
You can specify queue size in slots. You can also make use of red/gred (see
the manpage).

I might choose a larger mask than 0xffffffff because a new pipe will be
instantiated for each matching address.

a very simple example

ipfw pipe 1 config bw 800Byte/s mask src-ip 0xffffc000
ipfw add pipe 1 tcp from any to me ssh in recv $OIF setup

- M
Post by Will Squire
Can ipfw limit the number requests in a given amount of time from a
specific IP?
Post by Will Squire
To contextualise, if an IP sends requests in high concurrency (let's say
50 a second) can ipfw either block requests the exceed a threshold for that
second (lets say the threshold is 20, 30 would be blocked), or ban/deny the
given IP for exceeding a threshold?
Post by Will Squire
The aim is to lessen strain under DoS attacks, specifically for HTTP. The
system is using Apache and mod_evasive has been added and tested, but it is
not functioning correctly.
Post by Will Squire
(P.S. The freebsd-ipfw list seems to be for development of the technology
only, so asking this here. Please let me know if this isn’t the case)
Post by Will Squire
Thanks
Kind regards,
Will Squire
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "
freebsd-questions-***@freebsd.org"

Loading...