Welcome! Log In Create A New Profile

Advanced

nginx limit_req and limit_conn not working to prevent DoS attack

Posted by Phani Sreenivasa Prasad 
Phani Sreenivasa Prasad
nginx limit_req and limit_conn not working to prevent DoS attack
August 02, 2017 03:40AM
Hi All,

I am using nginx in our products. When I run goldeneye DoS attack script
against nginx, it is not able to defend against the attack and normal users
getting impacted.

python goldeneye.py http://<ipaddress>; -w 5 -s 10000 -m random -d

we are using below nginx limit_req options but didnt help. The nginx
documentation says that, these options are used to limit the request rating
limit per key. below is some sample configuration that we tried. The problem
is, when we use these nginx options, it still keeps nginx busy responding
with 503 or some other error code for all those requests beyond the rate
limit . Hence any genuine user when trying to access webserver during the
attack time, not getting chance to access our server and timing out or
getting 500 error.

http {
limit_req_zone $binary_remote_addr zone=one:10m rate=5r/s;

...

server {

limit_req zone=one ;
...

location /sampleurl/ {

}

(Note: also tried limit_conn options and behavior is same).



Why should nginx respond back with any error code rather it should drop
connections !! otherwise it can't protect itself against any DoS attack.
please share the thoughts.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275796,275796#msg-275796

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Hi!

I don’t think just drop the connection is a good idea, client will never
know what happens on the server end.
However, the code 444 may help you, nginx just close the connection in this
case.

On 2 August 2017 at 09:30:01, Phani Sreenivasa Prasad (
nginx-forum@forum.nginx.org) wrote:

Hi All,

I am using nginx in our products. When I run goldeneye DoS attack script
against nginx, it is not able to defend against the attack and normal users
getting impacted.

python goldeneye.py http://<ipaddress>; -w 5 -s 10000 -m random -d

we are using below nginx limit_req options but didnt help. The nginx
documentation says that, these options are used to limit the request rating
limit per key. below is some sample configuration that we tried. The
problem
is, when we use these nginx options, it still keeps nginx busy responding
with 503 or some other error code for all those requests beyond the rate
limit . Hence any genuine user when trying to access webserver during the
attack time, not getting chance to access our server and timing out or
getting 500 error.

http {
limit_req_zone $binary_remote_addr zone=one:10m rate=5r/s;

....

server {

limit_req zone=one ;
....

location /sampleurl/ {

}

(Note: also tried limit_conn options and behavior is same).



Why should nginx respond back with any error code rather it should drop
connections !! otherwise it can't protect itself against any DoS attack.
please share the thoughts.

Posted at Nginx Forum:
https://forum.nginx.org/read.php?2,275796,275796#msg-275796

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Phani Sreenivasa Prasad
Re: nginx limit_req and limit_conn not working to prevent DoS attack
August 02, 2017 04:20AM
I assume it would help dropping connections . since we are setting rate
limit per ip and any client IP which is suspicious by sending requests in
bulk(lets say 10000 connections/requests), it makes sense to not to accept
connections/requests from that IP.

Thoughts ??

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275796,275798#msg-275798

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
You can use an external tool to parse Nginx error log and block the IP in
iptables/netfilter

On Wed, Aug 2, 2017 at 7:43 AM, Phani Sreenivasa Prasad <
nginx-forum@forum.nginx.org> wrote:

> I assume it would help dropping connections . since we are setting rate
> limit per ip and any client IP which is suspicious by sending requests in
> bulk(lets say 10000 connections/requests), it makes sense to not to accept
> connections/requests from that IP.
>
> Thoughts ??
>
> Posted at Nginx Forum: https://forum.nginx.org/read.
> php?2,275796,275798#msg-275798
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>



--
*Anoop P Alias*
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Phani Sreenivasa Prasad
Re: nginx limit_req and limit_conn not working to prevent DoS attack
August 02, 2017 06:10AM
Yes. Firewall would be another option. But before to that, i would like to
try out all options at nginx level if one or other would resolve the issue
at nginx layer itself.

cant we put accept() filters? or
how the deny option works? can we use deny option to not to accept any new
connections if number of connections already exceeds max limit from a client
IP.?
are there any third party modules available for nginx to embed firewall
functionality? something reliable !!

My objective is, using limit_conn directive, when number of connections
exceeding limit, instead of sending 503, or 444, just do not accept any new
connections from that specific IP only(if a client is opening 10000
connections at a time, it should be fine to not accept connections from that
IP citing the reason that it could be malicious).

Thoughts !!

Thanks.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275796,275801#msg-275801

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
The trouble is nginx does a fair amount of work before blocking the IP address, unless things have changed. My recollection is it parses the whole request. Obviously it doesn't send any data. So you are better off blocking with the firewall.

You do need to know your audience. Something related to a university could generate a number of simultaneous users behind one IP. In my case Boeing triggered the limit.


  Original Message  
From: nginx-forum@forum.nginx.org
Sent: August 1, 2017 9:08 PM
To: nginx@nginx.org
Reply-to: nginx@nginx.org
Subject: Re: nginx limit_req and limit_conn not working to prevent DoS attack

Yes. Firewall would be another option. But before to that, i would like to
try out all options at nginx level if one or other would resolve the issue
at nginx layer itself.

cant we put accept() filters? or
how the deny option works? can we use deny option to not to accept any new
connections if number of connections already exceeds max limit from a client
IP.?
are there any third party modules available for nginx to embed firewall
functionality? something reliable !!

My objective is, using limit_conn directive, when number of connections
exceeding limit, instead of sending 503, or 444, just do not accept any new
connections from that specific IP only(if a client is opening 10000
connections at a time, it should be fine to not accept connections from that
IP citing the reason that it could be malicious).

Thoughts !!

Thanks.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275796,275801#msg-275801

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
The original confusion came from the fact you slided away from the basic
mantra of the Unix philosophy stating 'Make each program do one thing well'..

nginx is a Web server, which generalized itself into a stream server. It
serves content and manages access (protects it).
What you are trying to achieve is turning nginx into a firewall, which it
is not.
A content server does not simply cut connections. It behaves and responds
to requests. That is standard.
All you can do at the connection level is limiting their number (cf.
limit_conn).

It has been suggested you used iptables, as it is a firewall. At the
software level, I would rather recommend nftables.
Some log analyzers could help you make the interface between a content
server and a software firewall, such as fail2ban.

You could also go for hardware (D)DoS protection, depending on the scale of
your needs.

​There is nothing to be surprised of, the product you are using merely
doing the job. it has been made for​
---
*B. R.*

On Wed, Aug 2, 2017 at 6:27 AM, Gary Sellani <[email protected]> wrote:

> The trouble is nginx does a fair amount of work before blocking the IP
> address, unless things have changed. My recollection is it parses the whole
> request. Obviously it doesn't send any data. So you are better off blocking
> with the firewall.
>
> You do need to know your audience. Something related to a university could
> generate a number of simultaneous users behind one IP. In my case Boeing
> triggered the limit.
>
>
> Original Message
> From: nginx-forum@forum.nginx.org
> Sent: August 1, 2017 9:08 PM
> To: nginx@nginx.org
> Reply-to: nginx@nginx.org
> Subject: Re: nginx limit_req and limit_conn not working to prevent DoS
> attack
>
> Yes. Firewall would be another option. But before to that, i would like to
> try out all options at nginx level if one or other would resolve the issue
> at nginx layer itself.
>
> cant we put accept() filters? or
> how the deny option works? can we use deny option to not to accept any new
> connections if number of connections already exceeds max limit from a
> client
> IP.?
> are there any third party modules available for nginx to embed firewall
> functionality? something reliable !!
>
> My objective is, using limit_conn directive, when number of connections
> exceeding limit, instead of sending 503, or 444, just do not accept any new
> connections from that specific IP only(if a client is opening 10000
> connections at a time, it should be fine to not accept connections from
> that
> IP citing the reason that it could be malicious).
>
> Thoughts !!
>
> Thanks.
>
> Posted at Nginx Forum: https://forum.nginx.org/read.
> php?2,275796,275801#msg-275801
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
I think the confusion is still there and it is in the term 'firewall'.

While nginx is no good for level 3 firewall, also known as netfilter,
it's perfect for application level firewall and you already mentioned it
by saying that 'it manages access (protects it)'.

People turn nginx into application level firewall, no problem. The
question is rather if you want to pay the price of scrubbing the request
in the userland and not philosophical considerations.

Yet at certain scale and confidence level, indeed a
vertically-integrated solution might be more adequate.

On 02-08-17 10:15, B.R. via nginx wrote:
> The original confusion came from the fact you slided away from the basic
> mantra of the Unix philosophy stating 'Make each program do one thing well'.
>
> nginx is a Web server, which generalized itself into a stream server. It
> serves content and manages access (protects it).
> What you are trying to achieve is turning nginx into a firewall, which
> it is not.
> A content server does not simply cut connections. It behaves and
> responds to requests. That is standard.
> All you can do at the connection level is limiting their number (cf.
> limit_conn).
>
> It has been suggested you used iptables, as it is a firewall. At the
> software level, I would rather recommend nftables.
> Some log analyzers could help you make the interface between a content
> server and a software firewall, such as fail2ban.
>
> You could also go for hardware (D)DoS protection, depending on the scale
> of your needs.
>
> ​There is nothing to be surprised of, the product you are using merely
> doing the job. it has been made for​
> ---
> *B. R.*
>
> On Wed, Aug 2, 2017 at 6:27 AM, Gary Sellani <[email protected]
> <mailto:[email protected]>> wrote:
>
> The trouble is nginx does a fair amount of work before blocking the
> IP address, unless things have changed. My recollection is it parses
> the whole request. Obviously it doesn't send any data. So you are
> better off blocking with the firewall.
>
> You do need to know your audience. Something related to a university
> could generate a number of simultaneous users behind one IP. In my
> case Boeing triggered the limit.
>
>
> Original Message
> From: nginx-forum@forum.nginx.org <mailto:[email protected]>
> Sent: August 1, 2017 9:08 PM
> To: nginx@nginx.org <mailto:[email protected]>
> Reply-to: nginx@nginx.org <mailto:[email protected]>
> Subject: Re: nginx limit_req and limit_conn not working to prevent
> DoS attack
>
> Yes. Firewall would be another option. But before to that, i would
> like to
> try out all options at nginx level if one or other would resolve the
> issue
> at nginx layer itself.
>
> cant we put accept() filters? or
> how the deny option works? can we use deny option to not to accept
> any new
> connections if number of connections already exceeds max limit from
> a client
> IP.?
> are there any third party modules available for nginx to embed firewall
> functionality? something reliable !!
>
> My objective is, using limit_conn directive, when number of connections
> exceeding limit, instead of sending 503, or 444, just do not accept
> any new
> connections from that specific IP only(if a client is opening 10000
> connections at a time, it should be fine to not accept connections
> from that
> IP citing the reason that it could be malicious).
>
> Thoughts !!
>
> Thanks.
>
> Posted at Nginx Forum:
> https://forum.nginx.org/read.php?2,275796,275801#msg-275801
> <https://forum.nginx.org/read.php?2,275796,275801#msg-275801>;
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org <mailto:[email protected]>
> http://mailman.nginx.org/mailman/listinfo/nginx
> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx@nginx.org <mailto:[email protected]>
> http://mailman.nginx.org/mailman/listinfo/nginx
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login