Welcome! Log In Create A New Profile

Advanced

kill -HUP failing (0.6.32)

Posted by Guy Naor 
Guy Naor
kill -HUP failing (0.6.32)
October 11, 2008 10:10PM
Hi,

I have an nginx setup as a reverse proxy into a large number of Rails and
Java applications. As it's a very dynamic system, nginx incudes a whole
directory of conf files, one for each application.

nginx works really well, but now that we have about 320 different config
files nginx stopped responding to kill -HUP for reloading the conf files. It
just ignores the request and nothing happens. My test shows that at 247
files it stop responding to HUP. Up to that number it's all ok and HUP works
perfectly. The files are all pretty much the same, with this format:


# Definition of the different compute instances we proxy into
upstream redmine_nginx {
server 10.252.179.143:5150;
}

# vhost definition for this application
server {
listen 80;
client_max_body_size 10M;
client_body_buffer_size 128k;
server_name redmine.morphexchange.com ; # Add the redirected domains here
(like: www.gadgets.com that is redirected to gadgets.mex.com)
set $original_uri $uri;
access_log /mnt/nginx/logs/redmine/access.log main;
error_log /mnt/nginx/logs/redmine/error.log warn;
location ~* ^.*/(themes|stylesheets|javascripts|images)(/.*$|$) {
proxy_pass http://redmine_nginx;
proxy_set_header X-Real-IP $http_x_source_address;
proxy_set_header Host $host;
}
location / {
proxy_set_header X-Real-IP $http_x_source_address;
proxy_intercept_errors on;
proxy_set_header Host $host;
error_page 400 401 403 404 414 500 501 502 503 504 505 506 507 508 509
510 = @rails_redmine;
if ($request_method = POST ) {
proxy_pass http://redmine_nginx;
break;
}
if ($is_args = "?" ) {
proxy_pass http://redmine_nginx;
break;
}

}
location @rails_redmine {
rewrite (.*) $original_uri break;
proxy_pass http://redmine_nginx;
proxy_set_header X-Real-IP $http_x_source_address;
proxy_set_header Host $host;
}
}

I also tried to increase the server hash size and bucket size but it didn't
help.

Looking at the code, there are a lot of conditions that might result in HUP
failing, but those are not logged. Any idea what might be causing this? The
same config with less files has no problem with the HUP signal.


Regards,

Guy Naor.

--
Guy Naor
CTO
Morph Labs, Inc.
guy@mor.ph
www.morphexchange.com
Igor Sysoev
Re: kill -HUP failing (0.6.32)
October 11, 2008 10:10PM
On Fri, Oct 10, 2008 at 05:48:42AM -0700, Guy Naor wrote:

> I have an nginx setup as a reverse proxy into a large number of Rails and
> Java applications. As it's a very dynamic system, nginx incudes a whole
> directory of conf files, one for each application.
>
> nginx works really well, but now that we have about 320 different config
> files nginx stopped responding to kill -HUP for reloading the conf files. It
> just ignores the request and nothing happens. My test shows that at 247
> files it stop responding to HUP. Up to that number it's all ok and HUP works
> perfectly. The files are all pretty much the same, with this format:
>
>
> # Definition of the different compute instances we proxy into
> upstream redmine_nginx {
> server 10.252.179.143:5150;
> }
>
> # vhost definition for this application
> server {
> listen 80;
> client_max_body_size 10M;
> client_body_buffer_size 128k;
> server_name redmine.morphexchange.com ; # Add the redirected domains here
> (like: www.gadgets.com that is redirected to gadgets.mex.com)
> set $original_uri $uri;
> access_log /mnt/nginx/logs/redmine/access.log main;
> error_log /mnt/nginx/logs/redmine/error.log warn;
> location ~* ^.*/(themes|stylesheets|javascripts|images)(/.*$|$) {
> proxy_pass http://redmine_nginx;
> proxy_set_header X-Real-IP $http_x_source_address;
> proxy_set_header Host $host;
> }
> location / {
> proxy_set_header X-Real-IP $http_x_source_address;
> proxy_intercept_errors on;
> proxy_set_header Host $host;
> error_page 400 401 403 404 414 500 501 502 503 504 505 506 507 508 509
> 510 = @rails_redmine;
> if ($request_method = POST ) {
> proxy_pass http://redmine_nginx;
> break;
> }
> if ($is_args = "?" ) {
> proxy_pass http://redmine_nginx;
> break;
> }
>
> }
> location @rails_redmine {
> rewrite (.*) $original_uri break;
> proxy_pass http://redmine_nginx;
> proxy_set_header X-Real-IP $http_x_source_address;
> proxy_set_header Host $host;
> }
> }
>
> I also tried to increase the server hash size and bucket size but it didn't
> help.
>
> Looking at the code, there are a lot of conditions that might result in HUP
> failing, but those are not logged. Any idea what might be causing this? The
> same config with less files has no problem with the HUP signal.

What does "nginx -t" show ?


--
Igor Sysoev
http://sysoev.ru/en/
Kenneth Demanawa
Re: kill -HUP failing (0.6.32)
October 11, 2008 10:10PM
(Responding in behalf of Guy. :) )

Hi Igor.

"nginx -t" shows:

2008/10/10 14:43:32 [info] 23171#0: the configuration file
../nginx.conf syntax is ok
2008/10/10 14:43:32 [info] 23171#0: the configuration file
../nginx.conf was tested successfully

Kenneth

On Fri, Oct 10, 2008 at 8:58 PM, Igor Sysoev <[email protected]> wrote:
> On Fri, Oct 10, 2008 at 05:48:42AM -0700, Guy Naor wrote:
>
>> I have an nginx setup as a reverse proxy into a large number of Rails and
>> Java applications. As it's a very dynamic system, nginx incudes a whole
>> directory of conf files, one for each application.
>>
>> nginx works really well, but now that we have about 320 different config
>> files nginx stopped responding to kill -HUP for reloading the conf files. It
>> just ignores the request and nothing happens. My test shows that at 247
>> files it stop responding to HUP. Up to that number it's all ok and HUP works
>> perfectly. The files are all pretty much the same, with this format:
>>
>>
>> # Definition of the different compute instances we proxy into
>> upstream redmine_nginx {
>> server 10.252.179.143:5150;
>> }
>>
>> # vhost definition for this application
>> server {
>> listen 80;
>> client_max_body_size 10M;
>> client_body_buffer_size 128k;
>> server_name redmine.morphexchange.com ; # Add the redirected domains here
>> (like: www.gadgets.com that is redirected to gadgets.mex.com)
>> set $original_uri $uri;
>> access_log /mnt/nginx/logs/redmine/access.log main;
>> error_log /mnt/nginx/logs/redmine/error.log warn;
>> location ~* ^.*/(themes|stylesheets|javascripts|images)(/.*$|$) {
>> proxy_pass http://redmine_nginx;
>> proxy_set_header X-Real-IP $http_x_source_address;
>> proxy_set_header Host $host;
>> }
>> location / {
>> proxy_set_header X-Real-IP $http_x_source_address;
>> proxy_intercept_errors on;
>> proxy_set_header Host $host;
>> error_page 400 401 403 404 414 500 501 502 503 504 505 506 507 508 509
>> 510 = @rails_redmine;
>> if ($request_method = POST ) {
>> proxy_pass http://redmine_nginx;
>> break;
>> }
>> if ($is_args = "?" ) {
>> proxy_pass http://redmine_nginx;
>> break;
>> }
>>
>> }
>> location @rails_redmine {
>> rewrite (.*) $original_uri break;
>> proxy_pass http://redmine_nginx;
>> proxy_set_header X-Real-IP $http_x_source_address;
>> proxy_set_header Host $host;
>> }
>> }
>>
>> I also tried to increase the server hash size and bucket size but it didn't
>> help.
>>
>> Looking at the code, there are a lot of conditions that might result in HUP
>> failing, but those are not logged. Any idea what might be causing this? The
>> same config with less files has no problem with the HUP signal.
>
> What does "nginx -t" show ?
>
>
> --
> Igor Sysoev
> http://sysoev.ru/en/
>
>
Sorry, only registered users may post in this forum.

Click here to login