Welcome! Log In Create A New Profile

Advanced

504 gateway timeouts

Posted by Wade Girard 
Wade Girard
504 gateway timeouts
December 27, 2017 11:10PM
I am using nginx on an ubuntu server as a proxy to a tomcat server.

The nginx server is setup for https.

I don't know how to determine what version of nginx I am using, but I
install it on the ubuntu 1.16 server using apt-get.

I have an issue that I have resolved locally on my Mac (using version 1.12
of nginx and Tomcat 7) where requests through the proxy that take more than
60 seconds were failing, they are now working.

What seemed to be the fix was adding the following to the nginx.conf file

proxy_connect_timeout 600;

proxy_send_timeout 600;

proxy_read_timeout 600;

send_timeout 600;

in the location section for my proxy.


However this same change in the ubuntu servers has no effect at all.

The way I am testing this is that I create a request that sleeps the thread
for 5 minutes before retiring a response.

Any help appreciated.

Thanks

--
Wade Girard
c: 612.363.0902
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Igal @ Lucee.org
Re: 504 gateway timeouts
December 27, 2017 11:20PM
On 12/27/2017 2:03 PM, Wade Girard wrote:
> I am using nginx on an ubuntu server as a proxy to a tomcat server.
>
> The nginx server is setup for https.
>
> I don't know how to determine what version of nginx I am using, but I
> install it on the ubuntu 1.16 server using apt-get.
Run:  nginx -v

>
> I have an issue that I have resolved locally on my Mac (using version
> 1.12 of nginx and Tomcat 7) where requests through the proxy that take
> more than 60 seconds were failing, they are now working.
>
> What seemed to be the fix was adding the following to the nginx.conf file
>
> proxy_connect_timeout       600;
>
> proxy_send_timeout          600;
>
> proxy_read_timeout          600;
>
> send_timeout                600;
>
> in the location section for my proxy.
>
>
> However this same change in the ubuntu servers has no effect at all.
>
>
Try to flush out some output early on so that nginx will know that
Tomcat is alive.

Igal Sapir
Lucee Core Developer
Lucee.org http://lucee.org/



_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Wade Girard
Re: 504 gateway timeouts
January 04, 2018 02:20PM
The version that is on the ubuntu servers was 1.10.xx. I just updated it to

nginx version: nginx/1.13.8

And I am still having the same issue.

How do I "Try to flush out some output early on so that nginx will know
that Tomcat is alive."

The nginx and tomcat connection is working fine for all requests/responses
that take less than 60 seconds.

On Wed, Dec 27, 2017 at 4:18 PM, Igal @ Lucee.org <[email protected]> wrote:

> On 12/27/2017 2:03 PM, Wade Girard wrote:
>
> I am using nginx on an ubuntu server as a proxy to a tomcat server.
>
> The nginx server is setup for https.
>
> I don't know how to determine what version of nginx I am using, but I
> install it on the ubuntu 1.16 server using apt-get.
>
> Run: nginx -v
>
>
> I have an issue that I have resolved locally on my Mac (using version 1.12
> of nginx and Tomcat 7) where requests through the proxy that take more than
> 60 seconds were failing, they are now working.
>
> What seemed to be the fix was adding the following to the nginx.conf file
>
> proxy_connect_timeout 600;
>
> proxy_send_timeout 600;
>
> proxy_read_timeout 600;
>
> send_timeout 600;
>
> in the location section for my proxy.
>
>
> However this same change in the ubuntu servers has no effect at all.
>
> Try to flush out some output early on so that nginx will know that Tomcat
> is alive.
>
> Igal Sapir
> Lucee Core Developer
> Lucee.org http://lucee.org/
>
>
>


--
Wade Girard
c: 612.363.0902
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Zhang Chao
Re: 504 gateway timeouts
January 05, 2018 02:50AM
> The version that is on the ubuntu servers was 1.10.xx. I just updated it
to
>
> nginx version: nginx/1.13.8
>
> And I am still having the same issue.
>
> How do I "Try to flush out some output early on so that nginx will know
that Tomcat is alive."
>
> The nginx and tomcat connection is working fine for all
requests/responses that take less t

Maybe you can flush out the HTTP response headers quickly.
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Wade Girard
Re: 504 gateway timeouts
January 05, 2018 05:50AM
I am not sure what is meant by this or what action you are asking me to
take. The settings, when added to nginx conf file on Mac OS server and
nginx reloaded take effect immediately and work as expected, the same
settings when added to nginx conf file on Ubuntu and nginx reloaded have no
effect at all. What steps can I take to have the proxy in nginx honor these
timeouts, or what other settings/actions can I take to make this work?

Thanks

On Thu, Jan 4, 2018 at 7:46 PM, Zhang Chao <[email protected]> wrote:

> > The version that is on the ubuntu servers was 1.10.xx. I just updated it
> to
> >
> > nginx version: nginx/1.13.8
> >
> > And I am still having the same issue.
> >
> > How do I "Try to flush out some output early on so that nginx will know
> that Tomcat is alive."
> >
> > The nginx and tomcat connection is working fine for all
> requests/responses that take less t
>
> Maybe you can flush out the HTTP response headers quickly.
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>



--
Wade Girard
c: 612.363.0902
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Peter Booth
Re: 504 gateway timeouts
January 05, 2018 08:40AM
Wade,

I think that you are asking “hey why isn’t nginx behaving identically on MacOS and Linux when create a servlet that invokes Thread.sleep(300000) before it returns a response?.”

Am I reading you correctly?

A flippant response would be to say: “because OS/X and Linux are different OSes that behave differently”

It would probably help us if you explained a little more about your test, why the sleep is there and what your goals are?


Peter

> On Jan 4, 2018, at 11:45 PM, Wade Girard <[email protected]> wrote:
>
> I am not sure what is meant by this or what action you are asking me to take. The settings, when added to nginx conf file on Mac OS server and nginx reloaded take effect immediately and work as expected, the same settings when added to nginx conf file on Ubuntu and nginx reloaded have no effect at all. What steps can I take to have the proxy in nginx honor these timeouts, or what other settings/actions can I take to make this work?
>
> Thanks
>
> On Thu, Jan 4, 2018 at 7:46 PM, Zhang Chao <[email protected] <mailto:[email protected]>> wrote:
> > The version that is on the ubuntu servers was 1.10.xx. I just updated it to
> >
> > nginx version: nginx/1.13.8
> >
> > And I am still having the same issue.
> >
> > How do I "Try to flush out some output early on so that nginx will know that Tomcat is alive."
> >
> > The nginx and tomcat connection is working fine for all requests/responses that take less t
>
> Maybe you can flush out the HTTP response headers quickly.
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org <mailto:[email protected]>
> http://mailman.nginx.org/mailman/listinfo/nginx http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
> --
> Wade Girard
> c: 612.363.0902
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Wade Girard
Re: 504 gateway timeouts
January 05, 2018 01:30PM
Hi Peter,

Thank You.

In my servlet I am making https requests to third party vendors to get data
from them. The requests typically take 4~5 seconds, but every now any then
one of the requests will take more than 60 seconds. So the connection from
the client to nginx to tomcat will remain open, and at 60 seconds nginx is
terminating the request to tomcat, even though the connection from the
third party server to tomcat is still open.

I am also working with the third party vendor to have them see why their
connections sometimes take more than 60 seconds.

Through googling I discovered that adding the settings proxy_send_timeout,
proxy_read_timeout, proxy_connection_timeout, etc... to my location
definition in my conf file could change the timeout to be different
(higher) than the apparent default 60 second timeout. I use a Mac for
development. I added these to my local conf file, and added the long
connection request to test if the settings worked. They did. However they
do not have the same effect for nginx installed on my production Ubuntu
16.x servers. I did not realize that these settings were limited by the OS
that nginx is installed on. Are there are similar settings that will work
for the Ubuntu 16.x OS to achieve the same result?

Wade

On Fri, Jan 5, 2018 at 1:33 AM, Peter Booth <[email protected]> wrote:

> Wade,
>
> I think that you are asking “hey why isn’t nginx behaving identically on
> MacOS and Linux when create a servlet that invokes Thread.sleep(300000)
> before it returns a response?*.”*
>
> Am I reading you correctly?
>
> A flippant response would be to say: “because OS/X and Linux are different
> OSes that behave differently”
>
> It would probably help us if you explained a little more about your test,
> why the sleep is there and what your goals are?
>
>
> Peter
>
>
> On Jan 4, 2018, at 11:45 PM, Wade Girard <[email protected]> wrote:
>
> I am not sure what is meant by this or what action you are asking me to
> take. The settings, when added to nginx conf file on Mac OS server and
> nginx reloaded take effect immediately and work as expected, the same
> settings when added to nginx conf file on Ubuntu and nginx reloaded have no
> effect at all. What steps can I take to have the proxy in nginx honor these
> timeouts, or what other settings/actions can I take to make this work?
>
> Thanks
>
> On Thu, Jan 4, 2018 at 7:46 PM, Zhang Chao <[email protected]> wrote:
>
>> > The version that is on the ubuntu servers was 1.10.xx. I just updated
>> it to
>> >
>> > nginx version: nginx/1.13.8
>> >
>> > And I am still having the same issue.
>> >
>> > How do I "Try to flush out some output early on so that nginx will know
>> that Tomcat is alive."
>> >
>> > The nginx and tomcat connection is working fine for all
>> requests/responses that take less t
>>
>> Maybe you can flush out the HTTP response headers quickly.
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>
>
>
> --
> Wade Girard
> c: 612.363.0902 <(612)%20363-0902>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>



--
Wade Girard
c: 612.363.0902
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Wade Girard
Re: 504 gateway timeouts
January 09, 2018 08:00PM
Hi nginx group,

If anyone has any ides on this, they would be appreciated.

Thanks

On Fri, Jan 5, 2018 at 6:28 AM, Wade Girard <[email protected]> wrote:

> Hi Peter,
>
> Thank You.
>
> In my servlet I am making https requests to third party vendors to get
> data from them. The requests typically take 4~5 seconds, but every now any
> then one of the requests will take more than 60 seconds. So the connection
> from the client to nginx to tomcat will remain open, and at 60 seconds
> nginx is terminating the request to tomcat, even though the connection from
> the third party server to tomcat is still open.
>
> I am also working with the third party vendor to have them see why their
> connections sometimes take more than 60 seconds.
>
> Through googling I discovered that adding the settings proxy_send_timeout,
> proxy_read_timeout, proxy_connection_timeout, etc... to my location
> definition in my conf file could change the timeout to be different
> (higher) than the apparent default 60 second timeout. I use a Mac for
> development. I added these to my local conf file, and added the long
> connection request to test if the settings worked. They did. However they
> do not have the same effect for nginx installed on my production Ubuntu
> 16.x servers. I did not realize that these settings were limited by the OS
> that nginx is installed on. Are there are similar settings that will work
> for the Ubuntu 16.x OS to achieve the same result?
>
> Wade
>
> On Fri, Jan 5, 2018 at 1:33 AM, Peter Booth <[email protected]> wrote:
>
>> Wade,
>>
>> I think that you are asking “hey why isn’t nginx behaving identically on
>> MacOS and Linux when create a servlet that invokes Thread.sleep(300000)
>> before it returns a response?*.”*
>>
>> Am I reading you correctly?
>>
>> A flippant response would be to say: “because OS/X and Linux are
>> different OSes that behave differently”
>>
>> It would probably help us if you explained a little more about your test,
>> why the sleep is there and what your goals are?
>>
>>
>> Peter
>>
>>
>> On Jan 4, 2018, at 11:45 PM, Wade Girard <[email protected]> wrote:
>>
>> I am not sure what is meant by this or what action you are asking me to
>> take. The settings, when added to nginx conf file on Mac OS server and
>> nginx reloaded take effect immediately and work as expected, the same
>> settings when added to nginx conf file on Ubuntu and nginx reloaded have no
>> effect at all. What steps can I take to have the proxy in nginx honor these
>> timeouts, or what other settings/actions can I take to make this work?
>>
>> Thanks
>>
>> On Thu, Jan 4, 2018 at 7:46 PM, Zhang Chao <[email protected]> wrote:
>>
>>> > The version that is on the ubuntu servers was 1.10.xx. I just updated
>>> it to
>>> >
>>> > nginx version: nginx/1.13.8
>>> >
>>> > And I am still having the same issue.
>>> >
>>> > How do I "Try to flush out some output early on so that nginx will
>>> know that Tomcat is alive."
>>> >
>>> > The nginx and tomcat connection is working fine for all
>>> requests/responses that take less t
>>>
>>> Maybe you can flush out the HTTP response headers quickly.
>>>
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>>
>>
>> --
>> Wade Girard
>> c: 612.363.0902 <(612)%20363-0902>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>
>
>
> --
> Wade Girard
> c: 612.363.0902 <(612)%20363-0902>
>



--
Wade Girard
c: 612.363.0902
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Peter Booth
Re: 504 gateway timeouts
January 09, 2018 10:00PM
Wade,

This reminds me of something I once saw with an application that was making web service requests to FedEx. So are you saying that the response times are bimodal? That you either get a remote response within a few seconds or the request takes more than 60 seconds, and that you have no 20sec,30sec,40sec requests?

And, if so, do those 60+ sec requests ever get a healthy response?


Sent from my iPhone

> On Jan 9, 2018, at 1:52 PM, Wade Girard <[email protected]> wrote:
>
> Hi nginx group,
>
> If anyone has any ides on this, they would be appreciated.
>
> Thanks
>
>> On Fri, Jan 5, 2018 at 6:28 AM, Wade Girard <[email protected]> wrote:
>> Hi Peter,
>>
>> Thank You.
>>
>> In my servlet I am making https requests to third party vendors to get data from them. The requests typically take 4~5 seconds, but every now any then one of the requests will take more than 60 seconds. So the connection from the client to nginx to tomcat will remain open, and at 60 seconds nginx is terminating the request to tomcat, even though the connection from the third party server to tomcat is still open.
>>
>> I am also working with the third party vendor to have them see why their connections sometimes take more than 60 seconds.
>>
>> Through googling I discovered that adding the settings proxy_send_timeout, proxy_read_timeout, proxy_connection_timeout, etc... to my location definition in my conf file could change the timeout to be different (higher) than the apparent default 60 second timeout. I use a Mac for development. I added these to my local conf file, and added the long connection request to test if the settings worked. They did. However they do not have the same effect for nginx installed on my production Ubuntu 16.x servers. I did not realize that these settings were limited by the OS that nginx is installed on. Are there are similar settings that will work for the Ubuntu 16.x OS to achieve the same result?
>>
>> Wade
>>
>>> On Fri, Jan 5, 2018 at 1:33 AM, Peter Booth <[email protected]> wrote:
>>> Wade,
>>>
>>> I think that you are asking “hey why isn’t nginx behaving identically on MacOS and Linux when create a servlet that invokes Thread.sleep(300000) before it returns a response?.”
>>>
>>> Am I reading you correctly?
>>>
>>> A flippant response would be to say: “because OS/X and Linux are different OSes that behave differently”
>>>
>>> It would probably help us if you explained a little more about your test, why the sleep is there and what your goals are?
>>>
>>>
>>> Peter
>>>
>>>
>>>> On Jan 4, 2018, at 11:45 PM, Wade Girard <[email protected]> wrote:
>>>>
>>>> I am not sure what is meant by this or what action you are asking me to take. The settings, when added to nginx conf file on Mac OS server and nginx reloaded take effect immediately and work as expected, the same settings when added to nginx conf file on Ubuntu and nginx reloaded have no effect at all. What steps can I take to have the proxy in nginx honor these timeouts, or what other settings/actions can I take to make this work?
>>>>
>>>> Thanks
>>>>
>>>>> On Thu, Jan 4, 2018 at 7:46 PM, Zhang Chao <[email protected]> wrote:
>>>>> > The version that is on the ubuntu servers was 1.10.xx. I just updated it to
>>>>> >
>>>>> > nginx version: nginx/1.13.8
>>>>> >
>>>>> > And I am still having the same issue.
>>>>> >
>>>>> > How do I "Try to flush out some output early on so that nginx will know that Tomcat is alive."
>>>>> >
>>>>> > The nginx and tomcat connection is working fine for all requests/responses that take less t
>>>>>
>>>>> Maybe you can flush out the HTTP response headers quickly.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx@nginx.org
>>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>>
>>>>
>>>>
>>>> --
>>>> Wade Girard
>>>> c: 612.363.0902
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx@nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>>
>> --
>> Wade Girard
>> c: 612.363.0902
>
>
>
> --
> Wade Girard
> c: 612.363.0902
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Aziz Rozyev
Re: 504 gateway timeouts
January 10, 2018 01:30AM
Hi Wade,

At least provide the access/error log fragments, curl -ivvv <..> outputs directly to the 3rd party service and via the nginx,
jmeter (if you use that) outputs would make sense. Also, it would be nice to compare nginx configurations from the mac and linux.

Currently it’s barely possible to conclude something relevant about the issue, having exactly the same nginx configurations,
and getting different results looks rather strange.

Hint: change ‘error_log’ directive’s level to ‘info’, check error/access logs, gather tcpdump. And if that won’t make the problem
clearer, start nginx in debug mode. this article should be useful: https://www.nginx.com/resources/admin-guide/debug/



br,
Aziz.





> On 9 Jan 2018, at 23:56, Peter Booth <[email protected]> wrote:
>
> Wade,
>
> This reminds me of something I once saw with an application that was making web service requests to FedEx. So are you saying that the response times are bimodal? That you either get a remote response within a few seconds or the request takes more than 60 seconds, and that you have no 20sec,30sec,40sec requests?
>
> And, if so, do those 60+ sec requests ever get a healthy response?
>
>
> Sent from my iPhone
>
> On Jan 9, 2018, at 1:52 PM, Wade Girard <[email protected]> wrote:
>
>> Hi nginx group,
>>
>> If anyone has any ides on this, they would be appreciated.
>>
>> Thanks
>>
>> On Fri, Jan 5, 2018 at 6:28 AM, Wade Girard <[email protected]> wrote:
>> Hi Peter,
>>
>> Thank You.
>>
>> In my servlet I am making https requests to third party vendors to get data from them. The requests typically take 4~5 seconds, but every now any then one of the requests will take more than 60 seconds. So the connection from the client to nginx to tomcat will remain open, and at 60 seconds nginx is terminating the request to tomcat, even though the connection from the third party server to tomcat is still open.
>>
>> I am also working with the third party vendor to have them see why their connections sometimes take more than 60 seconds.
>>
>> Through googling I discovered that adding the settings proxy_send_timeout, proxy_read_timeout, proxy_connection_timeout, etc... to my location definition in my conf file could change the timeout to be different (higher) than the apparent default 60 second timeout. I use a Mac for development. I added these to my local conf file, and added the long connection request to test if the settings worked. They did. However they do not have the same effect for nginx installed on my production Ubuntu 16.x servers. I did not realize that these settings were limited by the OS that nginx is installed on. Are there are similar settings that will work for the Ubuntu 16.x OS to achieve the same result?
>>
>> Wade
>>
>> On Fri, Jan 5, 2018 at 1:33 AM, Peter Booth <[email protected]> wrote:
>> Wade,
>>
>> I think that you are asking “hey why isn’t nginx behaving identically on MacOS and Linux when create a servlet that invokes Thread.sleep(300000) before it returns a response?.”
>>
>> Am I reading you correctly?
>>
>> A flippant response would be to say: “because OS/X and Linux are different OSes that behave differently”
>>
>> It would probably help us if you explained a little more about your test, why the sleep is there and what your goals are?
>>
>>
>> Peter
>>
>>
>>> On Jan 4, 2018, at 11:45 PM, Wade Girard <[email protected]> wrote:
>>>
>>> I am not sure what is meant by this or what action you are asking me to take. The settings, when added to nginx conf file on Mac OS server and nginx reloaded take effect immediately and work as expected, the same settings when added to nginx conf file on Ubuntu and nginx reloaded have no effect at all. What steps can I take to have the proxy in nginx honor these timeouts, or what other settings/actions can I take to make this work?
>>>
>>> Thanks
>>>
>>> On Thu, Jan 4, 2018 at 7:46 PM, Zhang Chao <[email protected]> wrote:
>>> > The version that is on the ubuntu servers was 1.10.xx. I just updated it to
>>> >
>>> > nginx version: nginx/1.13.8
>>> >
>>> > And I am still having the same issue.
>>> >
>>> > How do I "Try to flush out some output early on so that nginx will know that Tomcat is alive."
>>> >
>>> > The nginx and tomcat connection is working fine for all requests/responses that take less t
>>>
>>> Maybe you can flush out the HTTP response headers quickly.
>>>
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>>
>>>
>>> --
>>> Wade Girard
>>> c: 612.363.0902
>>> _______________________________________________
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>>
>> --
>> Wade Girard
>> c: 612.363.0902
>>
>>
>>
>> --
>> Wade Girard
>> c: 612.363.0902
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login