Welcome! Log In Create A New Profile

Advanced

mirror delay

Posted by Eylon Saadon 
Eylon Saadon
mirror delay
August 30, 2018 03:40PM
Hi,
I'm using the mirror module in my "production" nginx in order to mirror
real traffic to a test envrionment.
I don't want this mirroring to affect the latency of the production
environment, but it looks like the nginx is waiting for the response from
the test environment.
is there a way to avoid this? I just want the request to get to the test
environment and let it process it. but it shouldn't wait fo r the response
from the test environment in order to respond to the request

--
Thanks,
Eylon Saadon
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Roman Arutyunyan
Re: mirror delay
August 30, 2018 04:00PM
Hi,

On Thu, Aug 30, 2018 at 04:34:29PM +0300, Eylon Saadon wrote:
> Hi,
> I'm using the mirror module in my "production" nginx in order to mirror
> real traffic to a test envrionment.
> I don't want this mirroring to affect the latency of the production
> environment, but it looks like the nginx is waiting for the response from
> the test environment.
> is there a way to avoid this? I just want the request to get to the test
> environment and let it process it. but it shouldn't wait fo r the response
> from the test environment in order to respond to the request

Usually a mirror subrequest does not affect the main request. However there
are two issues with mirroring:

- the next request on the same connection will not be processed until all
mirror subrequests finish. Try disabling keepalive and see if it helps.

- if you use sendfile and tcp_nopush, it's possible that the response is not
pushed properly because of a mirror subrequest, which may result in a delay.
Turn off sendfile and see if it helps.

--
Roman Arutyunyan
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Eylon Saadon
Re: mirror delay
August 30, 2018 04:30PM
hi,
thanks for the quick response!
I'm not using sendfile or tcp_nopush.
just to make sure. I should disable the keepalive for the mirror location.
and do it like so?

server {

resolver 8.8.8.8;

listen 80;

location / {
proxy_set_header Host $host;
proxy_pass http://server:9000;
}

location /mirror {
internal;
keepalive_timeout 0;
proxy_pass https://test_server$request_uri;
}
}


Thanks,

eylon saadon


On Thu, Aug 30, 2018 at 4:52 PM Roman Arutyunyan <[email protected]> wrote:

> Hi,
>
> On Thu, Aug 30, 2018 at 04:34:29PM +0300, Eylon Saadon wrote:
> > Hi,
> > I'm using the mirror module in my "production" nginx in order to mirror
> > real traffic to a test envrionment.
> > I don't want this mirroring to affect the latency of the production
> > environment, but it looks like the nginx is waiting for the response from
> > the test environment.
> > is there a way to avoid this? I just want the request to get to the test
> > environment and let it process it. but it shouldn't wait fo r the
> response
> > from the test environment in order to respond to the request
>
> Usually a mirror subrequest does not affect the main request. However
> there
> are two issues with mirroring:
>
> - the next request on the same connection will not be processed until all
> mirror subrequests finish. Try disabling keepalive and see if it helps.
>
> - if you use sendfile and tcp_nopush, it's possible that the response is
> not
> pushed properly because of a mirror subrequest, which may result in a
> delay.
> Turn off sendfile and see if it helps.
>
> --
> Roman Arutyunyan
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>


--
Thanks,
Eylon Saadon
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Roman Arutyunyan
Re: mirror delay
August 30, 2018 04:30PM
Hi,

On Thu, Aug 30, 2018 at 05:19:53PM +0300, Eylon Saadon wrote:
> hi,
> thanks for the quick response!
> I'm not using sendfile or tcp_nopush.
> just to make sure. I should disable the keepalive for the mirror location.
> and do it like so?

No, for the primary location. This will help us understand the reason
why you have the delay.

> server {
>
> resolver 8.8.8.8;
>
> listen 80;
>
> location / {
> proxy_set_header Host $host;
> proxy_pass http://server:9000;
> }
>
> location /mirror {
> internal;
> keepalive_timeout 0;
> proxy_pass https://test_server$request_uri;
> }
> }
>
>
> Thanks,
>
> eylon saadon
>
>
> On Thu, Aug 30, 2018 at 4:52 PM Roman Arutyunyan <[email protected]> wrote:
>
> > Hi,
> >
> > On Thu, Aug 30, 2018 at 04:34:29PM +0300, Eylon Saadon wrote:
> > > Hi,
> > > I'm using the mirror module in my "production" nginx in order to mirror
> > > real traffic to a test envrionment.
> > > I don't want this mirroring to affect the latency of the production
> > > environment, but it looks like the nginx is waiting for the response from
> > > the test environment.
> > > is there a way to avoid this? I just want the request to get to the test
> > > environment and let it process it. but it shouldn't wait fo r the
> > response
> > > from the test environment in order to respond to the request
> >
> > Usually a mirror subrequest does not affect the main request. However
> > there
> > are two issues with mirroring:
> >
> > - the next request on the same connection will not be processed until all
> > mirror subrequests finish. Try disabling keepalive and see if it helps.
> >
> > - if you use sendfile and tcp_nopush, it's possible that the response is
> > not
> > pushed properly because of a mirror subrequest, which may result in a
> > delay.
> > Turn off sendfile and see if it helps.
> >
> > --
> > Roman Arutyunyan
> > _______________________________________________
> > nginx mailing list
> > nginx@nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> >
>
>
> --
> Thanks,
> Eylon Saadon

> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx


--
Roman Arutyunyan
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Eylon Saadon
Re: mirror delay
September 02, 2018 04:10PM
HI,
when adding
keepalive_timeout 0;
to the main location it works fine.
even if the mirrored location doesn't respond immediately the latency
doesn't go up.

is this the solution for the issue or just a way to understand the issue?

Thanks,
Eylon Saadon

On Thu, Aug 30, 2018 at 5:28 PM Roman Arutyunyan <[email protected]> wrote:

> Hi,
>
> On Thu, Aug 30, 2018 at 05:19:53PM +0300, Eylon Saadon wrote:
> > hi,
> > thanks for the quick response!
> > I'm not using sendfile or tcp_nopush.
> > just to make sure. I should disable the keepalive for the mirror
> location.
> > and do it like so?
>
> No, for the primary location. This will help us understand the reason
> why you have the delay.
>
> > server {
> >
> > resolver 8.8.8.8;
> >
> > listen 80;
> >
> > location / {
> > proxy_set_header Host $host;
> > proxy_pass http://server:9000;
> > }
> >
> > location /mirror {
> > internal;
> > keepalive_timeout 0;
> > proxy_pass https://test_server$request_uri;
> > }
> > }
> >
> >
> > Thanks,
> >
> > eylon saadon
> >
> >
> > On Thu, Aug 30, 2018 at 4:52 PM Roman Arutyunyan <[email protected]> wrote:
> >
> > > Hi,
> > >
> > > On Thu, Aug 30, 2018 at 04:34:29PM +0300, Eylon Saadon wrote:
> > > > Hi,
> > > > I'm using the mirror module in my "production" nginx in order to
> mirror
> > > > real traffic to a test envrionment.
> > > > I don't want this mirroring to affect the latency of the production
> > > > environment, but it looks like the nginx is waiting for the response
> from
> > > > the test environment.
> > > > is there a way to avoid this? I just want the request to get to the
> test
> > > > environment and let it process it. but it shouldn't wait fo r the
> > > response
> > > > from the test environment in order to respond to the request
> > >
> > > Usually a mirror subrequest does not affect the main request. However
> > > there
> > > are two issues with mirroring:
> > >
> > > - the next request on the same connection will not be processed until
> all
> > > mirror subrequests finish. Try disabling keepalive and see if it
> helps.
> > >
> > > - if you use sendfile and tcp_nopush, it's possible that the response
> is
> > > not
> > > pushed properly because of a mirror subrequest, which may result in a
> > > delay.
> > > Turn off sendfile and see if it helps.
> > >
> > > --
> > > Roman Arutyunyan
> > > _______________________________________________
> > > nginx mailing list
> > > nginx@nginx.org
> > > http://mailman.nginx.org/mailman/listinfo/nginx
> > >
> >
> >
> > --
> > Thanks,
> > Eylon Saadon
>
> > _______________________________________________
> > nginx mailing list
> > nginx@nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> --
> Roman Arutyunyan
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>


--
Thanks,
Eylon Saadon
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Roman Arutyunyan
Re: mirror delay
September 03, 2018 01:10PM
Hi,

On Sun, Sep 02, 2018 at 05:01:19PM +0300, Eylon Saadon wrote:
> HI,
> when adding
> keepalive_timeout 0;
> to the main location it works fine.
> even if the mirrored location doesn't respond immediately the latency
> doesn't go up.
>
> is this the solution for the issue or just a way to understand the issue?

If you are ok with no keepalive for your clients, then you can use this
configuration.

Delayed processing of the next request is a known side-effect of how mirroring
is implemented in nginx, and this is unlikely to change. The point was to
make sure this was actually the case.

> Thanks,
> Eylon Saadon
>
> On Thu, Aug 30, 2018 at 5:28 PM Roman Arutyunyan <[email protected]> wrote:
>
> > Hi,
> >
> > On Thu, Aug 30, 2018 at 05:19:53PM +0300, Eylon Saadon wrote:
> > > hi,
> > > thanks for the quick response!
> > > I'm not using sendfile or tcp_nopush.
> > > just to make sure. I should disable the keepalive for the mirror
> > location.
> > > and do it like so?
> >
> > No, for the primary location. This will help us understand the reason
> > why you have the delay.
> >
> > > server {
> > >
> > > resolver 8.8.8.8;
> > >
> > > listen 80;
> > >
> > > location / {
> > > proxy_set_header Host $host;
> > > proxy_pass http://server:9000;
> > > }
> > >
> > > location /mirror {
> > > internal;
> > > keepalive_timeout 0;
> > > proxy_pass https://test_server$request_uri;
> > > }
> > > }
> > >
> > >
> > > Thanks,
> > >
> > > eylon saadon
> > >
> > >
> > > On Thu, Aug 30, 2018 at 4:52 PM Roman Arutyunyan <[email protected]> wrote:
> > >
> > > > Hi,
> > > >
> > > > On Thu, Aug 30, 2018 at 04:34:29PM +0300, Eylon Saadon wrote:
> > > > > Hi,
> > > > > I'm using the mirror module in my "production" nginx in order to
> > mirror
> > > > > real traffic to a test envrionment.
> > > > > I don't want this mirroring to affect the latency of the production
> > > > > environment, but it looks like the nginx is waiting for the response
> > from
> > > > > the test environment.
> > > > > is there a way to avoid this? I just want the request to get to the
> > test
> > > > > environment and let it process it. but it shouldn't wait fo r the
> > > > response
> > > > > from the test environment in order to respond to the request
> > > >
> > > > Usually a mirror subrequest does not affect the main request. However
> > > > there
> > > > are two issues with mirroring:
> > > >
> > > > - the next request on the same connection will not be processed until
> > all
> > > > mirror subrequests finish. Try disabling keepalive and see if it
> > helps.
> > > >
> > > > - if you use sendfile and tcp_nopush, it's possible that the response
> > is
> > > > not
> > > > pushed properly because of a mirror subrequest, which may result in a
> > > > delay.
> > > > Turn off sendfile and see if it helps.
> > > >
> > > > --
> > > > Roman Arutyunyan
> > > > _______________________________________________
> > > > nginx mailing list
> > > > nginx@nginx.org
> > > > http://mailman.nginx.org/mailman/listinfo/nginx
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > Eylon Saadon
> >
> > > _______________________________________________
> > > nginx mailing list
> > > nginx@nginx.org
> > > http://mailman.nginx.org/mailman/listinfo/nginx
> >
> >
> > --
> > Roman Arutyunyan
> > _______________________________________________
> > nginx mailing list
> > nginx@nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> >
>
>
> --
> Thanks,
> Eylon Saadon

> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx


--
Roman Arutyunyan
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login