Welcome! Log In Create A New Profile

Advanced

Multiple Cache Manager Processes or Threads

Posted by traquila 
traquila
Multiple Cache Manager Processes or Threads
November 30, 2017 06:30PM
Hello,

I have an issue with the cache manager and the way I use it.
When I configure 2 different caches zones, one very huge and one very fast,
the cache manager can't delete files quickly enough and lead to a partition
full.

For example:
proxy_cache_path /mnt/hdd/cache levels=1:2:2 keys_zone=cache_hdd:40g
max_size=40000g inactive=5d;
proxy_cache_path /mnt/ram/cache levels=1:2 keys_zone=cache_ram:300m
max_size=300g inactive=1h;

On the beginning, ram cache is correctly purge around 40GB (+/- Input
bandwidth*10sec) , but when the hdd cache begins to fill up, ram cache
growing over 50GB. I think the cache manager is stuck by the slowness of the
filesystem / hardware.

I can fix this by using 2 nginx on the same machine, one configured as ram
cache, the other as hdd cache; but I wonder if it would be possible to
create a cache manager process for each proxy_cache_path directive.

Thank in advance.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,277597,277597#msg-277597

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
traquila
Re: Multiple Cache Manager Processes or Threads
November 30, 2017 06:30PM
Sorry, I gave wrong values:

On the beginning, ram cache is correctly purge around 300GB (+/- Input
bandwidth*10sec) , but when the hdd cache begins to fill up, ram cache
growing over 320GB.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,277597,277598#msg-277598

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Maxim Dounin
Re: Multiple Cache Manager Processes or Threads
November 30, 2017 07:30PM
Hello!

On Thu, Nov 30, 2017 at 12:20:19PM -0500, traquila wrote:

> I have an issue with the cache manager and the way I use it.
> When I configure 2 different caches zones, one very huge and one very fast,
> the cache manager can't delete files quickly enough and lead to a partition
> full.
>
> For example:
> proxy_cache_path /mnt/hdd/cache levels=1:2:2 keys_zone=cache_hdd:40g
> max_size=40000g inactive=5d;
> proxy_cache_path /mnt/ram/cache levels=1:2 keys_zone=cache_ram:300m
> max_size=300g inactive=1h;
>
> On the beginning, ram cache is correctly purge around 40GB (+/- Input
> bandwidth*10sec) , but when the hdd cache begins to fill up, ram cache
> growing over 50GB. I think the cache manager is stuck by the slowness of the
> filesystem / hardware.
>
> I can fix this by using 2 nginx on the same machine, one configured as ram
> cache, the other as hdd cache; but I wonder if it would be possible to
> create a cache manager process for each proxy_cache_path directive.

Which nginx version you are using?

With nginx 1.11.5+, there are manager_files / manager_sleep /
manager_threshold parameters you may want to play with, see
http://nginx.org/r/proxy_cache_path. These parameters allows
limiting cache manager's work on a particular cache to some finite
time, and therefore help to better maintain specified max_size of
other caches.

If you are using an older version, an upgrade to the recent
version might help even without further tuning, as older versions
do not limit cache manager's work on a particular cache at all.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
traquila
Re: Multiple Cache Manager Processes or Threads
December 01, 2017 12:30PM
Thank you for your answer,
I am using an old version (1.8.1).
I will try to upgrade to 1.12 and check if it solve my problem.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,277597,277616#msg-277616

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login