Welcome! Log In Create A New Profile

Advanced

Suggestion required to add addition memcache capacity without losing the existing memcache data

Posted by subbu 
Hi,

We are on the edge of utilizing 80% of memcache RAM provided to our
application. Hence, we are planning to double the memcache capacity by
adding two additional servers.

At present, we are using 4 memcache servers, the application is using
java spymemcached client by defining an array of four IP and in our
CacheManager.

Some Question bothering us now are….

In case we add two, additional IP’s into existing defined array. Will
this result in invalidated data when new IP’s are added to the list?

Is there a strategy that we can follow to avoid invalidation of cache
that is build over some time on PROD.

Appreciate your suggestions…

Thanks,
Subbu
On Oct 5, 9:51 am, subbu <[email protected]> wrote:
> Hi,
>
> We are on the edge of utilizing 80% of memcache RAM provided to our
> application. Hence, we are planning to double the memcache capacity by
> adding two additional servers.
>
> At present, we are using 4 memcache servers, the application is using
> java spymemcached client by defining an array of four IP and in our
> CacheManager.
>
> Some Question bothering us now are….
>
> In case we add two, additional IP’s into existing defined array. Will
> this result in invalidated data when new IP’s are added to the list?
>
> Is there a strategy that we can follow to avoid invalidation of cache
> that is build over some time on PROD.

That requires some up-front planning. If you're using consistent
hashing, your miss rates won't go up too greatly as you add nodes.

In the not-too-distant future, you'll be able to use vbuckets to do
this in a bit more of a controlled fashion (e.g. invalidate a fixed
percentage of your cache as you migrate it to a new node on your own
time).
Hi Dustin

Appreciate your inputs…

Currently we are using a key pattern {applicationname}.{context}.
{resourceid} which is similar to RESTFul URI convention (examples:
shoppingApps.products.23235) as the key before storing the data into
memcache server.

Do we have any property in memcached server (or at time of
installation), by which we can set a HASHKEY per memcache server. So
that addition or deletion of memcache server will no effect on the
keys and data stored on existing memcache server.

Appreciate your suggestions..

Thanks,
Subbu



On Oct 5, 11:01 pm, Dustin <[email protected]> wrote:
> On Oct 5, 9:51 am, subbu <[email protected]> wrote:
>
>
>
> > Hi,
>
> > We are on the edge of utilizing 80% of memcache RAM provided to our
> > application. Hence, we are planning to double the memcache capacity by
> > adding two additional servers.
>
> > At present, we are using 4 memcache servers, the application is using
> > java spymemcached client by defining an array of four IP and in our
> > CacheManager.
>
> > Some Question bothering us now are….
>
> > In case we add two, additional IP’s into existing defined array. Will
> > this result in invalidated data when new IP’s are added to the list?
>
> > Is there a strategy that we can follow to avoid invalidation of cache
> > that is build over some time on PROD.
>
>   That requires some up-front planning.  If you're using consistent
> hashing, your miss rates won't go up too greatly as you add nodes.
>
>   In the not-too-distant future, you'll be able to use vbuckets to do
> this in a bit more of a controlled fashion (e.g. invalidate a fixed
> percentage of your cache as you migrate it to a new node on your own
> time).
On Oct 6, 5:28 am, subbu <[email protected]> wrote:
> Hi Dustin
>
> Appreciate your inputs…
>
> Currently we are using a key pattern {applicationname}.{context}.
> {resourceid} which is similar to RESTFul URI convention (examples:
> shoppingApps.products.23235) as the key before storing the data into
> memcache server.
>
> Do we have any property in memcached server (or at time of
> installation), by which we can set a HASHKEY per memcache server. So
> that addition or deletion of memcache server will no effect on the
> keys and data stored on existing memcache server.

The memcached server shouldn't do that, but it's a feature of many
clients. I haven't ever actually got around to implementing it,
though.

I believe you could do what you're *asking for* quite easily with a
custom NodeLocator in spymemcached, but it sounds like it could be a
bit fragile. How would you go about deciding which server a given key
lives on? The vbucket strategy is so far the best general purpose
mechanism to do this.
Sorry, only registered users may post in this forum.

Click here to login