Welcome! Log In Create A New Profile

Advanced

memcached-tool enhancements

Posted by Matthew 
Matthew
memcached-tool enhancements
July 12, 2010 08:50PM
Hey all,

Writing again about my changes to memcached-tool - never got a reply
to the first message, but here are the highlights:

1) Search
Added an option to search by key or value
2) Delete
Added an option to delete by key

This is particularly useful on a production system if you want to
delete a bunch of keys with 'XYZ' in the middle of them:

e.g.:

../memcached-tool localhost search key key XYZ | xargs ./memcached-tool
localhost delete
(searches for keys with XYZ in the midst, outputs key) (pipes the
result to delete them)

The search term is a regex, and you can search by key, by value, and
output keys, or values, or both.

I'm open to suggestions or changes - it's available for review here:

http://github.com/mmucklo/memcached/commit/fcbe16eac853bd3f89ad1c38cc69c5b314d09fd5
dormando
Re: memcached-tool enhancements
July 12, 2010 08:50PM
On Mon, 12 Jul 2010, Matthew wrote:

> Hey all,
>
> Writing again about my changes to memcached-tool - never got a reply
> to the first message, but here are the highlights:
>
> 1) Search
> Added an option to search by key or value
> 2) Delete
> Added an option to delete by key
>
> This is particularly useful on a production system if you want to
> delete a bunch of keys with 'XYZ' in the middle of them:
>
> e.g.:
>
> ./memcached-tool localhost search key key XYZ | xargs ./memcached-tool
> localhost delete
> (searches for keys with XYZ in the midst, outputs key) (pipes the
> result to delete them)
>
> The search term is a regex, and you can search by key, by value, and
> output keys, or values, or both.
>
> I'm open to suggestions or changes - it's available for review here:
>
> http://github.com/mmucklo/memcached/commit/fcbe16eac853bd3f89ad1c38cc69c5b314d09fd5

Hey,

Unfortunately those features will be missleading at best and ineffective
at worst; the debug commands it uses are slow and only return a subset of
the keys. So a user may be lead to think that they've found and deleted
all of those keys when they've really only pushed against the tip of it.

I don't think those commands are very fast either; running them on a
highly loaded system often may create lag bubbles.

Believe me, I understand that people want to do this, but it's just the
design tradeoff here. We try hard to only provide features that people
can't easily bite their feet off with, and saying this would be useful for
production folks probably isn't true :(

Any thoughts/rebuttals?

-Dormando
Matthew
Re: memcached-tool enhancements
July 13, 2010 09:30AM
Oh, I didn't know. I'm sorry -

I based search off of dump - Might I suggest a note in the code to
mention that dump is really a 'partial-dump'?

Secondly, though don't you think delete in itself work? The protocol
specification here http://github.com/memcached/memcached/blob/master/doc/protocol.txt
mentions it, and here http://code.google.com/p/memcached/wiki/NewCommands
it says that delete 'Removes an item from the cache, if it exists.'

So in theory, shouldn't the code work, or maybe I'm missing
something?!

As for speed, yes, it's not the fastest thing in the world to search,
but in development, and even on emergency occasion in production,
however, it's been reasonable enough. It's about as fast as dump
would be, so if you would run that in production, search (and delete
which is much faster), should be fine, so if you're providing dump,
search should be about the same speed.

Otherwise by your reasoning, should you not then remove dump from
memcached-tool?

--Matt

On Jul 12, 11:47 am, dormando <[email protected]> wrote:
> On Mon, 12 Jul 2010, Matthew wrote:
> > Hey all,
>
> > Writing again about my changes to memcached-tool - never got a reply
> > to the first message, but here are the highlights:
>
> > 1) Search
> >   Added an option to search by key or value
> > 2) Delete
> >   Added an option to delete by key
>
> > This is particularly useful on a production system if you want to
> > delete a bunch of keys with 'XYZ' in the middle of them:
>
> > e.g.:
>
> > ./memcached-tool localhost search key key XYZ | xargs ./memcached-tool
> > localhost delete
> > (searches for keys with XYZ in the midst, outputs key) (pipes the
> > result to delete them)
>
> > The search term is a regex, and you can search by key, by value, and
> > output keys, or values, or both.
>
> > I'm open to suggestions or changes - it's available for review here:
>
> >http://github.com/mmucklo/memcached/commit/fcbe16eac853bd3f89ad1c38cc...
>
> Hey,
>
> Unfortunately those features will be missleading at best and ineffective
> at worst; the debug commands it uses are slow and only return a subset of
> the keys. So a user may be lead to think that they've found and deleted
> all of those keys when they've really only pushed against the tip of it.
>
> I don't think those commands are very fast either; running them on a
> highly loaded system often may create lag bubbles.
>
> Believe me, I understand that people want to do this, but it's just the
> design tradeoff here. We try hard to only provide features that people
> can't easily bite their feet off with, and saying this would be useful for
> production folks probably isn't true :(
>
> Any thoughts/rebuttals?
>
> -Dormando
Sorry, only registered users may post in this forum.

Click here to login