Welcome! Log In Create A New Profile

Advanced

instrumenting memcached with sFlow

Posted by neilmckee 
neilmckee
instrumenting memcached with sFlow
September 24, 2010 04:30AM
Hello All,

Here is a patch against the central repo that adds sFlow monitoring.
Please comment!
http://www.inmon.com/technology/sfmc_20100923.zip

For background on this, please read:
http://blog.sflow.com/2010/08/sflow-host-structures.html

If you compile with this patch, then you can run with "memcached -u
nobody -o sflow=on", and it will pick up configuration from /etc/
hsflowd.auto, such as:

sampling=400
polling=20
agentIP=10.0.0.112
collector=10.0.0.111 6343

"agentIP" should be the IP of the server, and "collector" should be
the IP of an sflow collector such as "sflowtool":
http://www.inmon.com/bin/sflowtool-3.17.tar.gz

(This config file is generated automatically if you install hsflowd,
the host sFlow daemon from host-sflow.sourceforge.net which
contributes an sFlow feed of server performance stats)

Neil


NOTES:

(1). Provided the sampling-rate is set appropriately, the overhead
should be roughly equivalent to adding one extra stats counter. The
critical path is just a decrement-and-test on a per-thread counter (no
locking).

(2). The changes are all within "#ifdef SFLOW" except for (3) below.

(3). I changed memcached.h:struct conn->request_addr to be of type
struct sockaddr_storage so it would work for IPv6 too.
neilmckee
Re: instrumenting memcached with sFlow
September 27, 2010 07:00PM
I posted this patch as an enhancement to the tracking system:
http://code.google.com/p/memcached/issues/detail?id=157

My original post wasn't very clear about the problem space being
addressed: it is targeted at the challenge of identifying "hot keys"
in an operational cluster. For background, see:
http://blog.sflow.com/2010/09/memcached.html

Neil


On Sep 23, 5:19 pm, neilmckee <[email protected]> wrote:
> Hello All,
>
> Here is a patch against the central repo that adds sFlow monitoring.
> Please comment!http://www.inmon.com/technology/sfmc_20100923.zip
>
> For background on this, please read:http://blog.sflow.com/2010/08/sflow-host-structures.html
>
> If you compile with this patch,  then you can run with "memcached -u
> nobody -o sflow=on",  and it will pick up configuration from /etc/
> hsflowd.auto, such as:
>
> sampling=400
> polling=20
> agentIP=10.0.0.112
> collector=10.0.0.111 6343
>
> "agentIP" should be the IP of the server,  and "collector" should be
> the IP of an sflow collector such as "sflowtool":http://www.inmon.com/bin/sflowtool-3.17.tar.gz
>
> (This config file is generated automatically if you install hsflowd,
> the host sFlow daemon from host-sflow.sourceforge.net which
> contributes an sFlow feed of server performance stats)
>
> Neil
>
> NOTES:
>
> (1).  Provided the sampling-rate is set appropriately, the overhead
> should be roughly equivalent to adding one extra stats counter.  The
> critical path is just a decrement-and-test on a per-thread counter (no
> locking).
>
> (2).  The changes are all within "#ifdef SFLOW" except for (3) below.
>
> (3).  I changed memcached.h:struct conn->request_addr to be of type
> struct sockaddr_storage so it would work for IPv6 too.
neilmckee
Re: instrumenting memcached with sFlow
September 28, 2010 09:40PM
The following git fork includes these sFlow changes:

http://github.com/sflow/memcached

To run with sFlow enabled:

../autogen.sh
../configure --enable-sflow
make
../memcached -u nobody -o sflow=on

Neil


On Sep 27, 9:59 am, neilmckee <[email protected]> wrote:
> I posted this patch as an enhancement to the tracking system:http://code.google.com/p/memcached/issues/detail?id=157
>
> My original post wasn't very clear about the problem space being
> addressed:  it is targeted at the challenge of identifying "hot keys"
> in an operational cluster.  For background, see:http://blog.sflow.com/2010/09/memcached.html
>
> Neil
>
> On Sep 23, 5:19 pm, neilmckee <[email protected]> wrote:
>
> > Hello All,
>
> > Here is a patch against the central repo that adds sFlow monitoring.
> > Please comment!http://www.inmon.com/technology/sfmc_20100923.zip
>
> > For background on this, please read:http://blog.sflow.com/2010/08/sflow-host-structures.html
>
> > If you compile with this patch,  then you can run with "memcached -u
> > nobody -o sflow=on",  and it will pick up configuration from /etc/
> > hsflowd.auto, such as:
>
> > sampling=400
> > polling=20
> > agentIP=10.0.0.112
> > collector=10.0.0.111 6343
>
> > "agentIP" should be the IP of the server,  and "collector" should be
> > the IP of an sflow collector such as "sflowtool":http://www.inmon.com/bin/sflowtool-3.17.tar.gz
>
> > (This config file is generated automatically if you install hsflowd,
> > the host sFlow daemon from host-sflow.sourceforge.net which
> > contributes an sFlow feed of server performance stats)
>
> > Neil
>
> > NOTES:
>
> > (1).  Provided the sampling-rate is set appropriately, the overhead
> > should be roughly equivalent to adding one extra stats counter.  The
> > critical path is just a decrement-and-test on a per-thread counter (no
> > locking).
>
> > (2).  The changes are all within "#ifdef SFLOW" except for (3) below.
>
> > (3).  I changed memcached.h:struct conn->request_addr to be of type
> > struct sockaddr_storage so it would work for IPv6 too.
Peter Phaal
Re: instrumenting memcached with sFlow
October 02, 2010 09:40AM
The following article shows how to report on hot keys using sFlow data
collected from the Memcached cluster:
http://blog.sflow.com/2010/10/memcached-hot-keys.html

On Sep 28, 12:30 pm, neilmckee <[email protected]> wrote:
> The following git fork includes these sFlow changes:
>
> http://github.com/sflow/memcached
>
> To run with sFlow enabled:
>
> ./autogen.sh
> ./configure --enable-sflow
> make
> ./memcached -u nobody -o sflow=on
>
> Neil
Sorry, only registered users may post in this forum.

Click here to login