1MB limit on object size

Re: 1MB limit on object size

Sakuntala — Tue, 19 Oct 2010 20:40:16 +0200

Peter and Dustin,

Thanks a ton for your replies. The info did help me.

-Sakuntala.

On Oct 15, 12:58 pm, Dustin wrote:
> On Oct 15, 9:56 am, "Peter J. Holzer" wrote:
>
> > Yes, it's a hard limit (but you can recompile memcached if you want a
> > higher limit). But most client libraries can compress the
> > objects before storing it. If you have enabled compression in spy
> > memcached client (which might be the default) it is possible that you
> > can store items much larger than 1MB if they compress well.
>
> If you don't want to recompile it, you can just use -I at runtime.
>
> -I Override the size of each slab page. Adjusts max item
> size
> (default: 1mb, min: 1k, max: 128m)

Re: 1MB limit on object size

Dustin — Fri, 15 Oct 2010 19:00:13 +0200

On Oct 15, 9:56 am, "Peter J. Holzer" wrote:
> Yes, it's a hard limit (but you can recompile memcached if you want a
> higher limit). But most client libraries can compress the
> objects before storing it. If you have enabled compression in spy
> memcached client (which might be the default) it is possible that you
> can store items much larger than 1MB if they compress well.

If you don't want to recompile it, you can just use -I at runtime.

-I Override the size of each slab page. Adjusts max item
size
(default: 1mb, min: 1k, max: 128m)

Re: 1MB limit on object size

Peter J. Holzer — Fri, 15 Oct 2010 19:00:13 +0200

On 2010-10-14 11:08:59 -0700, Sakuntala wrote:
> I am using
> memcached server version 1.4.4 on Linux and
> spy memcached client version 2.4.2
>
> I performed a test by storing and retrieving objects of size 1.8MB and
> 3.6MB. I am able to retrieve the objects successfully without any
> error. My load tests show some timeout exceptions when memcached
> server is under heavy load, which is expected.
>
> Memcached wiki website states that one of the reasons we should not be
> using memcached is if the object size exceeds 1MB. I want to know if
> this is a hard limit, or just that memcached is not efficient in that
> scenario due to the storage model.

Yes, it's a hard limit (but you can recompile memcached if you want a
higher limit). But most client libraries can compress the
objects before storing it. If you have enabled compression in spy
memcached client (which might be the default) it is possible that you
can store items much larger than 1MB if they compress well.

hp

--
_ | Peter J. Holzer | Openmoko has already embedded
|_|_) | Sysadmin WSR | voting system.
| | | hjp@hjp.at | Named "If you want it -- write it"
__/ | http://www.hjp.at/ | -- Ilja O. on community@lists.openmoko.org

1MB limit on object size

Sakuntala — Thu, 14 Oct 2010 20:40:07 +0200

Hi,

I am using
memcached server version 1.4.4 on Linux and
spy memcached client version 2.4.2

I performed a test by storing and retrieving objects of size 1.8MB and
3.6MB. I am able to retrieve the objects successfully without any
error. My load tests show some timeout exceptions when memcached
server is under heavy load, which is expected.

Memcached wiki website states that one of the reasons we should not be
using memcached is if the object size exceeds 1MB. I want to know if
this is a hard limit, or just that memcached is not efficient in that
scenario due to the storage model.

The reason for my question is that I have just one object(which is a
list) which might exceed 1MB in future, but currently, it is not
exceeding the limit. I considered storing it in several chunks and
using bulkGet() to retrieve the multiple small objects. The response
times are not efficient in case of bulkGet(). I should not be using
memcached for this object( and should look into other caching
mechanisms) but dont want to change the implementation till next
release. Please suggest.

Thanks,