Welcome! Log In Create A New Profile

Advanced

1MB limit on object size

Posted by Sakuntala 
Sakuntala
1MB limit on object size
October 14, 2010 08:40PM
Hi,

I am using
memcached server version 1.4.4 on Linux and
spy memcached client version 2.4.2

I performed a test by storing and retrieving objects of size 1.8MB and
3.6MB. I am able to retrieve the objects successfully without any
error. My load tests show some timeout exceptions when memcached
server is under heavy load, which is expected.

Memcached wiki website states that one of the reasons we should not be
using memcached is if the object size exceeds 1MB. I want to know if
this is a hard limit, or just that memcached is not efficient in that
scenario due to the storage model.

The reason for my question is that I have just one object(which is a
list) which might exceed 1MB in future, but currently, it is not
exceeding the limit. I considered storing it in several chunks and
using bulkGet() to retrieve the multiple small objects. The response
times are not efficient in case of bulkGet(). I should not be using
memcached for this object( and should look into other caching
mechanisms) but dont want to change the implementation till next
release. Please suggest.

Thanks,
Peter J. Holzer
Re: 1MB limit on object size
October 15, 2010 07:00PM
On 2010-10-14 11:08:59 -0700, Sakuntala wrote:
> I am using
> memcached server version 1.4.4 on Linux and
> spy memcached client version 2.4.2
>
> I performed a test by storing and retrieving objects of size 1.8MB and
> 3.6MB. I am able to retrieve the objects successfully without any
> error. My load tests show some timeout exceptions when memcached
> server is under heavy load, which is expected.
>
> Memcached wiki website states that one of the reasons we should not be
> using memcached is if the object size exceeds 1MB. I want to know if
> this is a hard limit, or just that memcached is not efficient in that
> scenario due to the storage model.

Yes, it's a hard limit (but you can recompile memcached if you want a
higher limit). But most client libraries can compress the
objects before storing it. If you have enabled compression in spy
memcached client (which might be the default) it is possible that you
can store items much larger than 1MB if they compress well.

hp

--
_ | Peter J. Holzer | Openmoko has already embedded
|_|_) | Sysadmin WSR | voting system.
| | | hjp@hjp.at | Named "If you want it -- write it"
__/ | http://www.hjp.at/ | -- Ilja O. on community@lists.openmoko.org
Dustin
Re: 1MB limit on object size
October 15, 2010 07:00PM
On Oct 15, 9:56 am, "Peter J. Holzer" <[email protected]> wrote:
> Yes, it's a hard limit (but you can recompile memcached if you want a
> higher limit). But most client libraries can compress the
> objects before storing it. If you have enabled compression in spy
> memcached client (which might be the default) it is possible that you
> can store items much larger than 1MB if they compress well.

If you don't want to recompile it, you can just use -I at runtime.


-I Override the size of each slab page. Adjusts max item
size
(default: 1mb, min: 1k, max: 128m)
Sakuntala
Re: 1MB limit on object size
October 19, 2010 08:40PM
Peter and Dustin,

Thanks a ton for your replies. The info did help me.

-Sakuntala.


On Oct 15, 12:58 pm, Dustin <[email protected]> wrote:
> On Oct 15, 9:56 am, "Peter J. Holzer" <[email protected]> wrote:
>
> > Yes, it's a hard limit (but you can recompile memcached if you want a
> > higher limit). But most client libraries can compress the
> > objects before storing it. If you have enabled compression in spy
> > memcached client (which might be the default) it is possible that you
> > can store items much larger than 1MB if they compress well.
>
>  If you don't want to recompile it, you can just use -I at runtime.
>
> -I            Override the size of each slab page. Adjusts max item
> size
>               (default: 1mb, min: 1k, max: 128m)
Sorry, only registered users may post in this forum.

Click here to login