Matrix in memcache

Hi all..

I have dense matrix (large sized matrix).Because of its size i could not be
able to kept in memory.So i would like to use memcache.I configured
memcached and is working.
But how can i represent this Matrix input as a key,value pair.Is there any
need to use any Db like MongoDb...

Pls help me to find out the solution

Thank you
Sreejith

When dealing with large data sets I sometimes split it into parts that
will fit with a master object that stores how many parts it should
have. Of course it could still fall out of cache but it's no worse
than if it was one large object that fell out of cache and possible
better if you kept a primary copy somewhere so you only have to grab
the missing chunks from it.

big_object: 5
big_object_0: data
big_object_1: data
big_object_2: data
big_object_3: data
big_object_4: data

Hi..
Thnk u for ur reply...
let me explain my scenario
On a sample set od 1000 documents i am generating a matrix of (may be)
1000000 * 1000 .I have much larger document set than this say 50000.
So can memcahe be able to store these huge matrix representation as u said??

Sreejith


On Thu, Oct 28, 2010 at 6:56 AM, PlumbersStock.com <
[email protected]> wrote:

> When dealing with large data sets I sometimes split it into parts that
> will fit with a master object that stores how many parts it should
> have. Of course it could still fall out of cache but it's no worse
> than if it was one large object that fell out of cache and possible
> better if you kept a primary copy somewhere so you only have to grab
> the missing chunks from it.
>
> big_object: 5
> big_object_0: data
> big_object_1: data
> big_object_2: data
> big_object_3: data
> big_object_4: data

No, it's a cache, not a storage solution. When you put things in memcache
you are not guaranteed to be able to retrieve them again.


/Henrik

On Thu, Oct 28, 2010 at 07:23, Sreejith S <[email protected]> wrote:

> Hi..
> Thnk u for ur reply...
> let me explain my scenario
> On a sample set od 1000 documents i am generating a matrix of (may be)
> 1000000 * 1000 .I have much larger document set than this say 50000.
> So can memcahe be able to store these huge matrix representation as u
> said??
>
> Sreejith
>
>
>
> On Thu, Oct 28, 2010 at 6:56 AM, PlumbersStock.com <
> [email protected]> wrote:
>
>> When dealing with large data sets I sometimes split it into parts that
>> will fit with a master object that stores how many parts it should
>> have. Of course it could still fall out of cache but it's no worse
>> than if it was one large object that fell out of cache and possible
>> better if you kept a primary copy somewhere so you only have to grab
>> the missing chunks from it.
>>
>> big_object: 5
>> big_object_0: data
>> big_object_1: data
>> big_object_2: data
>> big_object_3: data
>> big_object_4: data
>
>
>

You can store them but it shouldn't be your only copy. I frequently
cache documents in memcache so I don't have to pull them off the
filesystem as often but I do store them in the filesystem. If using a
lot of memory you may have to increase the size of your cache.