web developer & system programmer

coder . cl

ramblings and thoughts on programming...


concurrency in redis and memcache

published: 19-06-2011 / updated: 19-06-2011
posted in: development, programming, projects, python, tips
by Daniel Molina Wegener

I have compared redis and memcache with a lighweight test. Now I have compared both key/value stores with a harder test, using 100 threads, and observing the speed between the parallel writes and parallel reads in both engines. I’ve used the Python binding, since I need to apply one of those key/value storage to certain Python application that I am currently developing. Certainly both have advantages between each other, but still I am preferring Redis over Memcached, because seems that Redis can manage more constant times between both operations, and seems to have a lighweight routine to make concurrent writes.

About the concurrent writes, we can see that Redis seems to be using a constant time to do each write. The test run was using 100000 different keys between 100 threads. The same test was applied to Memcached. The difference on key/value writes seems to have an average of one second, and is very clear that with an application that does heavy writes to the key/value storage is better to have a parallel write which uses less time.

Redis and Memcached Parallel Writes

Redis and Memcached Parallel Writes

The time used for each task on writes seems to be more homogeneous under Redis than Memcached. With an equal number of threads the time remains on very similar levels under Redis, either on Memcached seems to be affected by the number of threads.

The stored data was a python dictionary which was using random data. Despite the fact that Redis uses asynchronous persistent storage, is faster writing data than Memcached. Where Redis fails, seems to be the parallel reads. Where Redis seems to be using a constant time, without too much variability between the number of threads launched, and the difference seems to have an average of 1.5 seconds.

Redis and Memcached Parallel Reads

Redis and Memcached Parallel Reads

But on this application, key/value store reads are made asynchronously, so we can deal with that speed without problems. The other problem, is the fact that Memcached restricts the key compsition, it cannot have special characters and even it cannot have i18n characters, like utf-8 encoded keys. I insist to review the hashing algorithm that is using Redis and Memcached, probably I will do in a further post.

So, we can conclude that Redis has more stable response times with similar conditions and Memcached seems to be affected by the number of threads and environment variables. I will implement Redis on this case, which will not be used to store Web Page cache if you are thinking on it.


No coments yet.

post a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>