Thursday, May 15, 2008

Memcached, but do you need it?

With all due respect to the technology and it's advocates (myself included), after a surge in articles describing the merits of using memcached I'm just pushing a thought breakpoint for developers to think whether they actually need it or not?

Recently, I ran into cases where the developers have decided to use memcached over MySQL style architecture after reading some/many good/nice articles about it without giving a damn to their requirements. I would like to list few things as a checklist for developers to decide on their architecture. There is still no precise answer but sometimes few cases can be just ruled out :).

  1. What is the total size of your data? It might be a possibility that you can keep the data in memory in each node, or MySQL can just keep the whole thing (data+indexes) in a buffer.
  2. How frequently your data is updated? Very frequent updates may lead to low cache hit ratio for memcached data. And refreshing memcached too many times may lead to unnecessary overhead. Remember doing [get,get,set] vs [get].
  3. What is the peak load on your system? Consider if MySQL itself can handle the peak load or otherwise if even memcached cannot handle the peak load with given infrastructure.

I generally ask people a simple question, Why they think they should be using memcached (or something else even)? To shock them, I even ask them "Why they think they should be using MySQL?". And believe me, this is what I believe developers should be asking themselves.

There is only one good argument against this, what if tomorrow you need to suddenly scale or what if your projections need memcached? In such cases, I suggest people to design their data layers in a flexible way, flexible enough to allow things in and out.


Anonymous said...


People are just looking for design patterns and they pick up on what they read. Unless they have been introduced to multiple patterns they typically use the same one over and over again.

Also, few people design for "small". Most people hope their idea will get really big :)


Parvesh Garg said...

Hi Brian,

I do agree to what you say. But on the other hand would like to stress on both the aspects of scalability, that is scaling up and scaling down as well. That's where I mention "flexible enough to allow things in and out".

And no way I'm against memcached, I love it. But I won't use it, neither recommend it for the reason that I love it.

Your comment puts appreciable value to design reviews, my hidden intention :)

Anonymous said...

I recently left my previous employer mainly because their new "architect" decided to replace the persistent MySQL storage layer backed by memcache in favor of memcache alone, something I simply could not support, both in ideological and practical terms.

deadsunrise said...

I think that most people use memcached not only to be able to cache the database but the views and fragments of generated html.

I use it to cache the sessions of my site as well as thousands of fragments of html that I just put together for each user.

Anonymous said...

"just picking mysql" is definitely a valid choice for many developers. its free, open, fairly well debugged, well understood, and there is a large pool of talent.

you can easily get into analysis-paralysis by taking too long to design up front. slap something out, see what the fail points are and address them.