Thursday, May 15, 2008

Memcached, but do you need it?

With all due respect to the technology and it's advocates (myself included), after a surge in articles describing the merits of using memcached I'm just pushing a thought breakpoint for developers to think whether they actually need it or not?

Recently, I ran into cases where the developers have decided to use memcached over MySQL style architecture after reading some/many good/nice articles about it without giving a damn to their requirements. I would like to list few things as a checklist for developers to decide on their architecture. There is still no precise answer but sometimes few cases can be just ruled out :).

  1. What is the total size of your data? It might be a possibility that you can keep the data in memory in each node, or MySQL can just keep the whole thing (data+indexes) in a buffer.
  2. How frequently your data is updated? Very frequent updates may lead to low cache hit ratio for memcached data. And refreshing memcached too many times may lead to unnecessary overhead. Remember doing [get,get,set] vs [get].
  3. What is the peak load on your system? Consider if MySQL itself can handle the peak load or otherwise if even memcached cannot handle the peak load with given infrastructure.

I generally ask people a simple question, Why they think they should be using memcached (or something else even)? To shock them, I even ask them "Why they think they should be using MySQL?". And believe me, this is what I believe developers should be asking themselves.

There is only one good argument against this, what if tomorrow you need to suddenly scale or what if your projections need memcached? In such cases, I suggest people to design their data layers in a flexible way, flexible enough to allow things in and out.

8 comments:

Anonymous said...

Hi!

People are just looking for design patterns and they pick up on what they read. Unless they have been introduced to multiple patterns they typically use the same one over and over again.

Also, few people design for "small". Most people hope their idea will get really big :)

Cheers,
-Brian

Parvesh Garg said...

Hi Brian,

I do agree to what you say. But on the other hand would like to stress on both the aspects of scalability, that is scaling up and scaling down as well. That's where I mention "flexible enough to allow things in and out".

And no way I'm against memcached, I love it. But I won't use it, neither recommend it for the reason that I love it.

Your comment puts appreciable value to design reviews, my hidden intention :)
-
Parvesh

Anonymous said...

I recently left my previous employer mainly because their new "architect" decided to replace the persistent MySQL storage layer backed by memcache in favor of memcache alone, something I simply could not support, both in ideological and practical terms.

Anonymous said...

I think that most people use memcached not only to be able to cache the database but the views and fragments of generated html.

I use it to cache the sessions of my site as well as thousands of fragments of html that I just put together for each user.

Anonymous said...

"just picking mysql" is definitely a valid choice for many developers. its free, open, fairly well debugged, well understood, and there is a large pool of talent.

you can easily get into analysis-paralysis by taking too long to design up front. slap something out, see what the fail points are and address them.

John Fei said...

The SAP® software framework is utilized to computerize the expert exercises of agents of different specializations and industries. Such applications are honed for a particular area and incredibly improve on work and correspondence with other organizations' primary units. SAP® BusinessObject (SAP BO developer) is a bound together logical stage for BI arrangements, an amazing business examination apparatus that permits business' chiefs to oversee such cycles as business planning, information solidification, investigation, control, hazard appraisal, reporting, and so forth SAP® BI arrangement (SAP® Business Intelligence) is a business examination framework that takes into account key information investigation and backing for its administration dynamic cycle. It is designed to give admittance to and measure information in different frameworks or data sets of associations and its inside and out investigation. The sources can be information frameworks, accounting, financial projects, specific industry arrangements, and neighborhood hotspots (for instance, Excel or Access records). This item permits business proprietors to address a wide scope of undertakings identified with the financial area, business intelligence, and so on The BI framework gives a multidimensional investigation of execution results for predicting and planning future business execution. The arrangement includes creating and publishing redid interactive reports and applications that bring another mindfulness level to chiefs. The framework additionally gives apparatuses and instant models for information investigation and reporting, which gives an integrated way to deal with a legitimization for smooth dynamic.

hrdnmxdvlp said...

By consolidating customer data into a single database, businesses can gain a complete view of their customers and their interactions with them. This can help businesses to better understand their customers' needs and preferences. With dynamics crm consultant, you can find out what this function will bring to your business.

DmytroPortnoy said...

The task of processing vast amounts of data can be daunting, and finding ways to optimize the process can be difficult for a single person, especially in a large business environment. In this scenario, utilizing an online data labeling service like image video annotation that provides a team of experts may be worth exploring. You can customize the team's composition and working methods to meet your specific requirements.