Optimizing Drupal Database Performance with Memcache

What's Memcached?

Memcached is a free and open-source distributed memory object caching system designed to speed up dynamic web applications by alleviating database load. It allows traditionally expensive (in terms of server CPU and RAM) database queries to be pulled from the Memcached server instead of your database. 

If you're looking to increase performance and scale your web application, and you know that slow database (MySQL in my case) performance is the culprit, Memcached is a great place to start. Memcached is used all over the web by massive websites, including Wikipedia and Facebook. 

How to install Memcached

I won't go into a huge amount of detail here, as others have covered this pretty thoroughly. Here's a great article that's specific to Ubuntu and Drupal.

Note that if you're going to install Memcached yourself, you'll need root/sudo access. If you don't have root access, you might be able to ask your host very, very nicely to do it for you. However, most hosts will probably only allow Memcached on private or managed servers, so YMMV.

Also, if you're installing Memcached on a separate machine from your webserver, you'll need the ability to alter your firewall's rules (or can ask someone to do this for you) - more on this later. 

Where should I install Memcached?

This depends greatly on your setup, technical know-how and level of hosting service. For most small to medium sized sites experiencing scaling issues, if you don't have multiple servers available, but do have the ability to get Memcached up and running, I'd say you're fine installing Memcached on the same machine as your webserver of choice (Apache in my case) - just make sure you're careful with memory allocation.

For EmpowHER.com, we use managed hosting with Firehost. Our account has multiple VMs (virtual machines/servers), which we configure separately for different tasks. We decided to put Memcached on a separate VM from our web servers for greater parallelization. This is nice for a lot of reasons - besides better performance, it also spreads out risk a bit. If Memcached starts to have problems, we can dump the cache or restart without Apache really being affected. Another nice reason to have Memcached on it's own server is that you could set up separate bins for other websites - so long as you set up your firewall to allow your webserver to talk to Memcached, you could theoretically have several sites using the same Memcached server.

How to integrate Drupal w/ Memcached

There's two modules you can use to integrate Memcached with Drupal. Memcache and Cache Router. I recommend Memcache, for the reasons Acquia lists here:

  • Memcache is more selective about cache culling than Cache Router.
  • Memcache will cache user sessions, while Cache Router does not.

To install, use Drush or just download the zip, install it in your contrib folder and enable through the modules page. Crack open the README.txt inside the Memcache folder for instructions on setting up your settings.php file.

A note on configuring your settings.php file

You can have any number of cache bins enabled for any given website. The vanilla Drupal 6 caching system has 10 tables it gets and sets data from, plus modules have the ability to create their own cache tables.

There are lots of different cache configuration strategies and we tried several. I won't go into deep detail here, other than to share what's been working great for us for over a year now:

<?php
$conf
['cache_inc'] = './sites/all/modules/contrib/memcache/memcache.inc';
$conf['session_inc'] =  './sites/all/modules/contrib/memcache/memcache-session.inc';
$conf['memcache_servers'] = array(
 
'cache:11212' =&gt; 'default',
 
'cache:11213' =&gt; 'sessions'
);
$conf['memcache_bins'] = array(
 
'session'  =&gt; 'sessions',
 
'users'    =&gt; 'sessions',
);
?>

 

In a nutshell, we're putting the users and session cache (more or less the caches that do the most heavy lifting) in one bin, and everything else into a separate bin.

If you're looking for more info on running multiple cache bins for Drupal, go here.

Ok, it's running - I'm done, right?

Depends. If you're only using Core and Contributed modules and you've seen amazing speed increases on your site, then yes. Go have a drink of your choice and call it a day. If you've written a lot of custom modules like we have, then there's still more work to do.

One of the things the Memcache module does is hook into Drupal core's existing cache_get() and cache_set() functions. Instead of going to the database looking for cached objects, it goes to your Memcached server. However, your custom code (unless you were very thoughtful and are already using caching functions) may have a lot of heavy database queries that are slowing down your site. 

You're going to want to look through your custom module code base, and look for opportunities to store/pull your data from the cache instead of going straight to the DB. I'll save some code samples of this for a later article. 

To diagnose slow database queries, I recommend the Devel module. In using Devel, you may find DB queries in contributed modules that could benefit from caching best practices. If you're able to improve a contrib module by implementing caching, I recommend contacting the maintainer and submitting a patch for extra credit.

How To Monitor Memcached

The Drupal Memcache module comes with a sub-module called Memcache-admin, which includes an admin report for your cache status - but I'm not wild about it. 

We used Harun Yayli's memcache.php - which is based off of the popular APC.php monitoring script. It includes it's own built-in authentication, so make sure you set a reasonably secure username and password before you put that file in a production environment.

That's it! If you have any feedback or questions, feel free to leave a comment below.

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.
glqxz9283 sfy39587stf02 mnesdcuix8
sfy39587stf03
sfy39587stf04