Patrick Galbraith (capttofu ) wrote,

Libmemcached is faster ...except with increment and decrement (?)

I'm busy working on my book, "Developing Web Applications with Apache, MySQL, Memcached, and Perl", writing about how Libmemcached (particularly the perl interface to libmemcached, Cache::Memcached::libmemcached/Memcached::libmemcached) is faster. And it is, except when I was writing a test script to compare, I first used Daisuke Maki's script that comes with Cache::Memcached::libmemcached, tools/benchmark.pl (which I modified to only compare Cache::Memcached to Cache::Memcached::libmemcached)

==== Benchmark "Simple get() (scalar)" ====
Rate perl_memcached libmemcached
perl_memcached 6784/s -- -78%
libmemcached 30488/s 349% --
==== Benchmark "Simple get_multi() (scalar)" ====
Rate perl_memcached libmemcached
perl_memcached 1806/s -- -84%
libmemcached 11494/s 537% --
==== Benchmark "Serialization with get()" ====
Rate perl_memcached libmemcached
perl_memcached 6402/s -- -74%
libmemcached 24631/s 285% --
==== Benchmark "Simple set() (scalar)" ====
Rate perl_memcached libmemcached
perl_memcached 17007/s -- -39%
libmemcached 27933/s 64% --

Looks great! Get, Set, much faster with libmemcached.

Now, to make a long story short, I decided to write my own script which compared various operations in a loop, determined by command line value. All were much faster with Cache::Memcached::libmemcached than with Cache::Memcached. Until I added increment and decrement. So, I wrote a stripped down test that only runs increment and/or decrement.

use strict;
use Cache::Memcached;
use Cache::Memcached::libmemcached;
use Time::HiRes qw(tv_interval gettimeofday);
use Data::Dumper;

use Getopt::Long;

# global options
our $opt_num_loop;
our $opt_servers;
our $opt_libmemcached;
our $opt_incr;
our $opt_decr;
our $opt_num;

GetOptions(
  'decr|d'        => \$opt_decr,
  'inc|i'         => \$opt_incr,
  'num|n=s'       => \$opt_num,
  'loops|l=s'     => \$opt_num_loop,
  'libmemcached'  => \$opt_libmemcached,
  'servers|s=s'   => \$opt_servers,
) or usage();

# defaults
$opt_servers||= '127.0.0.1:11211';
$opt_num_loop||= 10;

sub main {
  my $servers; 
  my $key= 'inctest';
 # set up servers
  (@$servers)= split(',', $opt_servers);

  # toggle which library to use
  my $memc= $opt_libmemcached ?
    new Cache::Memcached::libmemcached({
        servers             => $servers,
        compress_threshold  => 10_000}) :
    new Cache::Memcached({
        servers             => $servers,
        compress_threshold  => 10_000});

  # clear all values
  $memc->flush_all();
  $memc->set($key, 100);

  # obtain start time
  my $t0 = [gettimeofday];

  # loop through entire hashref
  for my $counter (0 .. $opt_num_loop) {
    # set, get, replace, incr, decr, delete...
    $memc->incr($key, $opt_num) if $opt_incr;
    $memc->decr($key, $opt_num) if $opt_decr;
  }

  # record elapsed time
  my $elapsed = tv_interval ( $t0 );
  $memc->delete($key);

  print "Time took: $elapsed\n";
}



A couple test runs reveals something is odd:

[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -d -i --libmemcached
Time took: 127.003263
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i --libmemcached
Time took: 1.938455
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -d --libmemcached
Time took: 116.064165
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -d
Time took: 0.856013
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i
Time took: 0.814934
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i --libmemcached
Time took: 1.902308
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i --libmemcached
Time took: 190.948462
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i
Time took: 0.850609
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i --libmemcached
Time took: 2.001388
[patg@vidya perl]$ ./memtest_inc.pl -l 10000 -n 10 -i --libmemcached
Time took: 208.984698

208 seconds vs. 2 seconds, not good.

So, I decided to go to a lower level and use Memcached::libmemcached, which Cache::Memcached::libmemcached uses

use strict;
use Cache::Memcached;
use Memcached::libmemcached;
use Time::HiRes qw(tv_interval gettimeofday);

... same code as before ...

 # set up servers
  (@$servers)= split(',', $opt_servers);

  if ($opt_libmemcached) {
    $memc= new Memcached::libmemcached;

    for my $server(@$servers) {
      my ($host,$port)= split(/:/, $server);
      $memc->memcached_server_add($host, $port);
    }
  }
  else {
  # toggle which library to use
    $memc= new Cache::Memcached({
        servers             => $servers,
        compress_threshold  => 10_000});
  }

  # clear all values
  if ($opt_libmemcached) {
    $memc->memcached_flush();
    $memc->memcached_set($key, 100);
  }

 else {
    $memc->flush_all();
    $memc->set($key, 100);
  }

  # obtain start time
  my $t0 = [gettimeofday];

  # loop through entire hashref
  for my $counter (0 .. $opt_num_loop) {
    # set, get, replace, incr, decr, delete...
    if ($opt_libmemcached) {
      $memc->memcached_increment($key, $opt_num) if $opt_incr;
      $memc->memcached_decrement($key, $opt_num) if $opt_decr;
    }
    else {
      $memc->incr($key, $opt_num) if $opt_incr;
      $memc->decr($key, $opt_num) if $opt_decr;
    }
  }
  # record elapsed time
  my $elapsed = tv_interval ( $t0 );
  if ($opt_libmemcached) {
    $memc->memcached_delete($key);
  }
  else {
    $memc->delete($key);
  }

  print "Time took: $elapsed\n";
}



The results still didn't look good:

[patg@vidya perl]$ ./memtest_inc_memc.pl -l 10000 -n 10 -i -d
Time took: 1.701365
[patg@vidya perl]$ ./memtest_inc_memc.pl -l 10000 -n 10 -i -d --libmemcached
Time took: 192.818737
[patg@vidya perl]$ ./memtest_inc_memc.pl -l 100 -n 10 -i -d --libmemcached
Time took: 0.03991
[patg@vidya perl]$ ./memtest_inc_memc.pl -l 100 -n 10 -i -d
Time took: 0.016624
[patg@vidya perl]$ ./memtest_inc_memc.pl -l 1000 -n 10 -i -d
Time took: 0.172632
[patg@vidya perl]$ ./memtest_inc_memc.pl -l 1000 -n 10 -i -d --libmemcached
Time took: 0.383152

Since Memcached::libmemcached is really 1:1 perl interface to libmemcached, I would suspect something in libmemcached vs. the perl implementation. I would want to write a test in C using libmemcached just running increments and decrements. I'm not sure if memslap runs increment and/or decrement.

In the big picture of things, I'm not sure how many people use decrement and increment. Most people want to simply store data on memcached, and for that libmemcached is much faster.

I need to run memcached in verbose mode and see what really is happening.
Tags: libmemcached, memcached, performance, perl
  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic
  • 2 comments