Skip to content

Replace legacy benchmark results #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 3, 2025
Merged

Replace legacy benchmark results #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 16 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,25 +58,22 @@ This client start to connect using RESP2 and currently it has no option to upgra

Simple microbenchmark comparing PP and XS.
The benchmark script used can be found under examples directory.

Redis::Cluster::Fast is 0.084
Redis::ClusterRider is 0.26
### mset ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 13245/s -- -34%
Redis::Cluster::Fast 20080/s 52% --
### mget ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 14641/s -- -40%
Redis::Cluster::Fast 24510/s 67% --
### incr ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 18367/s -- -44%
Redis::Cluster::Fast 32879/s 79% --
### new and ping ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 146/s -- -96%
Redis::Cluster::Fast 3941/s 2598% --
Each operation was executed 100,000 times, and the execution time was measured in milliseconds.

+--------------------------------+-------+-------+-------+-------+-------+
| Operation | P50 | P80 | P95 | P99 | P100 |
+--------------------------------+-------+-------+-------+-------+-------+
| get_pp | 0.028 | 0.032 | 0.036 | 0.050 | 0.880 |
| get_xs | 0.020 | 0.023 | 0.025 | 0.044 | 0.881 |
| get_xs_pipeline | 0.014 | 0.015 | 0.018 | 0.021 | 0.472 |
| get_xs_pipeline_batched_100 | 0.003 | 0.003 | 0.004 | 0.074 | 0.323 |
| set_pp | 0.028 | 0.032 | 0.037 | 0.051 | 2.014 |
| set_xs | 0.021 | 0.024 | 0.027 | 0.047 | 0.729 |
| set_xs_pipeline | 0.014 | 0.016 | 0.018 | 0.021 | 0.393 |
| set_xs_pipeline_batched_100 | 0.003 | 0.004 | 0.005 | 0.073 | 0.379 |
+--------------------------------+-------+-------+-------+-------+-------+

c.f. https://github.com/plainbanana/Redis-Cluster-Fast-Benchmarks

# METHODS

Expand Down
185 changes: 98 additions & 87 deletions examples/benchmark-simple.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,103 +1,114 @@
use strict;
use warnings FATAL => 'all';
use feature qw/say/;
use lib ".";
use lib "./_build/lib";
use lib "./blib/arch";
use lib "./blib/lib";
use lib './xt/lib';
use Benchmark;

use Time::HiRes qw/gettimeofday tv_interval/;
use Redis::Cluster::Fast;
use Redis::ClusterRider;
use Test::More; # for Test::RedisCluster
use Test::Docker::RedisCluster qw/get_startup_nodes/;

print "Redis::Cluster::Fast is " . $Redis::Cluster::Fast::VERSION . "\n";
print "Redis::ClusterRider is " . $Redis::ClusterRider::VERSION . "\n";

my $nodes = get_startup_nodes;
my $nodes_str = $ENV{REDIS_NODES};
my $nodes = [
split(/,/, $nodes_str)
];

my $xs = Redis::Cluster::Fast->new(
startup_nodes => $nodes,
route_use_slots => 1,
);

my $pp = Redis::ClusterRider->new(
startup_nodes => $nodes,
);

my $cc = 0;
my $dd = 0;

my $loop = 100000;
print "### mset ###\n";
Benchmark::cmpthese($loop, {
"Redis::ClusterRider" => sub {
$cc++;
$pp->mset("{pp$cc}atest", $cc, "{pp$cc}btest", $cc, "{pp$cc}ctest", $cc);
},
"Redis::Cluster::Fast" => sub {
$dd++;
$xs->mset("{xs$dd}atest", $dd, "{xs$dd}btest", $dd, "{xs$dd}ctest", $dd);
},
});

$cc = 0;
$dd = 0;

print "### mget ###\n";
Benchmark::cmpthese($loop, {
"Redis::ClusterRider" => sub {
$cc++;
$pp->mget("{pp$cc}atest", "{pp$cc}btest", "{pp$cc}ctest");
},
"Redis::Cluster::Fast" => sub {
$dd++;
$xs->mget("{xs$dd}atest", "{xs$dd}btest", "{xs$dd}ctest");
},
});

print "### incr ###\n";
Benchmark::cmpthese(-2, {
"Redis::ClusterRider" => sub {
$pp->incr("incr_1");
},
"Redis::Cluster::Fast" => sub {
$xs->incr("incr_2");
},
});

print "### new and ping ###\n";
Benchmark::cmpthese(-2, {
"Redis::ClusterRider" => sub {
my $tmp = Redis::ClusterRider->new(
startup_nodes => $nodes,
);
$tmp->ping;
},
"Redis::Cluster::Fast" => sub {
my $tmp = Redis::Cluster::Fast->new(
startup_nodes => $nodes,
);
$tmp->ping;
},
});

is 1, 1;
done_testing;
__END__
% AUTHOR_TESTING=1 perl ./examples/benchmark-simple.pl
Redis::Cluster::Fast is 0.084
Redis::ClusterRider is 0.26
### mset ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 13245/s -- -34%
Redis::Cluster::Fast 20080/s 52% --
### mget ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 14641/s -- -40%
Redis::Cluster::Fast 24510/s 67% --
### incr ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 18367/s -- -44%
Redis::Cluster::Fast 32879/s 79% --
### new and ping ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 146/s -- -96%
Redis::Cluster::Fast 3941/s 2598% --
ok 1
1..1

######
# set
######
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$xs->set('1' . $num, 123);

my $elapsed_time = tv_interval($start_time);
printf "set_xs,%.10f\n", $elapsed_time * 1000;
}
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$xs->set('2' . $num, 123, sub {});
$xs->run_event_loop;

my $elapsed_time = tv_interval($start_time);
printf "set_xs_pipeline,%.10f\n", $elapsed_time * 1000;
}
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$xs->set('3' . $num, 123, sub {});
$xs->run_event_loop if $num % 100 == 0;

my $elapsed_time = tv_interval($start_time);
printf "set_xs_pipeline_batched_100,%.10f\n", $elapsed_time * 1000;
}
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$pp->set('4' . $num, 123);

my $elapsed_time = tv_interval($start_time);
printf "set_pp,%.10f\n", $elapsed_time * 1000;
}

######
# get
######
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$xs->get('1' . $num);

my $elapsed_time = tv_interval($start_time);
printf "get_xs,%.10f\n", $elapsed_time * 1000;
}
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$xs->get('2' . $num, sub {});
$xs->run_event_loop;

my $elapsed_time = tv_interval($start_time);
printf "get_xs_pipeline,%.10f\n", $elapsed_time * 1000;
}
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$xs->get('3' . $num, sub {});
$xs->run_event_loop if $num % 100 == 0;

my $elapsed_time = tv_interval($start_time);
printf "get_xs_pipeline_batched_100,%.10f\n", $elapsed_time * 1000;
}
sleep 1;
for my $num (1 .. $loop) {
my $start_time = [ gettimeofday ];

$pp->get('4' . $num);

my $elapsed_time = tv_interval($start_time);
printf "get_pp,%.10f\n", $elapsed_time * 1000;
}

__END__
35 changes: 16 additions & 19 deletions lib/Redis/Cluster/Fast.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,25 +192,22 @@ This client start to connect using RESP2 and currently it has no option to upgra

Simple microbenchmark comparing PP and XS.
The benchmark script used can be found under examples directory.

Redis::Cluster::Fast is 0.084
Redis::ClusterRider is 0.26
### mset ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 13245/s -- -34%
Redis::Cluster::Fast 20080/s 52% --
### mget ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 14641/s -- -40%
Redis::Cluster::Fast 24510/s 67% --
### incr ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 18367/s -- -44%
Redis::Cluster::Fast 32879/s 79% --
### new and ping ###
Rate Redis::ClusterRider Redis::Cluster::Fast
Redis::ClusterRider 146/s -- -96%
Redis::Cluster::Fast 3941/s 2598% --
Each operation was executed 100,000 times, and the execution time was measured in milliseconds.

+--------------------------------+-------+-------+-------+-------+-------+
| Operation | P50 | P80 | P95 | P99 | P100 |
+--------------------------------+-------+-------+-------+-------+-------+
| get_pp | 0.028 | 0.032 | 0.036 | 0.050 | 0.880 |
| get_xs | 0.020 | 0.023 | 0.025 | 0.044 | 0.881 |
| get_xs_pipeline | 0.014 | 0.015 | 0.018 | 0.021 | 0.472 |
| get_xs_pipeline_batched_100 | 0.003 | 0.003 | 0.004 | 0.074 | 0.323 |
| set_pp | 0.028 | 0.032 | 0.037 | 0.051 | 2.014 |
| set_xs | 0.021 | 0.024 | 0.027 | 0.047 | 0.729 |
| set_xs_pipeline | 0.014 | 0.016 | 0.018 | 0.021 | 0.393 |
| set_xs_pipeline_batched_100 | 0.003 | 0.004 | 0.005 | 0.073 | 0.379 |
+--------------------------------+-------+-------+-------+-------+-------+

c.f. https://github.com/plainbanana/Redis-Cluster-Fast-Benchmarks

=head1 METHODS

Expand Down