RhoConnect Benchmarks
===
Note, these numbers reflect synchronization performance using the rhoconnect bench utilities (see the rhoconnect/bench folder for more information).
We will be adding more results as we test across different server environments.
## Single-core Amazon EC2 server environment
### Configuration
Servers configuration (Amazon EC2 'm1.small' instances) :
* CPU Units - 1 ECU
* CPU Cores - 1 Core
* Memory - 1.7GB
* Platform - Ubuntu (2.6.38-8-virtual)
Environment (instance #1):
* Nginx 1.0.0
* Ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-linux], Ruby Enterprise Edition 2011.03
* Passenger 3.0.7 (mod_rails)
* Redis 2.2.11
* Rhoconnect 3.0.0
Nginx configuration file:
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-3.0.7;
passenger_ruby /usr/local/bin/ruby;
passenger_max_pool_size 20;
include mime.types;
default_type application/octet-stream;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
gzip on;
server {
listen 80;
server_name ec2-50-18-72-186.us-west-1.compute.amazonaws.com;
root /var/www/benchapp/public; # Bench application #1
# root /var/www/blobapp/public; # Bench application #2
passenger_enabled on;
passenger_min_instances 6;
}
}
Environment (instance #2):
* Ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-linux], Ruby Enterprise Edition 2011.03
* Rhoconnect 3.0.0 code with bench scripts
Server instance #1 is running rhoconnect bench applications,
the 2nd one executes rhoconnect bench scripts.
### Bench application #1 (benchapp)
* Simulates creating multiple objects
* 100 concurrent users
* 1 device per user
* 20 object dataset
* each object has 7 attributes
* 904 total HTTP requests
Request clientcreate : min: 0.0069, max: 1.6010, avg: 0.3404
Request ack-cud : min: 0.0339, max: 3.2250, avg: 0.5910
Request clientlogin : min: 0.0063, max: 0.8138, avg: 0.1216
Request get-cud : min: 0.0315, max: 3.0044, avg: 0.3988
Request create-object : min: 0.0791, max: 4.4959, avg: 1.3521
Throughput(req/s) : 7.13629650622702
Throughput(req/min) : 428.177790373621
### Bench application #2 (blobapp)
* Simulates creating multiple objects
* 100 concurrent users
* 1 device per user
* 10 object dataset
* each object has variable number of attributes (from 10 to 60) + image
* 904 total HTTP requests
Request clientcreate : min: 0.0067, max: 2.9286, avg: 0.6870
Request ack-cud : min: 0.0473, max: 5.5015, avg: 1.6219
Request clientlogin : min: 0.0061, max: 3.2326, avg: 0.4687
Request get-cud : min: 0.0382, max: 7.2202, avg: 1.0975
Request create-object : min: 0.3765, max: 11.1387, avg: 5.1424
Throughput(req/s) : 7.22833609137099
Throughput(req/min): 433.700165482259
### Prepare bench applications for running on server #1
Go to rhoconnect/bench directory and configure benchapp application:
sudo cp -R benchapp /var/www/
cd /var/www/
sudo mkdir benchapp/public
sudo chown -R www-data:www-data benchapp/
Do the same for blobapp bench:
sudo cp -R blobapp /var/www/
cd /var/www/
sudo mkdir blobapp/public
sudo chown -R www-data:www-data blobapp/
Edit Nginx configuration file, set server_name and root properties to public DNS server name
and bench application public folder respectively. After restarting server make sure that bench application is
running by visiting its web console, i.e. :
http://server_dns_public_name/console/
### How to execute bench scripts on server #2
Before running bench scripts you need configure bench application settings properly.
Edit rhoconnect/bench/benchapp/settings/settings.yml and set redis and syncserver properties:
:development:
:licensefile: settings/license.key
:redis: server_ip_address:6379
:syncserver: http://server_dns_public_name:80/api/application/
Do the same for blobapp settings.yml file.
Then from rhoconnect/bench directory execute run_cud_script.sh and run_blob_script.sh scripts respectively.
## Multi-core Amazon EC2 server environments
### Configuration
Server configurations:
* Large (m1.large, 2 cores, 7.5GB memory)
* Extra Large (m1.xlarge, 4 cores, 15GB memory)
* High-CPU Extra Large (c1.xlarge, 8 cores, 7GB memory)
* Platform - Basic 64-bit Amazon Linux AMI 2011.09 (Red Hat based)
Software environment:
* Nginx 1.0.6
* Ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-linux], Ruby Enterprise Edition 2011.03
* Passenger 3.0.9
* Redis 2.2.14
* Rhoconnect 3.0.0
All above software are installed on servers by executing the following commands:
sudo yum install make patch gcc-c++ zlib-devel curl-devel pcre-devel readline-devel openssl-devel
sudo rpm -i rhoconnect-3.0.0.noarch.rpm
### Tested bench application (benchapp)
* Simulates creating multiple objects
* 100 concurrent users
* 1 device per user
* 20 object dataset
* each object has 7 attributes
### Prepare bench application for running on servers
1. Go to rhoconnect/bench directory and configure benchapp application:
sudo cp -R benchapp /opt/nginx/html
cd /opt/nginx/html/benchapp
sudo /opt/rhoconnect/bin/bundle install --without=test development
sudo mkdir public
sudo chown -R nginx:nginx /opt/nginx/html/benchapp
2. Edit '/opt/nginx/conf/conf.d/rhoconnect.conf' file and point root entry to /opt/nginx/html/benchapp/public directory:
server {
listen 80;
root /opt/nginx/html/benchapp/public;
passenger_enabled on;
}
3. Start redis and nginx servers:
sudo /etc/init.d/redis start
sudo /etc/init.d/nginx start
### Benchmark results for large (2 cores) server
Request clientcreate : min: 0.0058, max: 1.0360, avg: 0.1138, err: 0, verification err: 0
Request ack-cud : min: 0.0258, max: 0.1998, avg: 0.0642, err: 0, verification err: 0
Request clientlogin : min: 0.0056, max: 0.0958, avg: 0.0128, err: 0, verification err: 0
Request get-cud : min: 0.0234, max: 0.1777, avg: 0.0476, err: 0, verification err: 0
Request create-object : min: 0.0526, max: 0.3074, avg: 0.1242, err: 0, verification err: 0
Throughput(req/s) : 8.40951725551376
Throughput(req/min): 504.571035330826
### Benchmark results for extra large (4 cores) server
Request clientcreate : min: 0.0080, max: 1.0194, avg: 0.0944, err: 0, verification err: 0
Request ack-cud : min: 0.0298, max: 0.1381, avg: 0.0573, err: 0, verification err: 0
Request clientlogin : min: 0.0091, max: 0.0591, avg: 0.0152, err: 0, verification err: 0
Request get-cud : min: 0.0259, max: 0.1679, avg: 0.0443, err: 0, verification err: 0
Request create-object : min: 0.0618, max: 0.2778, avg: 0.1112, err: 0, verification err: 0
Throughput(req/s) : 8.19117682939103
Throughput(req/min): 491.470609763462
### Benchmark results for high-cpu extra large (8 cores) server
Request clientcreate : min: 0.0083, max: 1.0162, avg: 0.0741, err: 0, verification err: 0
Request ack-cud : min: 0.0277, max: 0.1803, avg: 0.0525, err: 0, verification err: 0
Request clientlogin : min: 0.0092, max: 0.0681, avg: 0.0147, err: 0, verification err: 0
Request get-cud : min: 0.0259, max: 0.1729, avg: 0.0474, err: 0, verification err: 0
Request create-object : min: 0.0599, max: 0.2850, avg: 0.1102, err: 0, verification err: 0
Throughput(req/s) : 8.08355941878833
Throughput(req/min): 485.0135651273
All above results produced by bench script cud_script hosted by large (2 cores) EC2 server.