Nginx vs OpenLiteSpeed vs Cherokee Benchmarks

Overview

Siege benchmark comparisons against the basic phpinfo PHP page for three web servers: Centmin Mod Nginx, OpenLiteSpeed (open source LiteSpeed Enterprise), and Cherokee. Cherokee has PHP caching enabled by default, so tests were run with cache on and off.

Server Configuration

VirtualBox CentOS 6.4 64bit
Xeon W3540 @3.2Ghz allocated 4 CPU threads
1500MB memory
20GB disk on 1TB Western Digital Black Caviar

PHP Info and Locations

Centmin Mod Nginx 1.4.2 (php-fpm 5.4.18 + APC Cache 3.1.13)
OpenLiteSpeed 1.2.4 (LSAPI PHP 5.4.18 + APC Cache 3.1.13)
Cherokee 1.2.103 (php-fpm 5.4.18 + APC Cache 3.1.13)

Source Configure and Compile Options

All 3 web servers are configured/compiled via source install. All used 4 worker processes/threads.

nginx -V
nginx version: nginx/1.4.2
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
TLS SNI support enabled
configure arguments: --sbin-path=/usr/local/sbin --conf-path=/usr/local/nginx/conf/nginx.conf
  --add-module=../ngx_pagespeed-release-1.6.29.5-beta --with-http_spdy_module
  --with-http_ssl_module --with-http_gzip_static_module ...

Siege Benchmark v3.0.2 Results

siege -q -b -c25 -r50 http://192.168.0.199/phpinfo.php

Table Comparison

Web Server	Trans/sec	Elapsed	Throughput	Avail
Nginx (1 worker)	428.08	2.92s	5.02 MB/s	100%
Nginx (4 workers)	609.76	2.05s	7.15 MB/s	100%
OpenLiteSpeed (1 worker)	432.53	2.89s	4.38 MB/s	100%
OpenLiteSpeed (4 workers)	886.52	1.41s	8.98 MB/s	100%
Cherokee (cache on)	545.85	2.29s	6.33 MB/s	100%
Cherokee (cache off)	249.80	5.00s	2.91 MB/s	99.92%

Raw Results

Centmin Mod Nginx (4 worker_processes)

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                   2.05 secs
Data transferred:              14.65 MB
Response time:                  0.04 secs
Transaction rate:             609.76 trans/sec
Throughput:                     7.15 MB/sec
Concurrency:                   24.53
Successful transactions:        1250
Failed transactions:               0
Longest transaction:            0.11
Shortest transaction:           0.01

OpenLiteSpeed (4 workers)

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                   1.41 secs
Data transferred:              12.67 MB
Response time:                  0.03 secs
Transaction rate:             886.52 trans/sec
Throughput:                     8.98 MB/sec
Concurrency:                   23.39
Successful transactions:        1250
Failed transactions:               0
Longest transaction:            0.10
Shortest transaction:           0.00

Cherokee (PHP cache enabled)

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                   2.29 secs
Data transferred:              14.50 MB
Response time:                  0.05 secs
Transaction rate:             545.85 trans/sec
Throughput:                     6.33 MB/sec
Concurrency:                   24.67
Successful transactions:        1250
Failed transactions:               0
Longest transaction:            0.13
Shortest transaction:           0.02

Conclusions

OpenLiteSpeed with 4 workers achieved the highest transaction rate at 886.52 trans/sec for non-cached PHP via LSAPI.
Nginx with fastcgi_cache would need to be configured for a fair comparison against Cherokee's default PHP cache.
Cherokee's PHP cache is labelled experimental; with cache disabled results fluctuated between 249-573 req/s.
Nginx scaled from 428 to 610 trans/sec going from 1 to 4 workers (42% improvement).