An update to my previous post, here are some results for Apache/PHP/MySQL. I used Apache 2.2.3 (prefork), PHP 5.2.1, MySQL 5.0.27, and APC 3.0.13. These were custom compiled pretty much the same options the CoolStack uses, with Sun Studio 11.

APC is the PHP caching extension for PHP, and the apc.stat setting in the php.ini tells it to check if the php file has changed and if it has, recompile it and cache it. Using apc.stat on or off didn’t seem to make much of a difference. These tests were done with apc.stat on. For Apache I have mem_cache turned on as well.

5 rows and 1 column of MySQL returning:

# ./ab -n 50000 -c 500 x.x.x.x:30000/test.php
This is ApacheBench, Version 2.0.41-dev < $Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, www.zeustech.net/
Copyright (c) 2006 The Apache Software Foundation, www.apache.org/

Server Software: Apache/2.2.3
Server Hostname: x.x.x.x
Server Port: 30000

Document Path: /test.php
Document Length: 67 bytes

Concurrency Level: 500
Time taken for tests: 4.793971 seconds
Complete requests: 50000
Failed requests: 48583
(Connect: 0, Length: 48583, Exceptions: 0)
Write errors: 0
Total transferred: 11475208 bytes
HTML transferred: 823930 bytes
Requests per second: 10429.77 [#/sec] (mean)
Time per request: 47.940 [ms] (mean)
Time per request: 0.096 [ms] (mean, across all concurrent requests)
Transfer rate: 2337.52 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 4 3.8 4 30
Processing: 15 41 59.6 29 1036
Waiting: 14 40 59.1 28 1032
Total: 23 46 59.7 33 1042

36 rows and 6 columns of MySQL returning:

Requests per second: 10448.16 [#/sec] (mean)

134 rows and 7 columns of MySQL returning:

Requests per second: 801.05 [#/sec] (mean)

Performance really degrades on the 134 row test and I think MySQL is the bottleneck here. More to come on this when I get MySQL recompiled.

UPDATE: This is the results without APC cache turned on at all, with 36 rows and 6 columns:

Requests per second:    2242.18 [#/sec] (mean)

As you can see APC cache helps a lot, it went from 10,000 req/sec to 2,000 req/sec.

These tests were done with 2.2.3 prefork and 2.2.4 with worker. Both of them were tested with mem_cache and without. mem_cache seemed to really improve the performance of prefork but it didn’t do anything for worker, it actually hurt the performance by a little bit. Worker also seemed to drop a core file when it got too much traffic. The index.html is a small 175 byte file that loads a 5 of the apache powered by gif files and the httpd.conf were both stripped with minimal modules loaded. From the transfer rate apache bench shows I think the bottleneck here is the network.

The mem_cache settings:

CacheEnable mem /
MCacheSize 320000
MCacheMaxObjectCount 1000
MCacheMinObjectSize 1
MCacheMaxObjectSize 4048

Apache 2.2.3, prefork with mem_cache (-n 100000 -c 1000):

Requests per second: 11566.21 [#/sec] (mean)

Apache 2.2.3, prefork without mem_cache (-n 100000 -c 1000):

Requests per second: 9492.40 [#/sec] (mean)

Apache 2.2.4, worker with mem_cache (-n 100000 -c 1000):

Requests per second: 12118.41 [#/sec] (mean)

Apache 2.2.4, worker without mem_cache (-n 100000 -c 1000):

Requests per second: 12413.07 [#/sec] (mean)

Last week I missed the Lamb of God/Machine head tour, but I did not miss the Blackbox Tour when it came rolling through town. It was definitely a cool event to check out and I took some pics and added a blackbox section to my blog.

I wrote a ksh script and had it source a file for variables using:

. .properties

It kept saying .properties not found.. when I ran truss on it:

21651: lwp_sigmask(SIG_SETMASK, 0×00000000, 0×00000000) = 0xFFBFFEFF [0×0000FFFF]
21651: close(2) = 0
21651: fcntl(3, F_DUPFD, 0×00000002) = 2
21651: close(3) = 0
21651: close(2) = 0
21651: fcntl(10, F_DUPFD, 0×00000002) = 2
21651: close(10) = 0
21651: fcntl(2, F_SETFD, 0×00000000) = 0
21651: lwp_sigmask(SIG_SETMASK, 0×00020000, 0×00000000) = 0xFFBFFEFF [0×0000FFFF]
21651: open64(”/usr/sbin/.dproperties”, O_RDONLY) Err#2 ENOENT
21651: open64(”/usr/bin/.dproperties”, O_RDONLY) Err#2 ENOENT
21651: open64(”/usr/sfw/bin/.dproperties”, O_RDONLY) Err#2 ENOENT
21651: lwp_sigmask(SIG_SETMASK, 0×00000000, 0×00000000) = 0xFFBFFEFF [0×0000FFFF]
./deployer[295]: .dproperties: not found
21651: write(2, ” . / d e p l o y e r [ 2″.., 42) = 42
21651: _exit(1)

It shows it searching through my PATH (/usr/sbin:/usr/bin:/usr/sfw/bin), and not finding the .dproperties, so I set my PATH to have my current working dir and no more problems.  This can drive you nuts.

I jumpstarted my blade last night to build 56 and just wanted to make some notes here for later jumpstarts. As of Solaris 10 Update 3 there’s a ’service_profile’ option you can add to the sysidcfg file to set if you want limited or wide open services to run. service_profile=limited_net or service_profile=open. Here’s my sysidcfg file I had no problems with:

system_locale=en_US
timezone=US/Central
timeserver=localhost
network_interface=primary
{dhcp
protocol_ipv6=no}
terminal=dtterm
name_service=NONE
security_policy=NONE
root_password=crypt
nfs4_domain=dynamic
service_profile=limited_net

And I must say that Gnome 2.16 looks nice.

Ruby/Rails install on Solaris 10 is so easy, it takes about 5 minutes:

Download Ruby 1.8.5, configure:

# ./configure –prefix=/opt/local –enable-pthread; make; make install

Download RubyGems and run:

# /opt/local/bin/ruby setup.rb

set PATH and install rails:

# export PATH=/opt/local/bin:$PATH
# gem install rails –include-dependencies

and with the benefits of Zones, DTrace, and Duckhorn, why would you want to run it on anything else?

Run RoR on Sun Java System Web Server.

A while back I did a try and buy on a x2100, the first one they released.. the one everyone hates. That one was the only one I could afford to even try and buy, and I started to write up a review on it to try and get it free. Then life threw a thousand obstacles in my way and I never had time to finish it. But I did save some of the results and put them in nice little pretty graphs:

*Tests were done on Solaris 10 update 1 and 2 using ZFS and UFS. Hard drive tests were done with bonnie and MySQL with sysbench.

Sequential input KB/sec:

Sequential ouput KB/sec:

Random seeks/sec:

MySQL requests/sec:

How did I make these awesome graphs? Welcome to the world of Ruby and Gruff. Ruby is the kick ass language and Gruff is the gem you need for the graphs. Can be a pain to get installed though, requires imagemagick. For more examples check here. If you haven’t started using Ruby yet you should definitely give it a shot, or check out Joyent.

Here’s the ddclient SMF manifest I made for this entry.

ddclient.xml

I only had one machine to use for benchmark testing and apache bench seemed to be the only benchmark to hold the amount of requests. Siege would report around 2-3,000 req/sec but would bum out after that. This was Apache 2.2.3 compiled with Sun Studio 11 on Solaris 10 1/06:

export CFLAGS=’-DSSL_EXPERIMENTAL -DSSL_ENGINE -xO4 -xtarget=generic’
export LDFLAGS=”-R/usr/ucblib”

# ./configure –prefix=/path/apache2 –enable-mods-shared=all –enable-cache –enable-mem-cache –enable-ssl –with-mpm=prefork –enable-so –enable-rule=SSL_EXPERIMENTAL –with-ssl=/usr/sfw

Amount of time it took to build Apache 2.2.3 with 24 threads:

# time dmake -j 24
real 6m4.925s
user 18m12.183s
sys 13m37.746s

Results:

# ab -n 40000 -c 30 x.x.x.x:30000/index.html
This is ApacheBench, Version 1.3d < $Revision: 1.73 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, www.zeustech.net/
Copyright (c) 2006 The Apache Software Foundation, www.apache.org/

Benchmarking x.x.x.x (be patient)
Completed 4000 requests
Completed 8000 requests
Completed 12000 requests
Completed 16000 requests
Completed 20000 requests
Completed 24000 requests
Completed 28000 requests
Completed 32000 requests
Completed 36000 requests
Finished 40000 requests
Server Software: Apache/2.2.3
Server Hostname: x.x.x.x
Server Port: 30000

Document Path: /index.html
Document Length: 125 bytes

Concurrency Level: 30
Time taken for tests: 4.114 seconds
Complete requests: 40000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 16001200 bytes
HTML transferred: 5000375 bytes
Requests per second: 9722.90 [#/sec] (mean)
Time per request: 3.09 [ms] (mean)
Time per request: 0.10 [ms] (mean, across all concurrent requests)
Transfer rate: 3889.45 [Kbytes/sec] received

Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.9 0 16
Processing: 0 2 0.8 2 14
Waiting: 0 2 0.7 1 14
Total: 0 3 1.3 2 18

Percentage of the requests served within a certain time (ms)
50% 2
66% 3
75% 3
80% 3
90% 4
95% 6
98% 7
99% 9
100% 18 (last request)

httpd-mpm.conf prefork settings:

ServerLimit 8192
MaxClients 2048

StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 2048
MaxRequestsPerChild 0
ListenBacklog 8192
ServerLimit 2048

I couldn’t get it to go much over 9000-10000 req/sec with any more tweaking to the apache config. From checking other people’s results it looks like I should be able to get around double that using Nevada and certain OS tunings that I don’t have here. More to come later with PHP/MySQL/SSL.

Recently I’ve been playing with ddclient because I want to start running some things from my house. If you don’t know what ddclient is, it’s a perl program to run and update your free dynamic dns provider with your latest IP. ddclient is mainly supported on Linux (no mention of Solaris anywhere in the docs) but it’s written in Perl so it should work for Solaris.

The first problem I had was ddclient doesn’t have my Dell router as a supported router to grab the IP from. This is easily solved in the configuration file you specify custom config’s via the option fw-skip=’patter’ which says to skip any IP addresses before ‘pattern’. After that tested with ./ddclient -daemon=0 -query shows it grabbing the right IP and information.

Now that I had it grabbing the right IP, running ddclient as a daemon wasn’t doing anything or giving me any error:

# ./ddclient -daemon 300
# ps -ef | grep dd
#

The ddclient also has -verbose which wasn’t giving any information, and then I was starting to think something was wrong with my Perl since I am using a 64 bit Perl from Coolstack, so I ran it with truss to see what was happening:

# truss -a -f ./ddclient -daemon 300

7040: stat64(”/usr/local/lib/perl5/5.8.8/sun4-solaris/IO/Socket/SSL.pmc”, 0xFFBFF740) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/5.8.8/sun4-solaris/IO/Socket/SSL.pm”, 0xFFBFF610) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/5.8.8/IO/Socket/SSL.pmc”, 0xFFBFF740) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/5.8.8/IO/Socket/SSL.pm”, 0xFFBFF610) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/site_perl/5.8.8/sun4-solaris/IO/Socket/SSL.pmc”, 0xFFBFF740) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/site_perl/5.8.8/sun4-solaris/IO/Socket/SSL.pm”, 0xFFBFF610) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/site_perl/5.8.8/IO/Socket/SSL.pmc”, 0xFFBFF740) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/site_perl/5.8.8/IO/Socket/SSL.pm”, 0xFFBFF610) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/site_perl/IO/Socket/SSL.pmc”, 0xFFBFF740) Err#2 ENOENT
7040: stat64(”/usr/local/lib/perl5/site_perl/IO/Socket/SSL.pm”, 0xFFBFF610) Err#2 ENOENT
7040: stat64(”./IO/Socket/SSL.pmc”, 0xFFBFF740) Err#2 ENOENT
7040: stat64(”./IO/Socket/SSL.pm”, 0xFFBFF610) Err#2 ENOENT

To see the same thing with a DTrace one-liner:

# dtrace -n ’syscall::stat*:entry /execname == “ddclient”/ { trace(copyinstr(arg0)); } syscall::stat*:return /execname == “ddclient”/ { trace(arg0) }’

at the end of the truss it shows some missing SSL specific Perl modules. After looking, I noticed these didn’t exist on my system anywhere and I remembered a ssl option in the ddclient.conf file ssl=yes. I set this to ssl=no since I am not using SSL anyway.

# ./ddclient -daemon 300
# ps -ef | grep dd
root 7046 1 0 21:45:44 pts/2 0:00 /usr/local/bin/perl -w ./ddclient -daemon 300

Now I can write up a SMF manifest for this service and go to bed.