赖洪礼的 blog

This is my personal blog. For Phusion's corporate blog, please go to http://blog.phusion.nl

Censor banner | Show banner | (What's this?)

After weeks of teaching it hardcore computer science kung-fu, we’ve finally reached a stage in that we think our little baby is now all grown up. It is no longer solely dependent of our care, and from this moment on, Phusion Passenger is available for the masses.

Phusion Passenger has been the result of many many many (did I say many? I mean MANY) hours of hard work by both Hongli and I ( Ninh ), but it couldn’t have been made possible without the help of our beta testers. In particular, we’d like to thank Dallas of Dreamhost for being so patient and we’re looking forward to seeing Passenger being fully integrated in your hosting plans. Also, we’d like to thank Alex of Twitter for trying his best to test Passenger in time.

Also, we’d like to thank the Ruby on Rails core team for their insightful feedback and support, and in particular, we’d like to thank David Heinemeier Hansson, Jeremy Kemper and Pratik Naik for this.

Yukihiro Matsumoto (matz), thank you for the kind words on our project, it really kept us motivated.

Ryan Bates of Railscasts, thank you so much for doing the screencast on Phusion Passenger. We’re looking forward to meeting you in person at Railsconf and being able to thank you properly Depending on the paint we’re allowed to carry with us, shaking a green hand may be involved

Antonio Cangiano, thanks for the coverage. It was when we saw that a technical evangelist at IBM had written about us that it truly hit us that we were doing something important.

Russel Norris, you really should not underestimate the importance of your great sense of humor for an open source project .

Since we’re from the Netherlands, we’d also like to thank our fellow dutchmen Stefan Fountain of Soocial and Thijs van der Vossen of Fingertips for their support on this project.

Also, thank you Weyert de Boer for being a great friend and helping us streamline the markup of our website. Even though we haven’t been able to complete it yet, we’re confident that we’ll be able to finish this with your help and we thank you for this.

Lastly, we’d like to thank Hans Scholten of the University of Twente for offering to help us on writing a scientific paper on our work with Ruby Enterprise Edition and Phusion Passenger.

To you the reader, we hope that you’ll enjoy using Passenger and we thank you for your patience. We’ll be seeing you guys at Railsconf USA 2008 (and probably Apachecon USA 2008 as well! )

Be sure to check us out there, since we’ll be doing a talk on Phusion Passenger (mod_rails) and our garbage collector optimization work on MRI. The talk material will be based on our scientific paper to be published on eprints, so the tech savvy among you guys will be in for a treat .

For science and all things pragmatic, this was Hongli Lai and Ninh Bui of Phusion, The Computer Science Company, wishing you a good weekend as we’ll be seeing you guys monday again. We’re going to take that well-deserved time off now .

P.S. Even though we’re currently getting swamped with clients, that shouldn’t stop you from contacting us for business enquiries. As a matter of fact, we’re planning on expanding our team, so if you’re a computer scientist / developer located in the Netherlands with a lot of work experience and interested in working with us, please contact us as well

Ninh wrote this a while ago:

“Lastly, we’d like to take the opportunity to thank Ryan Bates of Railscasts for creating an excellent screencast on Passenger, and you’ll be able to see this for yourselves at the public release of Passenger this week. Needless to say, it totally kicks the *censored* out of our initial screencast ”

Are you guys ready?

So you guys are probably still waiting for the release of Passenger, and I’m pleased to say that we’ve been able to include the test results from Dreamhost (thanks Dallas ) in the first public release that you’ll see within this week. (Hold on just a little longer guys , with a bit of luck, Twitter’s results will come in as well prior to the release of Passenger this week).

Instead of writing an elaborate blog post, I’ll keep it concise for the time being as we’re currently getting swamped at Phusion.

Today, after planning to do this for quite a long time now, we went to go see our Operating Systems/Computer Architecture teacher, Ir. Hans Scholten to discuss the possibilities for receiving ECTS for our work on both Passenger and Ruby Enterprise Edition (our ruby branch containing our garbage collector optimization).

From this, it became clear that our work wasn’t something trivial and Hans elaborated on the possibilities for publishing this work scientifically. Needless to say, we’re more than thrilled about this. After all, we’re “Phusion, the computer science company” right?

All kidding aside though, we do intend on taking Hans up on his offer to write a scientific paper on our work, seeing as this is just too great an opportunity to ignore. With his help, we’re more than confident about the quality of its contents.

Lastly, we’d like to take the opportunity to thank Ryan Bates of Railscasts for creating an excellent screencast on Passenger, and you’ll be able to see this for yourself at the public release of Passenger this week. Needless to say, it totally kicks the *censored* out of our initial screencast .

To all the people that have submitted a voice sample, we’ll be crediting you on the screencast section of the Passenger as well .

Hang on in there for just a little longer people , we’re almost there!

Thanks,

Hongli Lai
Ninh Bui

We’ve written a robustness comparison between Phusion Passenger (mod_rails), Thin, Ebb and Mongrel. See Ninh’s blog for the article.

Here it is: the moment you’ve all been waiting for.

By the time of this writing, a lot of people have been claiming that Passenger will cause a revolution in the Ruby on Rails community. And even though we agree with them to a certain point ( ), we believe that one should always have a trump card in case all else fails. We’ve been working on our trump card for quite some time now in great secrecy.

Today, we’re pleased to announce this trump card to the world: Ruby Ain’t Hypertext Preprocessor (RAHP).

What is RAHP?

RAHP is a Ruby parser written in PHP, which will enable you to run all your Ruby applications within a PHP environment! This was possible due to the fact that Ruby and PHP are both Turing Complete.

Since PHP still enjoys more support among webhosts than Ruby on Rails, we’ve decided to combine the best of both worlds with RAHP. And as you may already have heard from a friend, PHP scales really well in Enterprise environments, unlike Rails applications.

But wait! It doesn’t end there! Using Zend optimizer we’ve been able to infer a tremendous performance boost, and we think we may need to retract our previous benchmark post on Ruby vs PHP. :$

Long story short:

• RAHP will enable you to run your Rails applications pretty much on every webhost out there!
• Because RAHP is based on PHP, your Rails applications will be able to use the thousands of existing PHP libraries with ease!
• It is common knowledge that PHP is extremely scalable (unlike Rails). So by the transitive property, your Rails applications will become scalable as well, with the help RAHP on Rails!

Come to think of it, we’re considering to discontinue our work on Passenger in favor of RAHP. We’ll have to sleep on it though.

Stay tuned!

Recently, Yukihiro Matsumoto (a.k.a. Matz, the creator of Ruby) blogged about Phusion Passenger (mod_rails) on his weblog. Since our Japanese is a bit rusty, i.e. non-existant , Matz was kind enough to mail us a summary in English from which we quote:

It is often said that Rails is weak on deployment; PHP runs fairly fast just by uploading scripts. Rails is slow on development mode, and requires restarting on production mode (and bit complex to configure). modrails might be the answer for it.

Needless to say, coming from Matz, this made us feel real warm and fuzzy from inside

Since the initial release of the teaser screencast, a lot of people have asked about Passenger’s (a.k.a. mod_rails) performance compared to Mongrel and Thin. Here’s the benchmark that you’ve all been waiting for.

In this benchmark, we compare the following software:

• Phusion Passenger (mod_rails), latest development version
• Mongrel 1.1.4
• Thin 0.7.1

System specification:

• Intel Core 2 Duo, T5300 @ 1.73GHz
• 2 GB RAM
• Ubuntu Linux 7.10
• Apache 2.2.4 (Ubuntu package)
• Ruby 1.8.6 (Ubuntu package)

Test applications:

• Typo 5.0.3. This is an out-of-the-box setup with only a hello world post. Uses semi-static caching.
• Petstore revision 360, an application that the JRuby guys use for benchmarking. At first it wouldn’t start in Passenger because of a bug Petstore: they forgot to add require 'forwardable' in app/models/cart_items.rb. I added this line myself.
• Eldorado 0.9.1. This is an out-of-the-box setup with only a single forum and a single post added.

Typo and Eldorado both use SQLite 3. Petstore uses MySQL, because for some reason Petstore performs very badly when SQLite 3 is used.

People have also recommended Lovd By Less as test application, but I couldn’t get it working (database setup failed).

The applications, when served by Passenger, are setup at the following test domains:

• http://typo.test/
• http://petstore.test/
• http://eldorado.test/

Both the Mongrel cluster and the Thin cluster are setup at http://mongrel_cluster.test/. (Only either Mongrel or Thin is running during the benchmark, not both at the same time.)

And of course, we always use the production environment.

Backend configuration:

• Default Apache configuration, as provided by Ubuntu.
• Passenger: RailsMaxPoolSize is set to 10. This means that Passenger will never spawn more than 10 Rails application instances. Note that Passenger is compiled without optimizations, so if you install Passenger yourself and benchmark it, you might find that Passenger is slightly faster.
• Mongrel cluster: a cluster of 10 servers for each test application, behind mod_proxy_balancer, using default settings:

  ProxyPass / balancer://mongrel_cluster/
  ProxyPassReverse / balancer://mongrel_cluster/
  <Proxy balancer://mongrel_cluster>
  	BalancerMember http://127.0.0.1:3000
  	BalancerMember http://127.0.0.1:3001
  	BalancerMember http://127.0.0.1:3002
  	BalancerMember http://127.0.0.1:3003
  	BalancerMember http://127.0.0.1:3004
  	BalancerMember http://127.0.0.1:3005
  	BalancerMember http://127.0.0.1:3006
  	BalancerMember http://127.0.0.1:3007
  	BalancerMember http://127.0.0.1:3008
  	BalancerMember http://127.0.0.1:3009
  </Proxy>

• Thin cluster: same configuration as Mongrel cluster.

Benchmark: Mongrel clusters

First we benchmark Typo served by Mongrel clusters. Because the actual application code is loaded by Mongrel during the first request, we do not want to count the first request’s time. So we run the following command to “warm up” the Mongrels, and discard its result:

ab -n 1000 -c 100 http://mongrel_cluster.test/

Next, we test 10,000 requests with 100 concurrent users:

ab -n 10000 -c 100 http://mongrel_cluster.test/

Result:

Concurrency Level:      100
Time taken for tests:   32.628480 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      54890000 bytes
HTML transferred:       51480000 bytes
Requests per second:    306.48 [#/sec] (mean)
Time per request:       326.285 [ms] (mean)
Time per request:       3.263 [ms] (mean, across all concurrent requests)
Transfer rate:          1642.83 [Kbytes/sec] received

We repeat these steps Petstore and Eldorado.

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   109.786026 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      40750000 bytes
HTML transferred:       37680000 bytes
Requests per second:    91.09 [#/sec] (mean)
Time per request:       1097.860 [ms] (mean)
Time per request:       10.979 [ms] (mean, across all concurrent requests)
Transfer rate:          362.47 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   70.889388 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      41130000 bytes
HTML transferred:       35530000 bytes
Requests per second:    141.06 [#/sec] (mean)
Time per request:       708.894 [ms] (mean)
Time per request:       7.089 [ms] (mean, across all concurrent requests)
Transfer rate:          566.60 [Kbytes/sec] received

Benchmark: Thin clusters

The benchmarking steps for Thin are the same.

Typo:

Concurrency Level:      100
Time taken for tests:   25.795470 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      54910000 bytes
HTML transferred:       51480000 bytes
Requests per second:    387.66 [#/sec] (mean)
Time per request:       257.955 [ms] (mean)
Time per request:       2.580 [ms] (mean, across all concurrent requests)
Transfer rate:          2078.78 [Kbytes/sec] received

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   96.874308 seconds
Complete requests:      10000
Failed requests:        42
   (Connect: 0, Length: 42, Exceptions: 0)
Write errors:           0
Non-2xx responses:      42
Total transferred:      40630350 bytes
HTML transferred:       37547028 bytes
Requests per second:    103.23 [#/sec] (mean)
Time per request:       968.743 [ms] (mean)
Time per request:       9.687 [ms] (mean, across all concurrent requests)
Transfer rate:          409.58 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   71.311031 seconds
Complete requests:      10000
Failed requests:        5
   (Connect: 0, Length: 5, Exceptions: 0)
Write errors:           0
Non-2xx responses:      5
Total transferred:      41137070 bytes
HTML transferred:       35518568 bytes
Requests per second:    140.23 [#/sec] (mean)
Time per request:       713.110 [ms] (mean)
Time per request:       7.131 [ms] (mean, across all concurrent requests)
Transfer rate:          563.34 [Kbytes/sec] received

Benchmark: Passenger

The benchmarking steps for Passenger are the same.

Typo:

Concurrency Level:      100
Time taken for tests:   25.131980 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      55200000 bytes
HTML transferred:       51260000 bytes
Requests per second:    397.90 [#/sec] (mean)
Time per request:       251.320 [ms] (mean)
Time per request:       2.513 [ms] (mean, across all concurrent requests)
Transfer rate:          2144.92 [Kbytes/sec] received

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   104.428742 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      40300000 bytes
HTML transferred:       36700000 bytes
Requests per second:    95.76 [#/sec] (mean)
Time per request:       1044.287 [ms] (mean)
Time per request:       10.443 [ms] (mean, across all concurrent requests)
Transfer rate:          376.86 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   72.587018 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      41660000 bytes
HTML transferred:       35530000 bytes
Requests per second:    137.77 [#/sec] (mean)
Time per request:       725.870 [ms] (mean)
Time per request:       7.259 [ms] (mean, across all concurrent requests)
Transfer rate:          560.47 [Kbytes/sec] received

Summary

Here’s a nice summary of the requests per second, in a table and in a graph:

Conclusion

I was pleasently surprised by the results. During Passenger’s development, our performance baseline was Mongrel behind mod_proxy. According to earlier (simple and naive) tests, Passenger performed similar to Mongrel. I expected Thin to be a bit faster than Passenger. These new benchmarks however suggest Passenger is faster than Mongrel, and is on par with Thin.

But you know what they say: there are lies, damn lies, and statistics. Your hardware and software is different, so if you benchmark it yourself, you’re likely to get (slightly?) different results.

I have good news and bad news.

Let’s start with the good news. Passenger (mod_rails) is almost ready for release. The code is as good as done (actually, it has been as good as done for well over a week now). The website is almost done. We’re just awaiting final test results from remaining beta testers.

The bad news: lots of people have commented about our current screencast, in particular about the now-infamous “scary green hand”. Being the perfectionists that we are, we’d like to redo the screencast before we release Passenger. Neither Ninh nor I are native English speakers, so we’re looking for a native English speaker who can help us with the voice-over part of the screencast. Preferably, someone with enough ‘gravitas’ in their voice, or in the case of a female voice, the ‘kindness’ should come out dripping from out of our audio speakers.

Would you like to volunteer? If so, please provide us with an audio sample. You can do this by either replying in this post, or by emailing us at “hongli at plan99 dot net” or “ninh dot bui at gmail dot com”! Thanks in advance.

Does anyone know a good Ruby on Rails app that I can use for benchmarking Passenger (mod_rails)? I don’t have any apps that I’m allowed to use for this purpose. Mephisto and Typo are out because they both use page caching, which would make the benchmark pointless.