Ruby vs PHP performance

Update: February 1, 2009

Many people have pointed out that using count($a) > 0 in PHP is inefficient and should be replaced with !empty($a):

“In ruby while !a.empty? does not recount the array at every loop.
In php you do count the array each iteration which completely mistake the results…”

So PHP doesn’t store the array length, and a count() call iterates through the entire array just to find out the number of items?!?!?!

Leaving aside the issue that that’s kind of stupid of PHP, I’ve updated the code and the corresponding numbers and graph.

I’ve also re-run the benchmarks against newer software versions (Ubuntu 8.04) and newer Ruby implementations such as Ruby Enterprise Edition, Ruby 1.9 and JRuby.

It’s a well-known stereotype that “Ruby is slow”. People say it on Slashdot. People say it on OSNews. People say it on my MSN contact list. A few quotes:

  • “If you care even a little about performance, you don’t use Ruby.”
  • “You thought Python is slow? Well Ruby is even slower!”

Another stereotype we hear is that “Ruby on Rails doesn’t scale”. Most of the people who say that seem to use PHP, so they imply that PHP does scale.

How much of this is true? If there’s anything I’ve learned in the past few years, it’s that I shouldn’t always take public opinion seriously. PHP is not really a great language; its interpreter and perhaps even the language specification (if that even exists) has many quirks. My friend Ninh has worked on PHP in the past. He has exposed many of those strange quirks in his blog (which is, unfortunately, down because of a recent data loss incident).

I can’t imagine that PHP is faster than Ruby (the language; I’m not talking about Rails). If Ruby on Rails is slower than PHP, then how much of that is to blame on the language, and how much on the framework? Let’s put this to the test.

Goal of and rationale behind this test

We must first agree on the following premises:
People claim that Ruby is so slow that it’s impossible to build anything in it that’s sufficiently fast for “production environments”. (1)
Yet at the same time, they accept PHP as being fast enough. (2)

From (1) and (2) follows this question: Is it possible at all to write a web framework in Ruby that’s faster than PHP?

If (1) and (2) are true, i.e. if Ruby web apps are indeed slower than PHP web apps, then the culprit might be:

  • The web framework that the Ruby web app is written in. If this is the case, then it is possible to write a faster web framework that matches PHP’s performance. (3)
  • Ruby itself, i.e. the language. If this is the case, then it does not matter what web framework we use or how we write it, it will always be slower than equivalent frameworks PHP. (4)
  • A combination of both, in which both parts play a significant role. (5)

The goal of this benchmark is to find out whether (4) is true. (4) is what people have been arguing for years. And that’s assuming that people know what they’re talking about; they often don’t, and confuse the Ruby language with Ruby on Rails the framework, or even with web server Ruby integration modules.

Benchmarking PHP and Ruby (and Perl, Python, C++)

I want to benchmark PHP and Ruby’s code execution speed.

  • I specifically do not want to benchmark code parsing speed. Ruby on Rails apps run in application servers, so all code is only parsed once. PHP has stuff like PHP Accelerator/Zend Accelerator (or whatever those products are called) which cache the parse tree.
  • I’m not trying to benchmark HTTP request processing. This is only about code execution speed.

So, I decided to implemented the mergesort algorithm in PHP and Ruby. I’m not trying to benchmark sorting speed, but code execution speed. Both Ruby and PHP have builtin sort functions which are written in C, but that’s not the point. The scripts will sort an array of 100 numbers (which where randomly generated and hardcoded into the source code), and perform this sorting 3000 times. The execution times, in seconds, are as follows:
benchmark-with-mergesort.png
As you can see, I’ve also written a Perl, Python and C++ version just for comparison fun. :)

The sources code of the test programs can be found on the bottom of this blog post. A few notes:

  • Operating system: Ubuntu Linux 8.04
  • Hardware: Intel Core 2 Duo T5300 1.73 Ghz, 2 GB RAM
  • Versions:
    • PHP 5.2.4-2ubuntu5.4
    • Ruby Enterprise Edition 1.8.6-20090201
    • Ruby 1.9.1p5000 (2009-02-01 trunk 21929)
    • JRuby 1.1.7
    • Perl 5.8.8
    • Python 2.5.2
  • The test programs not only test raw CPU execution speed, but also memory management speed. As you can see in the merge() functions, it creates a new array, and modify the given arrays. As a result, the mergesort() functions will create a lot of objects that will have to be garbage collected.
  • Ruby uses a conservative mark-and-sweep garbage collector. Perl and Python both use reference counting. The C++ test program uses Boost’s shared_ptr, and thus uses reference counting for memory management. I’m not sure how PHP manages its memory, or whether it does it at all, because all objects are freed after an HTTP request anyway.
  • The C++ test program is compiled with g++ 4.2.4, with -O2.
  • By using Psyco, a JIT optimizer for Python, the Python test program’s execution time went down to a whopping 0.640 seconds. That’s even faster than my C++ version (!)

But as you can see in the graph, Ruby beats PHP marginally even with a full-blown mark-and-sweep garbage collector. Ruby’s garbage collector was run several dozens times while running the test program, and Ruby still beats PHP in execution time.

Ruby 1.9 and JRuby are significantly faster than PHP, beating even Python.

I have no choice but to conclude that all the “Ruby is slow” stereotypes are bollocks. PHP the language is slower, yet we do not see anybody complaining about PHP “scalability”. The Rails framework could be faster, yes, but at least now we know the language is not as big as a bottleneck as some people might think.
Wait! Do I see a flame thrower there? Read on before you reply!

“But but but… you didn’t benchmark the web environment!”

Yes. I did not benchmark the HTTP request handling, and this is intentional. People claim that Ruby is so slow, that it is impossible to write anything in it that’s acceptably fast. This benchmark proves that, as a language, Ruby is faster than PHP. Therefore, if people can write frameworks in PHP that’s acceptably fast, then it is also possible to write frameworks in Ruby that’s acceptably fast.

Note that I do not make any claims about Ruby on Rails’s speed compared to typical PHP applications.

“Why don’t you use the language’s builtin sort function? This benchmark is crap!”

Implementing sort in the languge instead of using the builtin sort function is intentional. The goal of the benchmark is to test *code execution speed*, not to find out which language can sort faster. This benchmark is a good test because it tests different language primitives, like function calls, comarison operators, arrays, etc. I might as well have implemented A* graph searching, but mergesort is easier.

Appendix: Source codes

79 Comments »

  1. michael depetrillo said,

    July 6, 2008 @ 9:22 pm

    Your PHP script is seriously flawed. You shouldn’t use count() in a while loop or the array will be unnecessarily recounted during each iteration.

  2. sammy said,

    July 17, 2008 @ 1:24 pm

    Surprise! I wrote the perl code for the sort mentioned above (TMTOWTDI). I achieved better speed than Python. Perl is the good for most things we can imagine of. Modperl is wonderful beyond words. Perl’s downside are weird syntax and sucking catalyst framework.

    The author of this post clearly proves that Ruby is fast as a language (But the language’s popular framework is slow.)

    Thinking further !?
    ————-

    a) Ruby
    ——-
    1) Ruby on Rails is heavier but an eye-opener framework for all of us
    2) Since, Ruby is fast as a language, we need to use lighter frameworks like merb along with DataMapper for top speeds
    3) Ruby’s library completeness,documentation and licensing issues are to be taken into consideration before starting a project

    b) Python
    ——–
    1) Python is moving up in popularity but the language is still slow for my day-to-day work. I tried with PyPy, Pyrex, Psyco but had much difficulties converting OOP code to Pyx file. So, the applicability of these modules is still experimental. The speed I got using these modules for an application that reads and parses XL data was still lesser than my normal perl version. Also, in perl version I don’t care about C language, conversion issues etc. all. and thus less sweat and fast prototyping.
    2) Django is great. It’s the best in the world. (My Rating–u may disagree:-): Django>ROR>Pylons>TGears>CakePHP>CodeIgniter>Symphony

    c) PHP
    ——
    1) Does everything/anything for me but is still slow for algorithms and large datasets. So, I use it with control for large projects.
    2) Not much third party modules for heavy calculations, graphics (Obvious due to short life of web page)

    d) Perl
    ——
    1) I write legible/Maintainable code in perl. Develop webapps with Modperl but hate catalyst framework. Mason based templating is nice but I create small files that serve as templates for my small needs. Thus I bypass Mason, Catalyst and yet obtain top speeds.
    2) Perl is the fastest language in web domain that needs to be well written (very much possible to write legible code) or else we have red eyes to understand what we wrote.

    Love all,
    Sammy

  3. CaViCcHi said,

    August 22, 2008 @ 1:13 am

    What I really think is that u should choose a language according to their functionalities…
    I really trust C/PHP when we are talking about fsockopen and API communications, if you are talking about echo or print… then why are you using php or ruby? just get back to bare ASP…

    Sorry but this benchmark is really not important, and the information it provides are useless… if you know how to write code and how to make it work really sharp, then u don’t even need to change your language. I don’t really know why every programmer is called so when uses someone elses framework, guys you should develop your own libraries and then use them… so you can cut time in developing!

  4. 50个非常有用的PHP工具 | 胡言乱语 said,

    January 21, 2009 @ 9:26 am

    […] PHP是使用最为广泛的开源服务器端脚本语言之一,当然PHP并不是速度最快的,但它却是最常用的脚本语言。这里有50个有益的PHP工具,可以大大提高你的编程工作: […]

  5. Sean Grove said,

    January 21, 2009 @ 11:51 am

    Interesting article, thanks for this.

    Not sure what he difficulties are for others to understand your methodology. Given the same series of instructions/low-level tasks with roughly equal optimizations, how does each language perform.

    Thanks for the read!

  6. Stephen Bannasch said,

    February 5, 2009 @ 12:54 am

    Hongli,

    I wrapped your mergesort code in Ruby’s benchmark library and did a more detailed comparison of 1.8.6, 1.9, and JRuby running on Java 1.5, 1.6, and 1.7.

    http://blog.concord.org/archives/24-Comparison-of-Ruby-1.8.6-1.9-and-JRuby-running-on-Java-1.5-1.6-and-1.7.html

    Using the Benchmark library and running a “rehearsal” before the main measurement results in better timings for JRuby because the JVM ends up optimizing the code that is run the most.

    The fastest “safe” result shows that JRuby running on Java 1.6 in –server mode is a little more than twice as fast as Ruby 1.9.

    I also have measurements using the “–fast’ option with JRuby that give better results but the use of this option is still experimental.

  7. roger said,

    February 14, 2009 @ 7:43 pm

    all right if I include the ruby version in the ruby benchmark suite?

    I think my one response to this is “please don’t take this as an indication that rails is fast–that is NOT to be inferred from these results.”
    Glad to hear from one commentor that RoR is second only to Django to them.
    Cheers!
    -=r

  8. Hongli Lai said,

    February 14, 2009 @ 9:34 pm

    Sure, go ahead.

  9. 50 Extremely Useful PHP Tools - المجموعة التقنية PSD Groups said,

    April 6, 2009 @ 7:32 am

    […] PHP isn’t quick, but it is the most used scripting language in practice; it has detailed documentation, a huge […]

  10. someone said,

    April 30, 2009 @ 4:21 pm

    according to http://www.php.net/manual/en/language.types.array.php PHP’s array is not an usual array. It is an ordered map. If you need to test fairly, you shouldn’t use the built-in PHP array.

  11. Meow said,

    May 18, 2009 @ 10:47 pm

    Really now….

    Reality is what language to use is often dependant on what one is trying to do. For example, if one is creating client server app’s meant to work across large a windows environment then ASP.NET, IIs, and Microsoft are usually your best choice (though Adobe is working towards this as well).

    PHP is “Messy” but its popularity has spawned slews of documentation, frameworks and more. Its not compiled, that sux. But, one can create compiled extensions… so thats good… sorta.

    Ruby, Python on and on and on… Point being… There is no specific standard to suit “all forms” of applications. PHP for enterprise distributed applications is a nightmare, yet Java handles such things albeit not painless like. If one is making a social networking application thats going to deal with several thousands of participants PHP can do the trick. But, if one is making Facebook where there are going to be millions, PHP is dead on arrival.

    Now to me whats most amazing is essentially C++ and Java are the speed demons in overall performance, scalability and then some. Java has a great deal of support, frameworks etc. But, C++ does not yet is older and more established in “general programming” than all others. That’s weird.

    I’d seem to me as the “fastest” run time environment, in fact, fast enough to be what PHP and countless other web apps even Apache HTTP Server… C++.

    With that said one would think that there would be OODLES of native C++ Web frameworks out there and the rest could go home. Granted, C++ one need deal with perhaps a bit more in the way of hassles but I am sure nothing that could not be overcome by the myraids of web coders out there.

  12. Hans said,

    June 28, 2009 @ 2:30 pm

    “But, if one is making Facebook where there are going to be millions, PHP is dead on arrival.”
    and still Facebook is built using PHP…

    Personally I like PHP and if you write your coded right then it is fast enough.

  13. Is Ruby being blamed for nothing? « Pacoup.com said,

    July 3, 2009 @ 2:55 pm

    […] together in a neat package, but they tend to slow your application a bit. Basing ourselves on Hongli Lai’s blog post, it’s quite obvious that Ruby apps using Ruby previous to version 1.9 would be slower than […]

  14. Luis José said,

    July 16, 2009 @ 12:54 am

    Hans, well, twitter is written in RoR so…

  15. Rodrigo said,

    July 27, 2009 @ 10:42 pm

    Could you update this and use Python 3.0.1 and JPython??

  16. Rodrigo said,

    July 27, 2009 @ 10:51 pm

    Jython*!

  17. Hans said,

    August 11, 2009 @ 9:39 am

    @Luis Jose

    >>”Hans, well, twitter is written in RoR so…”

    Essential parts of Twitter is again written in Scala. (Threading issues).

    CodeIgniter and C++API for Threading could have handled Twitter with ease.
    (This combo can Kick assss of all language frameworks out there)

    Not putting you down. Ruby is a gr8 language but there is still 2 years time to mature in terms of :

    1) consistency – I still read people’s problems installing Ruby in Windows XP box.
    2) speed – Ruby 1.9 speed is not enuf for me. I need it to be close to Java.
    3) libraries – Worst part !!! Still there are no free charting libraries in Ruby (Graphviz suckss assss),
    Wx-Ruby and Fx-Ruby are half-baked
    Sci/Math modules – where are they hiding? Can I do PCA analysis using Ruby?
    ChemRuby – lot of bugs in it.

    4)Others – Till Ruby fights java, there is no victory to it. Twitter people put Ruby down and Scala up.
    Compatibility – Still I can’t use Ruby 1.9 on RoR
    I need a browser plug-in like JRE plug-in for my apps.
    Without using JRuby, I need a pure ruby solution for my tasks.
    Cheers,
    Hans

  18. Amit Verma said,

    August 21, 2009 @ 7:22 am

    If you write something in a loop in PHP like –
    count($a) > 0

    And if PHP is trying to evaluate the size of array using count() function than PHP is doing it RIGHT.

    PHP needs to do what is “asked to do” and that is perfectly legitimate.

    There may be a chance that the size of the array is changing/modifying due to some operation inside the loop? In that case, if PHP is minding it’s business than that’s perfectly alright. If you’re sane enough you’ll write it this way –

    $arrSize = count($a);
    $arrSize > 0

    {Pretty Easy, Hunh??}

    Hope I made my point clear??


    Amit Verma

  19. 悠閒莊園 » Blog Archive » Scripting language 之選擇 said,

    October 10, 2009 @ 9:50 pm

    […] 今天在這個赖洪礼的 blog – Ruby vs PHP performance中找到點跟據… 然而,我個人對 python,ruby 等都不很熟 ruby 更是沒做過有用的東西出來… 而且我自已在測試 ruby 1.8 時速度也沒很快,跟 php 差不多… Gentoo 也沒 ruby 1.9 所以現階段應不會多嘗試…但仍值得參考… […]

  20. retry said,

    November 4, 2009 @ 8:28 am

    I ran your code on my dev box a deb 5.0.3 (lenny) set up with stock php/ruby packages

    $ php -v
    PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009 22:16:23)
    $ ruby -v
    ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

    $ time ./mergesort.php

    real 0m6.303s
    user 0m6.200s
    sys 0m0.032s
    $ time ./mergesort.rb

    real 0m8.043s
    user 0m7.752s
    sys 0m0.292s

    But this one simple test is hardly enough to evaluate an entire language. These guys have been benchmarking languages for years:
    http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=php&lang2=ruby&box=1

    php is generally faster, although ruby 1.9 does better. Jruby doesn’t impress me at all.

  21. Exit said,

    January 9, 2010 @ 10:13 am

    Anyone run tests with compiled PHP (Zend encoded for instance)?

    That also should be considered.

  22. Robert K said,

    November 23, 2010 @ 11:18 pm

    Javascript version for testing Node.js (V8 interpreter):

    http://pastie.org/1321310

    Takes 0.26 seconds on my system (.vs. 4.6 seconds for ruby 1.8.7)

  23. 50个非常有用的PHP工具 | haohtml's blog said,

    December 20, 2010 @ 2:43 am

    […] PHP是使用最为广泛的开源服务器端脚本语言之一,当然PHP并不是速度最快的,但它却是最常用的脚本语言。这里有50个有益的PHP工具,可以大大提高你的编程工作: […]

  24. 50 个非常有用的PHP工具(转) - 王超 - 博客 BLOG said,

    March 9, 2011 @ 4:28 am

    […] PHP是使用最为广泛的开源服务器端脚本语言之一,当然PHP并不是速度最快的,但它却是最常用的脚本语言。这里有50个有益的PHP工具,可以大大提高你的编程工作: […]

  25. Why PHP? When do we need PHP? PHP vs Java vs Ruby vs ASP……. « Sujai SD said,

    March 26, 2011 @ 4:51 am

    […] means anything without numbers…Some Performance test gives very less score for PHP – Is PHP slower than Ruby? Another one here But the way language works and the bench marks done may not be the right way of […]

  26. dean said,

    July 14, 2011 @ 5:20 pm

    Ruby is marginally faster than PHP until you do anything with the database. If you’re using Rails forget about performance.

    Of course I’ve seen many, many other benchmarks of Ruby 1.9 and 1.91 that show it as significantly slower than 1.8.6 too, so I’m not sure if this is even legit or not.

  27. PHP开发工具。 | suliuer said,

    August 8, 2011 @ 7:28 pm

    […] PHP是使用最为广泛的开源服务器端脚本语言之一,当然PHP并 不是速度最快 的,但它却是 最常用的脚本语言 。这里有50个有益的PHP工具,可以大大提高你的编程工作: […]

  28. PHP开发工具大全(经典呀!不看后悔呀!) - WEB开发笔记 said,

    August 30, 2011 @ 3:30 pm

    […] PHP是使用最为广泛的开源服务器端脚本语言之一,当然PHP并不是速度最快的,但它却是最常用的脚本语言。这里有50个有益的PHP工具,可以大大提高你的编程工作: […]

  29. 50个非常有用的PHP工具 | 林洪优的博博 said,

    July 16, 2013 @ 6:16 am

    […] PHP是使用最为广泛的开源服务器端脚本语言之一,当然PHP并不是速度最快的,但它却是最常用的脚本语言。这里有50个有益的PHP工具,可以大大提高你的编程工作: […]

RSS feed for comments on this post · TrackBack URI

Leave a Comment

You must be logged in to post a comment.