Performance comparison: Rails 1.2.6 vs 2.0.2

Featured post

passengerscreencastpreview-small.jpg

There’s still a lot of misinformation about Rails on the Internet. People are still claiming that Rails doesn’t scale, is too slow, etc. Rails 2.0 has been released for a while now, and a year ago someone blished a benchmark, in which it is claimed that Rails edge (what is now 2.0) is slower than 1.2. I’ve decided to benchmark Rails 1.2 and 2.0.

I’ve written two dummy applications, one for 1.2 and one for 2.0. The reason why I’m not just running the same app under 2.0 is because Rails 2.0 introduced a new cookie session store, among other things. It is claimed that this session store is much faster than the old default PStore cookie store.

So first I create a Rails 2.0 application:

mkdir bench
rails 2.0
cd 2.0
./script/generate scaffold Post title:string content:text created_on:timestamp updated_on:timestamp
rake db:migrate RAILS_ENV=production

Note that scaffolding in Rails 2.0 is no longer dynamic. The scaffold generator now generates a database schema and some static HTML and code. This is equivalent to Rails 1.2’s scaffold_resource.

I also modified routes.rb so that the root URL is mapped to the ‘posts’ controller. And of course, I delete ‘public/index.html’.

Next, I create a Rails 1.2 application:

cd ..
rails _1.2.6_ 1.2            # This is a nice RubyGems trick. It allows me to specify which Rails version I want to use.
cd 1.2
./script/generate scaffold_resource Post title:string content:text created_on:timestamp updated_on:timestamp
cp ../2.0/config/database.yml config/    # We want to use SQLite, just like Rails 2.0.
rake db:migrate RAILS_ENV=production

I also modified routes.rb so that the root URL is mapped to the ‘posts’ controller, and deleted ‘public/index.html’.

I ran both applications in Mongrel (with the ‘production’ environment). The results of ‘ab -n 5000 http://localhost:3000/’ are as follows:

  • Rails 1.2.6: 141.19 requests/sec
  • Rails 2.0.2: 214.76 requests/sec

Wow, what a difference! Rails 2.0 is 50% faster in a dummy application!

In Rails 1.2, a lot of time is spent in the session store. Let’s see what happens if we specify “session :off” in both applications:

  • Rails 1.2.6: 189.51 requests/sec
  • Rails 2.0.2: 246.69 requests/sec

Wow! Even with sessions off, Rails 2.0 is still 30% faster! So the cookie session store isn’t the only thing responsible for the performance improvement!

Edge Rails has even more performance improvements. Things are definitely going the right direction.

13 Comments »

  1. Eric Allam said,

    March 19, 2008 @ 4:59 am

    Hmm, another benchmark that completely ignores memory use and memory usage growth. I’ve been guilty of the same, but we really should be looking at memory usage just as important as req/s.

  2. 赖洪礼的 blog » Memory usage comparison: Rails 1.2.6 vs 2.0.2 said,

    March 19, 2008 @ 11:08 am

    […] benchmarked Rails 1.2 vs 2.0 in my last blog post, and Rails 2.0 turned out to be 30%-50% faster. Then Eric Allam said: […]

  3. David Mabelle said,

    March 19, 2008 @ 10:39 pm

    The major bottleneck still exist with using fixtures to import an existing database. It takes 30 seconds to restore a database in Postgresql. It takes a day to bring in the same data with rails.

  4. David Mabelle said,

    March 19, 2008 @ 10:57 pm

    Large fixtures (20 megs) can completely tie up all the memory or crash the process.

  5. links for 2008-03-20 « Bloggitation said,

    March 20, 2008 @ 1:20 am

    […] Performance comparison: Rails 1.2.6 vs 2.0.2 (tags: rails ruby sysadmin tuning) […]

  6. Crónica de una vida - Rendimiento de Rails 2.0 y el futuro Rails 2.1 said,

    March 20, 2008 @ 9:33 am

    […] Riding Rails nos informan que alguien ha hecho una comparativa demostrando que Rails 2.0 es entre un 30 y 50% más rápido que Rails 1.2.X. Además otro estudio […]

  7. links for 2008-03-20 « Brent Sordyl’s Blog said,

    March 20, 2008 @ 4:23 pm

    […] Performance comparison: Rails 1.2.6 vs 2.0.2 Even with sessions off, Rails 2.0 is still 30% faster! So the cookie session store isn’t the only thing responsible for the performance improvement! (tags: performance rubyonrails) […]

  8. tangfucius said,

    March 20, 2008 @ 11:02 pm

    you really can’t benchmark a software framework based on a dummy application… what about doing benchmarks for a real app that uses the many features of rails?

  9. Hongli said,

    March 20, 2008 @ 11:08 pm

    tangfucius: You can. The dummy application essentially benchmarks the foundation.

    Every Rails application consists of two parts, A (the Rails platform) and B (the application-specific code). Let the t(x) be the time that part x spends one request. Then the total time that the application spends on one request is t(A) + t(B). Suppose A1 is Rails 1.2.6 and A2 is Rails 2.0.2. If t(A2) < t(A1), then t(A2) + t(B) < t(A1) + t(B).

    Now, we observe that the dummy application has very little app-specific code. So t(B) ≈ 0. Then we can say: t(A) + t(B) ≈ t(A).

    I have mathematically proven that benchmarking the dummy application is a useful benchmark for the Rails framework. QED.

  10. RX-7乗りの適当な日々 said,

    March 24, 2008 @ 2:54 pm

    [IT] Rails 1.2.6 と 2.0.2 のパフォーマンス比較…

    先日のエントリ「RedmineでのRailsパフォーマンス比較(1.2.6 vs 2.0.2)」(d:id:rx7:20080221:p1)に続いて、Ruby on Railsのパフォーマンスに関する話題を1つ。 他のサイトでも、Rails2.0系と1.2系のパフォー…

  11. tangfucius said,

    March 24, 2008 @ 11:08 pm

    Hongli, that was definitely one of the more interesting responses I’ve read in a while… Your formula only works if part A and part B are separate, non-correlated entities… The “mathematically proven” formula used is based on assumptions that vary in validity when the application specific code changes…

    You should know that rails and the applications run on it are rather sophisticated, a A + B formula is really an insufficient way to claim any plausible performance benchmark results. It’s like saying mysql is faster than oracle just because it does select statements faster on myisam tables than regular oracle selects…

    I didn’t mean to take any validity out of your benchmark results, but was simply suggesting more ways to improve the benchmark tests, thus hoping for a more thorough, accurate benchmark.

    Good luck.

  12. Hongli said,

    March 24, 2008 @ 11:23 pm

    My response was a joke. ;) But point taken.

  13. Rails2.0と1.2のパフォーマンスの比較 | ひげろぐ said,

    January 8, 2011 @ 11:47 am

    […] 赖洪礼的 blog ≫ Performance comparison: Rails 1.2.6 vs 2.0.2 […]

RSS feed for comments on this post · TrackBack URI

Leave a Comment