赖洪礼的 blog
This is my personal blog. For Phusion's corporate blog, please go to http://blog.phusion.nl

Since the initial release of the teaser screencast, a lot of people have asked about Passenger’s (a.k.a. mod_rails) performance compared to Mongrel and Thin. Here’s the benchmark that you’ve all been waiting for.

In this benchmark, we compare the following software:

• Phusion Passenger (mod_rails), latest development version
• Mongrel 1.1.4
• Thin 0.7.1

System specification:

• Intel Core 2 Duo, T5300 @ 1.73GHz
• 2 GB RAM
• Ubuntu Linux 7.10
• Apache 2.2.4 (Ubuntu package)
• Ruby 1.8.6 (Ubuntu package)

Test applications:

• Typo 5.0.3. This is an out-of-the-box setup with only a hello world post. Uses semi-static caching.
• Petstore revision 360, an application that the JRuby guys use for benchmarking. At first it wouldn’t start in Passenger because of a bug Petstore: they forgot to add require 'forwardable' in app/models/cart_items.rb. I added this line myself.
• Eldorado 0.9.1. This is an out-of-the-box setup with only a single forum and a single post added.

Typo and Eldorado both use SQLite 3. Petstore uses MySQL, because for some reason Petstore performs very badly when SQLite 3 is used.

People have also recommended Lovd By Less as test application, but I couldn’t get it working (database setup failed).

The applications, when served by Passenger, are setup at the following test domains:

• http://typo.test/
• http://petstore.test/
• http://eldorado.test/

Both the Mongrel cluster and the Thin cluster are setup at http://mongrel_cluster.test/. (Only either Mongrel or Thin is running during the benchmark, not both at the same time.)

And of course, we always use the production environment.

Backend configuration:

• Default Apache configuration, as provided by Ubuntu.
• Passenger: RailsMaxPoolSize is set to 10. This means that Passenger will never spawn more than 10 Rails application instances. Note that Passenger is compiled without optimizations, so if you install Passenger yourself and benchmark it, you might find that Passenger is slightly faster.
• Mongrel cluster: a cluster of 10 servers for each test application, behind mod_proxy_balancer, using default settings:

  ProxyPass / balancer://mongrel_cluster/
  ProxyPassReverse / balancer://mongrel_cluster/
  <Proxy balancer://mongrel_cluster>
  	BalancerMember http://127.0.0.1:3000
  	BalancerMember http://127.0.0.1:3001
  	BalancerMember http://127.0.0.1:3002
  	BalancerMember http://127.0.0.1:3003
  	BalancerMember http://127.0.0.1:3004
  	BalancerMember http://127.0.0.1:3005
  	BalancerMember http://127.0.0.1:3006
  	BalancerMember http://127.0.0.1:3007
  	BalancerMember http://127.0.0.1:3008
  	BalancerMember http://127.0.0.1:3009
  </Proxy>

• Thin cluster: same configuration as Mongrel cluster.

Benchmark: Mongrel clusters

First we benchmark Typo served by Mongrel clusters. Because the actual application code is loaded by Mongrel during the first request, we do not want to count the first request’s time. So we run the following command to “warm up” the Mongrels, and discard its result:

ab -n 1000 -c 100 http://mongrel_cluster.test/

Next, we test 10,000 requests with 100 concurrent users:

ab -n 10000 -c 100 http://mongrel_cluster.test/

Result:

Concurrency Level:      100
Time taken for tests:   32.628480 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      54890000 bytes
HTML transferred:       51480000 bytes
Requests per second:    306.48 [#/sec] (mean)
Time per request:       326.285 [ms] (mean)
Time per request:       3.263 [ms] (mean, across all concurrent requests)
Transfer rate:          1642.83 [Kbytes/sec] received

We repeat these steps Petstore and Eldorado.

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   109.786026 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      40750000 bytes
HTML transferred:       37680000 bytes
Requests per second:    91.09 [#/sec] (mean)
Time per request:       1097.860 [ms] (mean)
Time per request:       10.979 [ms] (mean, across all concurrent requests)
Transfer rate:          362.47 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   70.889388 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      41130000 bytes
HTML transferred:       35530000 bytes
Requests per second:    141.06 [#/sec] (mean)
Time per request:       708.894 [ms] (mean)
Time per request:       7.089 [ms] (mean, across all concurrent requests)
Transfer rate:          566.60 [Kbytes/sec] received

Benchmark: Thin clusters

The benchmarking steps for Thin are the same.

Typo:

Concurrency Level:      100
Time taken for tests:   25.795470 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      54910000 bytes
HTML transferred:       51480000 bytes
Requests per second:    387.66 [#/sec] (mean)
Time per request:       257.955 [ms] (mean)
Time per request:       2.580 [ms] (mean, across all concurrent requests)
Transfer rate:          2078.78 [Kbytes/sec] received

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   96.874308 seconds
Complete requests:      10000
Failed requests:        42
   (Connect: 0, Length: 42, Exceptions: 0)
Write errors:           0
Non-2xx responses:      42
Total transferred:      40630350 bytes
HTML transferred:       37547028 bytes
Requests per second:    103.23 [#/sec] (mean)
Time per request:       968.743 [ms] (mean)
Time per request:       9.687 [ms] (mean, across all concurrent requests)
Transfer rate:          409.58 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   71.311031 seconds
Complete requests:      10000
Failed requests:        5
   (Connect: 0, Length: 5, Exceptions: 0)
Write errors:           0
Non-2xx responses:      5
Total transferred:      41137070 bytes
HTML transferred:       35518568 bytes
Requests per second:    140.23 [#/sec] (mean)
Time per request:       713.110 [ms] (mean)
Time per request:       7.131 [ms] (mean, across all concurrent requests)
Transfer rate:          563.34 [Kbytes/sec] received

Benchmark: Passenger

The benchmarking steps for Passenger are the same.

Typo:

Concurrency Level:      100
Time taken for tests:   25.131980 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      55200000 bytes
HTML transferred:       51260000 bytes
Requests per second:    397.90 [#/sec] (mean)
Time per request:       251.320 [ms] (mean)
Time per request:       2.513 [ms] (mean, across all concurrent requests)
Transfer rate:          2144.92 [Kbytes/sec] received

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   104.428742 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      40300000 bytes
HTML transferred:       36700000 bytes
Requests per second:    95.76 [#/sec] (mean)
Time per request:       1044.287 [ms] (mean)
Time per request:       10.443 [ms] (mean, across all concurrent requests)
Transfer rate:          376.86 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   72.587018 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      41660000 bytes
HTML transferred:       35530000 bytes
Requests per second:    137.77 [#/sec] (mean)
Time per request:       725.870 [ms] (mean)
Time per request:       7.259 [ms] (mean, across all concurrent requests)
Transfer rate:          560.47 [Kbytes/sec] received

Summary

Here’s a nice summary of the requests per second, in a table and in a graph:

Conclusion

I was pleasently surprised by the results. During Passenger’s development, our performance baseline was Mongrel behind mod_proxy. According to earlier (simple and naive) tests, Passenger performed similar to Mongrel. I expected Thin to be a bit faster than Passenger. These new benchmarks however suggest Passenger is faster than Mongrel, and is on par with Thin.

But you know what they say: there are lies, damn lies, and statistics. Your hardware and software is different, so if you benchmark it yourself, you’re likely to get (slightly?) different results.

I have good news and bad news.

Let’s start with the good news. Passenger (mod_rails) is almost ready for release. The code is as good as done (actually, it has been as good as done for well over a week now). The website is almost done. We’re just awaiting final test results from remaining beta testers.

The bad news: lots of people have commented about our current screencast, in particular about the now-infamous “scary green hand”. Being the perfectionists that we are, we’d like to redo the screencast before we release Passenger. Neither Ninh nor I are native English speakers, so we’re looking for a native English speaker who can help us with the voice-over part of the screencast. Preferably, someone with enough ‘gravitas’ in their voice, or in the case of a female voice, the ‘kindness’ should come out dripping from out of our audio speakers.

Would you like to volunteer? If so, please provide us with an audio sample. You can do this by either replying in this post, or by emailing us at “hongli at plan99 dot net” or “ninh dot bui at gmail dot com”! Thanks in advance.

Does anyone know a good Ruby on Rails app that I can use for benchmarking Passenger (mod_rails)? I don’t have any apps that I’m allowed to use for this purpose. Mephisto and Typo are out because they both use page caching, which would make the benchmark pointless.

Traffic for our Passenger (mod_rails) preview site has increased three fold ever since RailsEnvy and weblog.rubyonrails.org (thanks David) linked to us. We’ve received quite some emails from interested people. One of the questions that we receive most is:

“What about other Ruby frameworks, like Merb, Camping, etc?”

There is the following saying: jack of all trades, master of none. At the moment, our goal is to make Passenger a great deployment system for Ruby on Rails, one that is matched by none in terms of ease of use and low maintenance overhead.

But some people ask:

“What’s wrong with mod_ruby?”

• For one, look at its website. Last release was from 2006. I’m not sure whether it’s still maintained anymore.
• Nobody (or almost nobody) uses it in production. This fact alone says a lot.
• The Rails wiki page for mod_ruby lists a pretty bad limitation:
  

  mod_ruby uses a shared interpreter per Apache process, which means that multiple Rails applications on the same Apache setup with mod_ruby would share the framework classes. This doesn’t work well with the Rails model of using class-level configuration attributes, so it’s considered unsafe to use mod_ruby and Rails with more than one application running per Apache setup, because different applications may start sharing the classes.

Granted, these are implementation problems and not conceptual problems, so they can be fixed. So some people ask:

“Why didn’t you just make a solid mod_ruby implementation?”

First, we must agree on the premise that Rails is a more popular framework than other Ruby frameworks. Wait, hold your flame thrower, this is not the reason.

Next, try thinking from the point of view of a new Rails developer. He’s introduced to lala-land: Don’t-Repeat-Yourself, convention over configuration, stuff that’s supposed to Just Work(tm).

Then he deploys his Rails app and finds out that he has to setup a Mongrel cluster (1 command but usually with 3 or 4 options that he’ll have to look up), configure Apache to proxy to the cluster (about 5 or 6 lines that he’ll have to setup), configure init.d/daemon tools/monit/god/whatever so that his Mongrel cluster is started at system startup, etc.

Oh, and you need to repeat this for every single Rails app.

Ouch, is this the proper Rails experience? I don’t think so. This is Passenger’s philosophy: deployment should be in line with the Rails philosophy. In other words, it should be brain-dead simple. Just imagine the money that you’ll save on health care for your system administrator if he doesn’t have to worry about all that stuff anymore.

Of course, the Rails philosophy doesn’t prevent deployment of other Ruby framework apps from being easy. So what’s holding us back? Well, here’s a few things that Passenger does but mod_ruby doesn’t:

• Remember the screencast that we showed you? You can deploy a Rails app to a virtual host without any Ruby-specific or Rails-specific configuration! From this fact one can deduce that Passenger somehow automatically detects Rails applications (and yes, this autodetection can be disabled). Autodetection heuristics are framework-dependent. We’ll need different heuristics for each framework.
• Passenger preloads framework code and application code in order to reduce startup time and to reduce memory usage. Preloading logic is framework-dependent.
• Look at Troubleshooting obscure errors with mod_ruby. Notice the word “obscure errors” - errors should never be obscure or hard to track!

  We’ve seen many people failing to deploy Rails apps because of errors during application startup. Lighttpd’s FastCGI implementation redirects those errors to /dev/null. Not good, this fact bit me 2 years ago — when I was deploying my first Rails app — and made me wonder why the thing wouldn’t work. Mongrel prints errors either to stderr or to its log file. But people who are new to Rails deployment don’t know that they should be looking in the log files (or that there are log files in the first place). Plus, looking into the log file is yet-another-step. Wouldn’t it be great if you can see startup errors right in the browser?

  Passenger provides user-friendly error pages. If a Rails app failed to start, then Passenger will tell you what happened. A few possible error pages are:

  • Whether a (specific version of a) system-wide Rails framework failed to load. Passenger will even tell you the correct command to (re)install it.
  • Whether a vendor’ed Rails framework failed to load.
  • Whether the app failed to start because of a syntax error, or a missing gem. Passenger will suggest solutions.
  • Whether a database error occured during startup. Passenger will tell you to check your database.yml and to run ‘rake db:migrate’.

  Here are previews of typical error pages:
  

  

  As you can see, a lot of these error pages are framework-dependent.

As you may have noticed by now, we’ve put a lot of work into making sure that deployment is dead-easy. This is only possible if Passenger has knowledge about the frameworks themselves. We have a tight schedule and we only have 2 developers working on Passenger. Rails is more popular so it has more priority. We’re working around the clock to get this thing released.

Or we can get rid of the autodetection/preloading logic and just provide generic configuration options. But would you really want that? By no means should we force the system administrator to configure things. By default, stuff should Just Work(tm) out-of-the-box.

Hope this clears things up.

Thought I’d give you guys a little update on where we currently stand (yeah, we’re still alive ;)). As you may have already know, we’ve announced Phusion Passenger (a.k.a. mod_rails) last week and the reaction has been overwhelmingly positive!

Our beta testers now not only include several of the Ruby on Rails core team, but also some of the largest Ruby on Rails webhosts you may know of. Needless to say, we’re very thrilled about this, and in particular, we’d like to thank Pratik Naik, David Heinemeier Hansson and Derek Perez for their insightful feedback and support. Also, we’d like to thank community members such as Russel Norris for keeping the conversations fun. With this kind of support, we’re convinced of the quality of Passenger, since a lot of love has gone into making this.

Again, we’d like to express our gratitude to all for supporting us on this. It’s important to note that for an open source project, it’s things like these that keep developers like us going on.

And while we’re on the subject of ‘going on’: we’ve been working around the clock the past few days to get things done as fast as we can. While Passenger is nearing completion and its website still needs a bit of work, it still remains uncertain if we’re able to obtain a preview/announcement slot at Railsconf . We’d like to let you guys know however, that we’re putting a lot of effort into making this happen, since we really believe that Passenger has the potential to set of a revolution in the Rubyverse by possibly allowing more people to enjoy Ruby on Rails.

For now we ask you for your patience and we’ll try to keep you guys posted on a regular basis regarding our activity.

With kind regards we are,

Hongli Lai
Ninh Bui

- Phusion.nl

PStore is a library in Ruby that “implements a file based persistance mechanism based on a Hash”. Ruby on Rails up until version 1.2 uses PStore as the default mechanism for storing sessions. But it’s pretty well-known that PStore “sucks”: people say that it’s slow, causes file corruptions, etc. This is one of the reasons why Rails 2.0 uses the cookie session store by default: it’s faster and needs less maintenance at the cost of a few (easy-to-avoid) caveats.

If you look at the Mongrel FAQ, then you’ll see that 3 items are devoted to telling you that PStore is the work of satan and that any sane web developer should ritually burn it. But this particular sentence caught my eye:

“Other things that can cause big pauses are:
- …
- Locking files wrong. Multiple processes locking files is a delicate thing to do. “

Locking files wrong? How can that possibly be? File locking, just like mutex locking, isn’t really rocket science.

Note that PStore’s RDoc advertises itself as transactional:

“# The transactional behavior ensures that any changes succeed or fail together.
# This can be used to ensure that the data store is not left in a transitory
# state, where some values were upated but others were not.”

I decided to take a look at its source code. What possibly could have gone wrong?

Well, I’m not sure what is wrong with it, but the relevant code, PStore#transaction, is a bit messy. I had a hard figuring out what it is exactly doing and why, but I figured that it does these things:

• It locks the file.
• It reads and unmarshals the file.
• It generates an MD5 checksum of the file.
• It runs the transaction block, then generates an MD5 checksum of the new contents of the file. The file is only written to if the MD5 checksum or the size of the new content doesn’t match that of the original file.
• It writes new data to a temp file, then renames that to the original file (or at least, I think that’s what it does; it seems to use 2 temp files files). File renames are atomic on Unix but not Windows. This is why on Windows, one needs to implement file recovery as well. PStore seems to have some code for recovery, but it’s unclear how well that works. It doesn’t match the algorithm given on MSDN.

It’s not clear whether PStore has locked everything correctly. Oh, and PStore isn’t thread-safe (though it is reentrant).

• PStore is usually used for writing small amounts of data, so calculating an MD5 is definitely not worth it - just write the file already!
• Writing to a temp file ensures atomicity on Unix, but it adds another system call, and system calls are expensive. If one is able to open the file for writing-and-appending, then writing to it shouldn’t raise any errors except in rare conditions, such as out-of-disk-space conditions or hardware errors. Furthermore, I’ve never seen anybody using PStore for anything other than for session data and other not-so-important stuff, so performance is more important.
• Aah, Windows…. All the recovery code just to work around the fact that Windows doesn’t support atomic file renames, cause a lot of performance loss.

So I rewrote PStore. The code is now easier to read and is faster. And as far as I know, everything is locked correctly so there shouldn’t be any concurrent issues.

By default, it doesn’t try as hard to ensure file integrity because harddisk I/O errors are very rare, but if file integrity is really an issue, then you can set pstore.ultra_safe_transactions = true to enable it. This option only has effect on Unix though: as of now, I haven’t bothered writing complex recovery code for Windows.

This is the benchmark program that I used:
http://pastie.org/170997

The results are as follows:

As you can see, PStore has become 2 times of even 3 times faster, depending on whether is ultra_safe_transactions is enabled! But let’s see what kind of effect it has on a dummy Rails 2.0 app that uses the PStore session store:

• Dummy Rails 2.0 app, using original PStore library: 170.22 requests/sec
• Dummy Rails 2.0 app, using my PStore library: 196.72 requests/sec

A small performance increase. (Though the cookie session store is still faster: 221.74 requests/sec.)

You can download it here:
http://pastebin.com/f163702f2

If you want your Rails app to make use of it, simply save it as “lib/pstore.rb”. For maximum stability, Rails’s PStore handler should be modified to ignore any errors encountered during unmarshalling.

One of these days I’ll send a patch to ruby-core so that this can be merged back upstream. But for now Passenger (a.k.a. mod_rails) has priority.

A video says more than a thousand words, so just click on the video link below.

http://www.modrails.com/

Is this for real?

Yes people, this is real! We at Phusion have been developing this for a while now, and it’s almost done. Currently we’re in private beta testing phase. Our beta testers include several Ruby on Rails core developers, and so far the feedback from our testers is overwhelmingly positive.

Stay tuned for more news!

I benchmarked Rails 1.2 vs 2.0 in my last blog post, and Rails 2.0 turned out to be 30%-50% faster. Then Eric Allam said:

“Hmm, another benchmark that completely ignores memory use and memory usage growth.”

Okay, let’s give it a try.

Wait, hold your horses!

When measuring memory, people usually measure the wrong thing. There are 3 things that one can measure:

1. Virtual memory size

All modern operating systems - probably including yours - use virtual memory. The memory that an application sees is not the same as your physical memory. Even if you only have 256 MB RAM installed, all applications on your system still see 4 GB RAM, and can use more than 256 MB RAM (on x86, that is). See the Wikipedia entry.

Even if you only have 256 MB RAM, the virtual memory size reported by a process can still be 4 GB! This is because processes can map files, hardware and other resources into its memory address space, without really loading them into physical memory. This is complicated by the fact that processes can share memory with each other. The VM size is quite meaningless when measuring real memory usage.

2. Resident Set Size (RSS)

This is the amount of the process’s memory that’s currently in physical memory. At first, this seems to be a good candidate for measuring real memory usage. But parts of the process may be swapped to the hard disk. The RSS also seems to include things like shared memory, which we clearly don’t want to measure. It’s unclear whether the RSS also contains kernel buffers and caches, which we also don’t want to measure.

3. Private dirty memory

This is the best measurement. It does not count shared memory. This blog post explains it all. The private dirty memory is easy to measure on Linux (I’m using Ubuntu 7.10), but I’m not sure how to do it on other operating systems.

4. Ruby heap information

3 is usually the best measurement - but not for Ruby! In the past I’ve explained how Ruby’s heap is implemented. Ruby allocates several heaps, and the heaps contain equally-sized slots in which Ruby objects are stored. If we measure the private dirty memory (that is, assuming fork() is never called), we’ll be effectively measuring the total heap size, including any free heap slots. Free heap slots are reused by Ruby, so they really shouldn’t be counted.

We want to count the number of used heap slots. Usually, there’s no way to do this. But I maintain a private Ruby branch (which will be released soon) with statistics information. We’ll use that for memory measurement. (My GC patch also provides statistics information, though the version I’m working on provides more accurate statistics.)

People interested in details should read a good book about operating systems. I recommend “Operating System Concepts” by Silberschatz et al. My “Operating Systems” classes professor has Dutch slides on the subject.

The measurement

After a cold start with Mongrel 1.1.3, the private dirty memories were as follows:

• Rails 1.2.6: 25.5 MB
• Rails 2.0.2: 19.7 MB

Nice! A 6 MB memory reduction after a cold boot!

Though, as I’ve said earlier, these numbers don’t really mean much. The application code (i.e. controller and model code) haven’t been loaded into memory yet. And Ruby doesn’t garbage collect until its heap is full. So let’s find out what happens after 3000 requests and a garbage collection run. I added these actions to PostsController:

def gc
    headers["Content-Type"] = "text/plain"
    GC.start
    render :text => ObjectSpace.statistics
end

And this route:

map.connect '/gc', :controller => "posts", :action => 'gc'

Then I ran, for both apps:

ab -n 3000 http://localhost:3000/
links http://localhost:3000/gc

The memory usages were:

Rails 1.2.6

Private dirty memory:28.1 MB
Total heap size: 14,702 KB
Free heap space: 7,789 KB

Rails 2.0.2

Private dirty memory: 20.9 MB
Total heap size: 8,059 KB
Free heap space: 2,098 KB

Nice! I didn’t expect this, but apparently Rails 2.0 uses a lot less memory!

[EDIT: Some people have contributed a chart. Thanks.]

There’s still a lot of misinformation about Rails on the Internet. People are still claiming that Rails doesn’t scale, is too slow, etc. Rails 2.0 has been released for a while now, and a year ago someone blished a benchmark, in which it is claimed that Rails edge (what is now 2.0) is slower than 1.2. I’ve decided to benchmark Rails 1.2 and 2.0.

I’ve written two dummy applications, one for 1.2 and one for 2.0. The reason why I’m not just running the same app under 2.0 is because Rails 2.0 introduced a new cookie session store, among other things. It is claimed that this session store is much faster than the old default PStore cookie store.

So first I create a Rails 2.0 application:

mkdir bench
rails 2.0
cd 2.0
./script/generate scaffold Post title:string content:text created_on:timestamp updated_on:timestamp
rake db:migrate RAILS_ENV=production

Note that scaffolding in Rails 2.0 is no longer dynamic. The scaffold generator now generates a database schema and some static HTML and code. This is equivalent to Rails 1.2’s scaffold_resource.

I also modified routes.rb so that the root URL is mapped to the ‘posts’ controller. And of course, I delete ‘public/index.html’.

Next, I create a Rails 1.2 application:

cd ..
rails _1.2.6_ 1.2            # This is a nice RubyGems trick. It allows me to specify which Rails version I want to use.
cd 1.2
./script/generate scaffold_resource Post title:string content:text created_on:timestamp updated_on:timestamp
cp ../2.0/config/database.yml config/    # We want to use SQLite, just like Rails 2.0.
rake db:migrate RAILS_ENV=production

I also modified routes.rb so that the root URL is mapped to the ‘posts’ controller, and deleted ‘public/index.html’.

I ran both applications in Mongrel (with the ‘production’ environment). The results of ‘ab -n 5000 http://localhost:3000/’ are as follows:

• Rails 1.2.6: 141.19 requests/sec
• Rails 2.0.2: 214.76 requests/sec

Wow, what a difference! Rails 2.0 is 50% faster in a dummy application!

In Rails 1.2, a lot of time is spent in the session store. Let’s see what happens if we specify “session :off” in both applications:

• Rails 1.2.6: 189.51 requests/sec
• Rails 2.0.2: 246.69 requests/sec

Wow! Even with sessions off, Rails 2.0 is still 30% faster! So the cookie session store isn’t the only thing responsible for the performance improvement!

Edge Rails has even more performance improvements. Things are definitely going the right direction.