Memory usage comparison: Rails 1.2.6 vs 2.0.2

Featured post

passengerscreencastpreview-small.jpg

I benchmarked Rails 1.2 vs 2.0 in my last blog post, and Rails 2.0 turned out to be 30%-50% faster. Then Eric Allam said:

“Hmm, another benchmark that completely ignores memory use and memory usage growth.”

Okay, let’s give it a try.

Wait, hold your horses!

When measuring memory, people usually measure the wrong thing. There are 3 things that one can measure:

1. Virtual memory size
All modern operating systems – probably including yours – use virtual memory. The memory that an application sees is not the same as your physical memory. Even if you only have 256 MB RAM installed, all applications on your system still see 4 GB RAM, and can use more than 256 MB RAM (on x86, that is). See the Wikipedia entry.

Even if you only have 256 MB RAM, the virtual memory size reported by a process can still be 4 GB! This is because processes can map files, hardware and other resources into its memory address space, without really loading them into physical memory. This is complicated by the fact that processes can share memory with each other. The VM size is quite meaningless when measuring real memory usage.

2. Resident Set Size (RSS)
This is the amount of the process’s memory that’s currently in physical memory. At first, this seems to be a good candidate for measuring real memory usage. But parts of the process may be swapped to the hard disk. The RSS also seems to include things like shared memory, which we clearly don’t want to measure. It’s unclear whether the RSS also contains kernel buffers and caches, which we also don’t want to measure.
3. Private dirty memory
This is the best measurement. It does not count shared memory. This blog post explains it all. The private dirty memory is easy to measure on Linux (I’m using Ubuntu 7.10), but I’m not sure how to do it on other operating systems.
4. Ruby heap information
3 is usually the best measurement – but not for Ruby! In the past I’ve explained how Ruby’s heap is implemented. Ruby allocates several heaps, and the heaps contain equally-sized slots in which Ruby objects are stored. If we measure the private dirty memory (that is, assuming fork() is never called), we’ll be effectively measuring the total heap size, including any free heap slots. Free heap slots are reused by Ruby, so they really shouldn’t be counted.

We want to count the number of used heap slots. Usually, there’s no way to do this. But I maintain a private Ruby branch (which will be released soon) with statistics information. We’ll use that for memory measurement. (My GC patch also provides statistics information, though the version I’m working on provides more accurate statistics.)

People interested in details should read a good book about operating systems. I recommend “Operating System Concepts” by Silberschatz et al. My “Operating Systems” classes professor has Dutch slides on the subject.

The measurement

After a cold start with Mongrel 1.1.3, the private dirty memories were as follows:

  • Rails 1.2.6: 25.5 MB
  • Rails 2.0.2: 19.7 MB

Nice! A 6 MB memory reduction after a cold boot!

Though, as I’ve said earlier, these numbers don’t really mean much. The application code (i.e. controller and model code) haven’t been loaded into memory yet. And Ruby doesn’t garbage collect until its heap is full. So let’s find out what happens after 3000 requests and a garbage collection run. I added these actions to PostsController:

def gc
    headers["Content-Type"] = "text/plain"
    GC.start
    render :text => ObjectSpace.statistics
end

And this route:

map.connect '/gc', :controller => "posts", :action => 'gc'

Then I ran, for both apps:

ab -n 3000 http://localhost:3000/
links http://localhost:3000/gc

The memory usages were:

Rails 1.2.6
Private dirty memory:28.1 MB
Total heap size: 14,702 KB
Free heap space: 7,789 KB
Rails 2.0.2
Private dirty memory: 20.9 MB
Total heap size: 8,059 KB
Free heap space: 2,098 KB

Nice! I didn’t expect this, but apparently Rails 2.0 uses a lot less memory!

[EDIT: Some people have contributed a chart. Thanks.]
chart.png

12 Comments »

  1. Eric Allam said,

    March 19, 2008 @ 6:14 pm

    Awesome! Thanks for taking my “suggestion” and making a kickass post out of it.

  2. Gubatron said,

    March 19, 2008 @ 9:53 pm

    It’d be cool if you could just put graphics for those of us in a hurry, I guess I’ll come back later.

  3. Colin Curtin said,

    March 20, 2008 @ 12:57 am

    http://chart.apis.google.com/chart?cht=bvg&chs=300×200&chd=t:28.1|20.9|14.7|8.1|7.8|2.1&chco=4d89f9,c6d9fd&chdl=Rails%201.2.6|Rails%202.0

  4. Colin Curtin said,

    March 20, 2008 @ 12:58 am

    Sorry for spamming, there’s the google chart link for what you want as a graph. (I couldn’t make an img tag)

  5. roScripts - Webmaster resources and websites said,

    March 20, 2008 @ 5:15 am

    Memory usage comparison: Rails 1.2.6 vs 2.0.2…

    Memory usage comparison: Rails 1.2.6 vs 2.0.2…

  6. Pete Cockerell said,

    March 20, 2008 @ 5:29 am

    Somehow your “x” in the size got converted to a “times” character. Try:

    http://chart.apis.google.com/chart?cht=bvg&chs=300×200&chd=t:28.1|20.9|14.7|8.1|7.8|2.1&chco=4d89f9,c6d9fd&chdl=Rails%201.2.6|Rails%202.0

  7. Pete Cockerell said,

    March 20, 2008 @ 5:34 am

    Hmmp! Happened to me too. Oh well. After you paste the (complete) URL into the URL field on your browser, replace the “×” in 300×200 with a proper lower case x character. (Or hit Enter to see it converted to %D7 by the browser [firefox, anyway] and replace that with a lower-case x).

  8. Crónica de una vida - Rendimiento de Rails 2.0 y el futuro Rails 2.1 said,

    March 20, 2008 @ 12:45 pm

    [...] comparativa demostrando que Rails 2.0 es entre un 30 y 50% más rápido que Rails 1.2.X. Además otro estudio muestra que consume menos memoria que la versión [...]

  9. RX-7乗りの適当な日々 said,

    March 24, 2008 @ 2:54 pm

    [IT] Rails 1.2.6 と 2.0.2 のパフォーマンス比較…

    先日のエントリ「RedmineでのRailsパフォーマンス比較(1.2.6 vs 2.0.2)」(d:id:rx7:20080221:p1)に続いて、Ruby on Railsのパフォーマンスに関する話題を1つ。 他のサイトでも、Rails2.0系と1.2系のパフォー…

  10. cris said,

    July 25, 2008 @ 5:15 pm

    Hi.

    Can you explain, how do you measure dirty memory on Linux? I have googled and haven’t found anything about dirty memory in your sence of term.

  11. Hongli said,

    July 25, 2008 @ 6:23 pm

    I usually just use GNOME System Monitor. It displays that information under the “Writable Memory” column. You can also do it yourself by inspecting /proc/(pid)/smaps: sum up all the Private_Dirty fields. Finally, there’s a Perl module called Linux::Smaps for querying this information.

  12. cris said,

    July 28, 2008 @ 3:47 pm

    Thanks, GSM works for me.

RSS feed for comments on this post · TrackBack URI

Leave a Comment