Saving memory in Ruby on Rails with fork() – failed

In the past few days I’ve been working on mongrel_light_cluster, an extension for mongrel_cluster which automatically uses copy-on-write semantics to save memory in Ruby on Rails applications. The initial measurements were exciting – one can reduce memory usage by as much as 5 times when running 10 Mongrel instances!

Unfortunately I was hit by a wall. Ruby garbage collector marks all memory pages as dirty, causing all memory in forked processes to be copied, thereby destroying copy-on-write semantics. I was unable to reproduce this problem before, so I kept working on mongrel_light_cluster, but now I see how serious the problem really is.

So I give up. The project failed. This cannot be fixed outside the Ruby interpreter. Someone must fix the Ruby garbage collector to use a mark table instead of marking inside the object’s own memory. Dough Beaver has published a slide about this, but I’m not sure whether anyone’s actually working on fixing the garbage collector. Until the garbage collector has been fixed, we’ll have to tolerate Ruby on Rails wasting memory. :(

3 Comments »

  1. Pratik said,

    July 23, 2007 @ 3:40 pm

    Well, that’s sad. But well, if it were possible, it would have already been done :-)

    You might wanna get in touch with rubinius guys to see how they’re gonna implement GC. May be if you put this point, they can work in that direction. Would kick ass.

  2. Hongli said,

    July 23, 2007 @ 3:52 pm

    I’m sure that it’s, just nobody has done it yet.

    I spent this afternoon trying to implement it myself. But now Ruby segfaults on a NULL pointer, and I don’t know how to debug this. :(

  3. 赖洪礼的 blog » Making Ruby’s garbage collector copy-on-write friendly said,

    July 25, 2007 @ 1:22 pm

    [...] that I said that I give up? Well, I lied. Sleep has to be the best thing ever invented. After a night of sleep and some [...]

RSS feed for comments on this post · TrackBack URI

Leave a Comment