In the past few days I’ve been working on mongrel_light_cluster, an extension for mongrel_cluster which automatically uses copy-on-write semantics to save memory in Ruby on Rails applications. The initial measurements were exciting – one can reduce memory usage by as much as 5 times when running 10 Mongrel instances!
Unfortunately I was hit by a wall. Ruby garbage collector marks all memory pages as dirty, causing all memory in forked processes to be copied, thereby destroying copy-on-write semantics. I was unable to reproduce this problem before, so I kept working on mongrel_light_cluster, but now I see how serious the problem really is.
So I give up. The project failed. This cannot be fixed outside the Ruby interpreter. Someone must fix the Ruby garbage collector to use a mark table instead of marking inside the object’s own memory. Dough Beaver has published a slide about this, but I’m not sure whether anyone’s actually working on fixing the garbage collector. Until the garbage collector has been fixed, we’ll have to tolerate Ruby on Rails wasting memory.