赖洪礼的 blog
This is my personal blog. For Phusion's corporate blog, please go to http://blog.phusion.nl

In my previous blog entries (here and here), I blogged about using fork() and copy-on-write semantics to reduce memory usage in Ruby on Rails.

I was wondering whether I’ve preloaded enough Ruby libraries. If I didn’t, then each child process will load its own copy of the required libraries, and that memory will not be shared between the child processes. Unfortunately, Ruby doesn’t seem to have a variable somewhere which lists all loaded Ruby files in the interpreter. So I hacked Ruby, by inserting the following code in ruby.c, function load_file(fname, script):

static void
load_file(fname, script)
    const char *fname;
    int script;
{
    extern VALUE rb_stdin;
    VALUE f;
    int line_start = 1;
    ...
    }
fprintf(stderr, “RUBY-LOAD: %s\n”, fname);
    if (script) {

This will make the Ruby interpreter print all loaded files to stderr.

1. I made a special version of my prefork script, which exits immediately after preloading the Rails libraries. I redirected its stderr to a file, which allows me see which files are loaded by the preloading procedure.
2. Next, I ran the normal version of my prefork script. Once again, stderr is redirected to a file. I terminate the script after the child processes have been fully initialized.

I compared the difference between the two files. I found out that only dispatch.fcgi had to be loaded by the child processes! Great.

I then launched a web browser and visited a page in my application. I compared the files again. This time I found out that each child process had to individually load the source code for the Rails application itself.

Conclusion

The prefork script doesn’t preload the source files for the Rails application itself. It should - this will save us a few hundred kilobytes to a few megabytes of memory, depending on the size of the application.

Based on this finding, as well as the results of my initial research on memory usage in Rails, I can conclude that most of the memory in a Rails application is spent on storing the Ruby on Rails (and dependencies) code, and that only a small fraction is spent on storing application code.

In other news, I cleaned up and restructured the prefork script. It can now gracefully terminate child processes whenever it receives a termination signal. It also performs more error checking. I’m going to do more testing. If this technique turns out to be reliable I’ll publish the final version of the script to the public.