@rdonly = read_only

Then a context switch occurs. Thread 2 enters a read-write transaction and passes the same line as well, setting @rdonly to false. Thread 2 exits transaction. Context switch occurs. Thread 1, which was supposed to be executing a read-only transaction, now thinks that the transaction is read-write and writes stuff to disk. Boom.

For pstore physical writes your version is 2x faster for me:

user system total real
std 20.240000 36.490000 56.730000 ( 58.234056)
new_canonical_marshal 13.090000 12.830000 25.920000 ( 26.656768)

That’s using the fast store strategy and read/writing the same file 50k times.

For logical writes where the data hasn’t changed, the new version is
roughly 15-20% slower:

user system total real
std 6.500000 3.160000 9.660000 ( 9.841138)
new_canonical_marshal 8.640000 3.300000 11.940000 ( 12.119400)

I did some quick profiling and I don’t see anything obvious. I think it’s the
overhead of the added method calls. I inlined most of them and got pretty
close to the standard version.

new_mla 6.810000 3.110000 9.920000 ( 10.008198)

OTOH, even with the slight slow-down we’re still talking 4k+ ops/sec
which is unlikely to be a bottleneck.

http://pastebin.com/m12689e12
Patch is against 1.8.6.

(If you don’t want to apply the patch, then you can simulate the ‘canonical’ behavior by calling rehash() on all your Hash elements, including the @table instance variable in PStore.)

By the way, what benchmark script did you use to generate the results seen in your pstore.png graph?

Here is the patch to add partitioning:
http://mla.homeunix.com/tmp/pstore_hashed_directories.patch.txt

I think this is the script I used to test it. See the database_manager param which
I toggled for successive runs:

http://mla.homeunix.com/tmp/pstore-bench.rb

That was one of my first forays into Ruby so be gentle ;/

I think you should take another look at the md5 logic. Under Rails, the session is always being opened for R/W, correct?
Which means the session will always be written even if it hasn’t changed. But in most environments, the read/write ratio is often 10:1 or more.

In a quick benchmark of your new version, it appears to be 2x as slow when the session has not changed, suggesting that the write is still substantially more expensive than the md5 check.

As for Marshal: it’s implemented in C. Changing it would require a bit of work as well as patching the interpreter.

This is true for listing all files in a directory, but are you sure it’s also true for reading the contents of a single of which the filename is already known?

I believe so. Take a look at this graph that shows the performance gain by partitioning the sessions
into directories (this is on ext3 filesystem):

http://mla.homeunix.com/tmp/pstore.png

I’ll look at the marshal stuff later, but here’s a example of how it can appear that the contents has changed when it hasn’t:

http://mla.homeunix.com/tmp/marshal-test.rb
http://mla.homeunix.com/tmp/marshal-test.txt

I’m very much interested in gettting an improved version of the file session store into rails. Esp. when using a load balancer that has session affinity, this is a very scalable solution if you can’t or don’t want to use the cookie store approach.