赖洪礼的 blog
This is my personal blog. For Phusion's corporate blog, please go to http://blog.phusion.nl

Passenger 1.0.5 has just been released. Please read the Phusion corporate blog for the announcement and upgrade instructions.

AkitaOnRails had recently interviewed us. The interview covers our company’s background and information about Passenger, among other things. He asked us quite some interesting questions, so we recommend you to read the interview.

Luke Kanies wrote an article, in which he says that Ruby has a distribution problem. It criticizes Rails applications for vendoring a lot of stuff, and criticizes RubyGems for not being able to handle native package dependencies, among other things.

I beg to disagree. The described problem is not a pure Ruby problem: it’s a general software distribution problem. I’m concerned that people would use this as another “reason” to oppose Ruby, even though it’s not specific to Ruby.

My response

I posted a reply to Luke’s article, and it was as follows:

Luke, I don’t really understand what you’re expecting from the Ruby/Rails community. You’re dealing with cross-platform distribution issues. That’s hard by its very nature. I was a developer in the Autopackage project (www.autopackage.org), a software packaging system that works across multiple Linux distributions. I’m currently a developer of Passenger (www.modrails.com). We’ve run into many of the issues that you mention here.

If I understand it correctly, you’re claiming that vendoring stuff is bad because:

• You cannot vendor everything (glibc, web browser, etc).
• As someone else has mentioned, vendoring stuff creates potential security issues. This is not unlike static linking in C/C++ projects.

On the other hand, vendoring stuff does have benefits:

• Guaranteed compatibility. Suppose you rely on GTK 2.2. On a faithful day, GTK 2.2.5 was released, but accidentally introduced a regression, and now your application fails left and right. Uh oh. If you vendored GTK then that wouldn’t have happened. This is not a theoretical possibility: GTK 2.2.15 or something actually broke AbiWord. (I’m not saying that vendoring GTK is a good idea, as GTK is quite large. I’m just pointing out a benefit of vendoring.)
• Less installation hassle. Not all platforms have good packaging systems. As far as I know, Debian-based distros are the only ones. RedHat-based YUM repositories tend to be quite small compared to Debian’s APT repositories. MacOS X and Windows don’t have native package management systems at all. If your app is cross-platform, then vendoring stuff is a lot easier for both the developer and the end user.

Vendoring stuff is not only common in the Java world, but also in the Windows and MacOS X world. Windows apps tend to bundle all their dependencies (with the exception of obvious stuff, such as Internet Explorer). How many games have you seen that bundle DirectX? Actually I’d say that not vendoring stuff is only common in Linux, and languages that have strong ties to Linux, such as Perl. Here’s where Debian’s package management system and huge package repositories really shine.

There’s almost no common ground in the world of package management. We at Autopackage had to invent our own dependency resolution mechanism (similar to APT) because there’s no lowest common denominator, even amongst Linux distros. RubyGems is probably created for the same reason: not all Ruby-supported platforms have (decent) package management, so they just wrote their own. Autopackage experimented with native package management integration (i.e. being able to use the system’s native package manager to resolve dependencies) but that proved to be much, much harder than initially expected, and up until today that feature still isn’t finished. So I don’t think you can reasonably expect the Ruby community to do something about this. It’s not a pure Ruby problem: it’s a general software distribution problem.

As for being FHS-compliant: I can only say “don’t bother” if your application is cross-platform. It seems that only hardcore Linux users care about that. Outside the Linux world, FHS is being criticized by pretty much everyone. Windows and OS X users complain that application files in Linux are scattered everywhere, instead of being self-contained. Scattering files isn’t a problem in Linux because of package management, but it is a problem in all other platforms that don’t have decent package management. I’ve found that being FHS-compliant is more trouble than it’s worth.

So how do you solve this problem? I don’t think it’s possible to come up with a general silver bullet solution. So that leaves the following choices to you, the developer:

• Create a native package for every platform that you support, i.e. .deb for Ubuntu/Debian, .rpm for RedHat, another .rpm for Mandriva and SuSE because their package names are different, a .exe for Windows, .dmg for MacOS X, .tgz for Slackware, .??? for Solaris, etc.
• Write a cross-platform installer. Passenger/mod_rails chose this option because it’s a lot easier than the first one. Passenger depends on Ruby packages (Rails, fastthread, Rake, etc.) as well as native packages (GCC, Apache, APR). It checks whether all dependencies are available, and if not, it tells the user how to install those dependencies. We’ve put platform autodetection and Linux distro autodetection code in the installer. So on Debian/Ubuntu it would tell you to run “apt-get install apache2-prefork-dev” while on Fedora/RHEL/CentOS it would tell you to run “yum install apache-devel”. We’ve found that this approach works extremely well.

Finally, we at Autopackage fully recognized the pros and cons and vendoring/static linking. Autopackage recommends the following: dynamically depend on stuff that are common, but vendor/static link stuff that are uncommon. We believe that this is a good trade off between the pros and cons. Passenger follows this recommendation as well: we vendor the Boost C++ library. Few people have Boost installed, and when they have it installed it often isn’t the version that Passenger requires. Installing Boost is a huge, huge pain on MacOS X. In this case, the benefits that vendoring Boost gives us outweight the cons by far.

On the other hand, Apache is fairly common, and easy to install on most platforms. Rake, fastthread, etc. are also easy to install because of RubyGems. That’s why we dynamically depend on those things instead of vendoring them.

So it all boils down to making the right choices and correctly balancing the pros and cons of vendoring. There’s no silver bullet.

Gunnar Wolf’s follow-up

Luke’s article was quickly followed by a blog post by Debian developer Gunnar Wolf:

By using Ruby Gems, you dramatically increase entropy and harm your systems’ security.

To this, I say nonsense. It’s pretty well-known that Debian, and Linux distros in general, dislike foreign packaging systems, regardless of their merits. I see quite a lot of conversations on #rubyonrails @ irc.freenode.net that are somewhat like this:

Person A: hi, I’m using Debian/Ubuntu/some-other-Debian-derived-distro. I typed “gem install rails”. But when I type “rails foo”, it says “command not found”. what’s going on?
Person B: type “gem update –system && gem install rails”
Person A: wow, it worked! thank you!

It is painfully obvious that Debian did something to RubyGems. I was bitten by this very issue as well. Debian’s RubyGems package places binaries in /var/gems (or something like that, I don’t remember the exact location) instead of /usr/bin. Fine, I understand that they don’t want foreign packages to pollute /usr/bin, which is a managed directly. But the least they can do is adding /var/gems to $PATH by default, something which they didn’t. As a result, many people who installed Rails via Debian thought that Rails is broken, when it’s actually Debian who crippled RubyGems.

“Increase entropy”? I don’t even know what this is supposed to mean in the context of Ruby software distribution.
“Harm your systems’ security”? As I’ve already stated, vendoring has both pros and cons. If one dynamically depends on a library, then it means that the system administrator and library author are responsible for security updates, but it also means that a security update can potentially break the application. By vendoring stuff, the responsibility of security updates is mostly shifted to the application developer. This is a trade-off: there’s nothing wrong with it, and whether it’s the best thing to do depends on the situation. Gunnar however has an extremely purist view, along the lines of “if you don’t agree with us then you’re an idiot, regardless of the circumstances”.

It seems more like that Gunnar’s words are carefully picked, with the goal of creating knee-jerk reactions. Debian said pretty much the same thing about Autopackage in the past, and now they’re doing it again with RubyGems.

Luke, Gunnar, will you realize that all you’re doing is ranting about a problem, without offering any solutions? And no, creating native Debian packages is not a solution, the world isn’t comprised of just Debian.

Passenger 1.0.2 fixed MacOS X compatibility. Unfortunately, the OS X fix broke compatibility with 64-bit Linux. The 1.0.4 release was supposed to fix it for both platforms. Before we released 1.0.4, we had an OS X user test it, and the test result was positive. But soon after the release we got bug reports from people, telling us that 1.0.4 broke OS X compatibility.

The main problem lies in the file descriptor passing code. We’re struggling to get it right:
1. The original code was copied from Ruby. Apparently that didn’t work well on 64-bit MacOS X. This also means that file descriptor passing on 64-bit MacOS X Ruby is also broken, not just Passenger.
2. There is generally very little documentation about file descriptor passing. We found a bunch of code samples, but it turns out that all of the ones that we’ve tried have portability problems. The developer man pages don’t provide a lot of information on file descriptor passing.

We want to get this right, but we don’t have any 64-bit machines at all. So we’re looking for people who can help us with testing Passenger on 64-bit OS X and 64-bit Linux. We’d like to have at least 3 people from both categories. If you’re interested, please send a message to our discussion board, or join #passenger on irc.freenode.net.

Thank you.