Some people know me as the retired “Chief Pain-in-the-Ass” (a.k.a. “Director of Platform Architecture”) of DigitalRailroad (warning: previous link may not work too much longer!) (DRR). Not quite sure how much weight should be placed in that title as the primary applications that ran DRR merely grew rather than being “architected”. Oh, don’t get me wrong, there were parts of DRR’s software that had a lot of careful design and thought put into them, but the core apps were basically monolithic ASP.NET applications that started out with a particular structure and simply grew that structure over time.
Eventually it became clear to all that something had to change.
Sometime in the last two years I finally made a successful case to not only do something about the code base that was crushing the life out of the development team (especially me), but to finally get off of the Microsoft platform and migrate to GNU/Linux. I vigorously argued for a complete rewrite in some new framework, but lost that argument. Instead, it was agreed that we would eventually get everything off of ASP.NET but it had to be done incrementally.
What should we use?
The first thing at that point was to decide on some framework to work with. I wanted to avoid a compile+deploy system so I wasn’t considering Java-based solution at the time (more on this in another post). The shortlist basically consisted of:
- Ruby on Rails
- PHP+<something that made developing with php not suck>
- Perl+<some wicked-cool toolset to make perl Web development nice>
Yes, some will say that I should have considered Python and a framework like Django or Twisted, but seriously, I’m just not a Python fan, I can code in it, but I get no joy out of it (yes, I know it’s good enough for Google ;-) ).
After some testing, and a lot of discussion Rails was the winner. Now, I had been doing a lot of work in Rails the prior year and change. I had been showing off mockups of DRR in Rails since something like version 0.10 of Rails, but in all that I had learned a fair amount about what does and does not work well in Ruby and Rails.
Rails? What worries?
There were, and are, a lot of things to like about Rails, and frankly, I am rather happy that David Heinemeier Hansson (a.k.a. DHH) got inspired to use Ruby the way he did. Ruby is really a pretty language, and Rails is a pretty framework, but both have their demons.
Firstly, Ruby is slow. Yeah, yeah, I’ve read all over the place that I shouldn’t obsess over Ruby performance: “it’s almost never Ruby’s fault the your application runs slowly: focus on the DB.” Except that that bit of advice is complete bullshit. Ruby is slow and exhibits it slowness in painful ways, i.e. when you’ve done something to waken the Ruby slowness demon you will know it right away, it is not bashful. Like what for instance? How about looping over something with conditional code in it, such as:
def do_something(args)
result = []
args.each do |i|
if i < 0
result << "Less than zero"
elsif i == 0
result << "Zero"
elsif i == 1
result << "One"
else
result << "Greater than one"
end
end
result
endNow, the above is a bit contrived, I agree, but it shows the basic nature of the beast: if your code is chock full of executable statements as opposed to things that map directly to a C-call (like working with a Hashtable) you run the risk of your code being very slow. Much of this problem is being addressed in Ruby 1.9.x with the YARV VM, but in Ruby 1.8.x innocuous day-to-day programming can become your worst nightmare. The solution, in general, is to re-design your code to eliminate the use of intrinsic Ruby statements and structure your code in a way that almost exclusively uses constructs that are directly implemented in C (i.e. native). This performance feature ;-) of Ruby periodically makes an appearance resulting in lost development time appeasing the Ruby performance gods. Then there are all sorts of performance problems that arise from garbage collection activity, but I won’t go into details on that (lucky you).
What about Rails? Well, Rails has it’s own performance issues (not the speediest of frameworks, but not horrible either), but more importantly, Rails had (at the time) a tendency of being a memory hog and potentially unstable, furthermore, deploying Rails apps was not fun using FCGI. Fortunately, the FCGI problem was solved with Mongrel, but memory and stability were still a concern. Also, one of the painful lessons we learned with the ASP.NET site is that monolithic applications suck-ass in all directions: we wanted our “new” system to be modular, so we could update parts of the site easily, as well as scale-out apps individually instead of having one big app scaled to meet the performance needs of the slowest part. So, how would Rails help with that? Well, it didn’t out of the box. There were also a number of wacky things DRR was doing with URLs that simply did not play nice with Rails either.
Arguing with Rails
DHH says Rails is “opinionated software”, so when you have to change something about Rails, I call it arguing with Rails;-). I had a number of fundamental arguments with Rails, the two most devilish were:
- Getting Rails to work with and generate the URLs that DRR used on the site
- Making it easy to build what appeared to be a single application to the outside world as a discreet set of Rails instances.
On our site there were 1 or 2 ways to address a particular photographer’s archive. If you did not have a custom domain for your archive then your URLs looked like this:
http://www.digitalrailroad.net/<archive name>/...
If you had a custom domain (a.k.a. “domain pointed”) for your archive then either of the following URLs would work, but preference was given to the one using your custom domain:
http://<custom domain>/...
- OR -
http://www.digitalrailroad.net/<archive name>/...Now, for various reasons, including the fact that in a development environment you didn’t want to keep having to edit your /etc/hosts files it was necessary for Rails to intrinsically be able to deal with and generate both kinds of URLs using Rails’ built-in routing mechanism. In order to accomplish this I created the DRR Rails plugin. This plugin monkey-patched Rails’ routing guts to transparently deal with the DRR urls, and to boot it made available all sorts of wonderful data about the archive the user “was in”.
The other problem of making it easy to build DRR as a set of Rails instances was solved with a number of additions to the DRR Rails plugin. The plugin enabled an easy way to package a shared collection of routes and route files in the plugin itself. It also effectively implemented a namespace mechanism (I called them grouped routes) that allowed URLs to be generated to any route (using named routes only, then again, naked route generation has been shunned for quite some time now) but only recognize routes for the current application instance. This approach promoted a lot of DRY code and made it possible to work on and develop isolated parts of the DRR application. The whole approach I called the “micro-apps” way to build a site.
The above approach worked really well, and it resulted in achieving the sought-after goals: componentized development, and performance and process isolation. Slowly but surely we could replace the old ASP.NET application.
Towards the end of DRR I began working on a port of the plugin to Rails 2.x. Since the plugin is a lot of monkey-patching of Rails core I knew there was no way it would just work in Rails 2.x. Probably 1-2 weeks of concerted effort and it would have been done. Rails 2.x did also promise performance improvements.
Hey, didn’t you mention something about Merb in the title?
Yes, yes I did. The tie in to Merb is that Merb has a more hacker friendly approach to building things than Rails IMAO. In particular, because of Merb “slices” Merb has a clean and robust implementation of URL namespaces and application componentization that would have worked very nicely at DRR, also the Merb routing mechanism is nicely pluggable meaning a lot less (perhaps even no) monkey-patching! This would have been a real productivity boost had I had it early on. Alas, at the time I was doing all of this Merb was barely a glimmer in someone’s eye, but it seems clear that the Merb folks have intelligently learned from Rails’ history. Furthermore, the Merb folks take performance very seriously and have done wonders to achieve high performance with a powerful framework that runs in a fraction of the footprint of a typical Rails app. In fact, Merb has shown up on this site as it runs all of the non-blog content (not much at the moment, but I “have plans”).
Expect to see more on Merb in the future. For now, if you are considering a Ruby Web+ framework Merb should rank prominently on your “short-list”.
Tags: Fun, GNU/Linux, Hacking, Merb, Programming, Rails, Ruby, Web
