08 May 2012 Benchmarking Rails 3

tl;dr

While Rails having become faster from version 3.0 to 3.2 regarding most parts of it, it actually became much slower when simply reading from the database.

Details

A couple of days weeks ago, when starting to switch over to Rails 3.1 from Rails 3.0, our project Sharesight got slower and we simply couldn't find out what the reason was. Especially where my own experience with Rails 3.1 was a good one, and most projects were at least a little bit faster than running with Rails 3.0 . But not this time. We finally decided to start creating new blank Rails apps, benchmark certain things and compare the results. And here we are now.

We created the GitHub repository Simple Benchmark and could reuse some of TST Media's groundwork, so we didn't start from scratch. The project consists of three Rails apps, one based on Rails 3.0.12, one on 3.1.4, and finally one on 3.2.3, the most recent stable one. We provided some tests with certain scenarios, which we think are the most interesting: reading from and writing to the database, and processing a couple of get requests (without database access). We wrapped it all up in some benchmark reports, resulting in two charts.

The first chart shows four lines: reading from the database (blue line), writing to the database, writing with some life-cycle hooks turned on, and finally writing with some validations turned on. First surprise was that running validations (green line) took much longer, but maybe it's because of the regular expression we were using.

Chart 1: Accessing the database

But there's one more thing which actually comes as a surprise. All write requests got slightly faster while moving up the Rails versions. But reading from the database got slower. And it's actually much slower, it's more than 40% slower if you switch from Rails 3.0 to Rails 3.2! This means that displaying content from the database got more expensive, while writing to it was improved, which is a major point for most websites, I guess.

As you can see in the second chart, times improved from Rails 3.0 to 3.2, though being even better in Rails 3.1. Rendering with partials (the yellow line) took - as expected - the longest time to process.

Chart 2: Rendering

So, what does this all mean?

For websites heavily reading from the database this is a major impact, since a surcharge of 40% is quite a change to the worse. So if you have a lot of database selects per request (actually hitting the database, not coming from any cache), you better start off optimising your site before switching over to Rails 3.1 or 3.2 . We had an impact of around +25% on our heaviest request, so we started optimising the processing time now.

Feel free to take a look at the benchmarking setup we chose. Any comment is appreciated!