Sep 05, 2011
I recently started a new client project. Lucky me, Rails 3.1 has just been released so, it means I'll get to learn a bunch of things while building it, you know, the fancy asset pipeline stuff everyone's talking about and so on.
I wrote a couple of rspec specs and cucumber features, made them pass, and while I was about to start new features, decided to stop to check the current state of all the different rubies to try them and pick the faster one, at least for development.
Well, I was already using MRI 1.9.2, so, for research purposes I proceeded to install MRI 1.9.3-preview, Rubinus 1.2.5dev, REE-1.8.7 and jRuby 1.6.3. (Thank you RVM for making this as easy as possible).
Now, so far, my app has 4 rspec specs and 3 cucumber scenarios, but even so, I came up with interesting time results.
Below the results of running time rake for each implementation.
# Rspec Finished in 0.13588 seconds 4 examples, 0 failures # Cucumber 3 scenarios (3 passed) 10 steps (10 passed) 0m0.756s # Time real 0m24.259s user 0m20.752s sys 0m2.530s
# Rspec Finished in 0.05074 seconds 4 examples, 0 failures # Cucumber 3 scenarios (3 passed) 10 steps (10 passed) 0m0.379s # Time real 0m13.354s user 0m10.120s sys 0m1.546s
# RSpec Finished in 0.186 seconds 4 examples, 0 failures # Cucumber 3 scenarios (3 passed) 10 steps (10 passed) 0m2.141s # Time real 1m3.823s user 2m9.875s sys 0m7.005s
# RSpec Finished in 0.27061 seconds 4 examples, 0 failures # Cucumber 3 scenarios (3 passed) 10 steps (10 passed) 0m2.011s # Time real 0m47.328s user 1m1.049s sys 0m2.938s
Note that I used the following environment variables for this one:
RUBY_GC_MALLOC_LIMIT="50000000" RUBY_HEAP_MIN_SLOTS="500000" RUBY_HEAP_SLOTS_GROWTH_FACTOR="1" RUBY_HEAP_SLOTS_INCREMENT="250000"
To understand how this affects its performance, check out this post
# RSpec Finished in 0.05163 seconds 4 examples, 0 failures # Cucumber 3 scenarios (3 passed) 10 steps (10 passed) 0m0.277s # Time real 0m10.769s user 0m7.519s sys 0m2.106s
It looks like there's huge speed improvements from 1.9.2 to 1.9.3. It took 24 seconds on 1.9.2 and 12 on 1.9.3, that's a lot of speed boost!
JRuby, at 1 minute where the other rubies took seconds is light years behind them, though, I might be doing something wrong. I really have no practical experience with it and I just benchmarked it out of the box.
Rubinus felt kind of slow too, but, I consider it to be work in progress, or at least haven't heard of production apps running on it yet, which doesn't necesarily mean there aren't any. I'd actually love to use it on production for a few weeks when we launch this app just to see how it behaves in real life.
And last but not least, a properly configured REE kicks butt. No surprise there, we've been using it on another app on production for a year now with good performance and no problems at all so far.
So, there it is. I realize it is a small benchmark, but perhaps I can revisit it when we have more specs and more features to see if that changes how the rubies behave.