Nov 13, 2011

Ruby on Rails upgrade from 2.3 to 3.1

Recently I've performed the upgrade of Ruby on Rails which we use in Checkvist project from 2.3 to 3.1. There were tons of issues I've had to overcome, some of them I share here, may be they could save someone's time.

 First, upgrade or not upgrade? This is a tough question. Upgrading of a non-trivial project may take several days, and at the end you may get a system which performs worse than before. So you'd better have a good set of system tests, including performance tests.

 In my case, I used Rails 2.3.11 with my performance-related patch for using it with ruby 1.9.2, and I aimed at migration to Rails 3.1.1 (latest at the time of migration). I really like the assets pipeline feature introduced in the 3.1.x version of Rails, and I wanted to start using ruby 1.9.3. After all, new features, plugins, enhancements are developed against latest version of RoR.

At the time of writing  Checkvist is running RoR 3.1.2, and I'm pretty comfortable with its performance.

And here is a list of notes I made during the upgrade (the more complicated is your project, the more unexpected the issues):

General upgrade notes

  • Upgrade from 2.3 to 3.1 consists, actually, of 2 parts: upgrade to 3.0 and upgrade from 3.0 to 3.1. Some deprecations, which were allowed in 3.0, are obsolete in 3.1 (for instance, you must use the new routes syntax in your project).
  • One step which you can do before "breaking everything" during the upgrade - migrate your 2.3 project to use bundler. See the bundler page for the instructions - it is not that painful. Do not forget to update your Capistrano deployment script to handle bundler.
  • In my case, Checkvist was already compatible with ruby 1.9.2, so I used this version of ruby during the upgrade, and after that, updated ruby to 1.9.3
  • All mentiones of RAILS_ROOT, RAILS_ENV should be replaced with Rails.root and Rails.env, respectively. The same applies to Rails.logger .
  • uninitialized constant Syck::Syck problem should be resolved by running gem update --system
  • Rework all old-style ActionMailers to use the new syntax for sending mails - old syntax doesn't work in Rails 3.0 anymore. Also, check naming of your e-mail templates.
  • All files with extension .rhtml should be renamed to .html.erb

Upgrading to 3.0

    • For upgrade steps, I followed instructions from railscast, and used rails upgrade plugin
    • Upgrade of the routes.rb file may be non-trivial. Having tests for routes makes this process less painful. I don't recommend using automatic converter - it is better to understand what's going on there and make your route rules cleaner.
    • Update/rewrite of environment.rb file is rather simple: old stuff goes mostly to application.rb + session_store.rb; non-typical mime types are specified in mime_types.rb .
    • Gemfile: most gems have new, updated version for Rails 3+, no problems encountered here. All issues described in the early migration tutorials with will_paginate or thinking-sphinx gems are already resolved. 
    • After the update, I've removed some obsolete plugins and initializers (and some monkey-patches for rails 2.3 code). May be you have to review them as well :)
    • I had to add 'dynamic_form' gem to enable old-style helpers for error_message_on and others
    • Ruby files from 'lib' subdirectory were invisible until I uncommented "config.autoload_paths += %W(#{config.root}/lib)" line in application.rb
    • I didn't have lots of link_to_remote and remote_form_for helpers (which disappeared in 3.0), so I updated the code explicitly, using my own AJAX helpers in the corresponding places.
    • Cucumber integration tests. This part was tricky, because I used Cucumber with usual Rails integration tests (no webrat, no capibara). I needed POST and PUT requests. And support for such direct integration was broken in the newer version of Cucumber. But, you can use Cucumber directly with Rack::Test, and get the application response from last_response method of Rack::Test - and this was sufficient to modify my tests accordingly.

    Upgrading to 3.1

    • The path for upgrading to 3.1 was described in this post, but in my case the switch to using assets was not simple. 
    • I didn't use a single application.js and application.css files for the whole application (because in this case, after each update all .js and .css code would be downloaded by users, no caches). I've created several files, more specific, per page/type of page. To accomplish this, section config.assets.precompile of environments/production.rb  should include all paths for all .js and .css files used by your application
    • To deliver asset files in production, you have to change Nginx configuration as described in this great guide

    Performance notes

    • To get a decent app performance, comparable with 2.3.x, use ruby 1.9.3 and  specify garbage collection parameters.
    • It is a bad idea to override often-used attribute method in your model (for instance, to provide a default value). In Rails 3 such code works damn inefficiently - it doesn't create effective accessors for such methods
    • Do not use changes method to get information about which attributes changed - use changed_attributes for that

      So much for the upgrade stories. Will be very glad if you find some of these notes useful in your riding on Rails.

      Jun 13, 2011

      Prototype 1.7 memory leak

      Lately, I've been trying to fix a memory leak in TeamCity. After a long investigation, I found out that DOM elements on the page remains in memory even after a simple construct like:
        element.on("click", Prototype.emptyFunction).stop();

      This code adds a fake event listener on a element and immediately removes it (all using Prototype javascript library).

      I.e., after executing the code and removing the element from the page, it remained in browser memory.
      (BTW, I used Google Chrome memory profiler to find out hanging elements).

      So, the leak was caused by the Prototype's code, and I found out this nasty line:

      Ironically, Prototype tries to avoid memory leak in IE by removing all event handlers on page unload, and for that, keeps a collection of the elements in the CACHE array. But, it never removes elements from the array (only on page unload), even when all event handlers are removed from an element. For any heavy ajax application, when elements are added/removed/replaced on the fly, this may be very unpleasant.

      I've fixed the problem in my Prototype fork on GitHub, and added a pull request to Prototype. Hope, it will save someone some hairs.

      You can also download patched prototype.js with the fix.

      Update: my original fix introduced a performance problem in stopObserving method (when many elements were observed on the page). Now, the problem is fixed, all related resources above were updated.

      Feb 13, 2011

      Checkvist downtime postmortem

      Checkvist service was unavailable since Feb, 13 04:13 UTC till 07:40 UTC.

      The total downtime was 3 hours 27 minutes. Users couldn't see/modify their information, but no data corruption occurred.

      We are sorry to everyone who was unable to access their data during that time. We've already taken some measures to prevent such problems in the future, see more details below.

      What went wrong

      • On Saturday, Feb 12, we've updated software on the production server (ruby, passenger, nginx).
      • At 04:13 UTC a daily server maintenance procedure was started. Some parts of the procedure were incompatible with the new installed software, and access to Checkvist was broken.
      • E-mail and SMS notification were sent to us. We didn't see e-mail notification because we were away from computers. The SMS notification was sent to an obsolete phone number :( So the problem was unnoticed until the morning, when we've checked e-mail.

      Problem resolution

      Measures to prevent similar problems in the future

      • Daily maintenance procedure was corrected
      • We've set another time for the daily maintenance, so it would take place in the daytime
      • We've updated our monitoring service. Existing phone number for SMS notifications was corrected, we also added another phone number for the notifications. We've also installed iPhone app which can notify about service failures.
      And finally, why all these explanations. I think it's better to be transparent and honest about the problems than make people guess about system's reliability.

      Jan 28, 2011

      Javascript error reporting in various browsers

      Again, about fighting with Javascript errors.

      I've just found out, that Opera browser in some situations provides the most detailed information about javascript errors. At least Opera 11 is rather good.

      Compare the error reporting in various browsers (all on Mac):

      FireFox 3.6.13+FireBug 1.6.1 - didn't report the problem, neither in FireBug console, nor in FireFox javascript errors window.

      Chrome 8.0.x
      gave this:

      And the winner, Opera 11:

      See the difference?