RSpec run fails, returns exit code 0

My team recently setup CI for a few Rails projects using TeamCity at work. When new commits hit a project’s repo if an RSpec test run passes the CI setup deploys the code to a Heroku site used by our QA team. One way or another some specs started failing on a project but the code still deployed to Heroku. That’s not how it’s supposed to work.

TeamCity looks at each command’s exit code to determine if a CI step fails or succeeds. If the exit code is 0 everything is copacetic and it continues on to the next step. Anything other than 0 means shit’s broke so it stops the process and logs failure.

It turns out the command rspec spec was returning exit code 0 on the CI server even when specs were failing.

After some research I found this Ruby bug which seemed to explain the behavior: at_exit bug with exception handling

The suggested fix solved our problem. I put the monkey patch into place in the project’s spec_helper.rb file and rspec spec exit codes are coming back 1 with failing specs and 0 with no fails on the server. TeamCity now stops short of sending broken code off to Heroku.

if defined?(RUBY_ENGINE) && RUBY_ENGINE == "ruby" && RUBY_VERSION >= "1.9"
  module Kernel
    alias :__at_exit :at_exit
    def at_exit(&block)
      __at_exit do
        exit_status = $!.status if $!.is_a?(SystemExit)
        block.call
        exit exit_status if exit_status
      end
    end
  end
end