Note: This is not actually describing existing functionality in Rails but is more of a brainstorming about how Selenium integration could look like.

There is a ticket for this:
http://dev.rubyonrails.com/ticket/1056
The latest patch (v3) implements most of the features described below.

[Also added to the ticket a simple driven version of selenium for rails. needs some work to get going on anything other than webrick with firefox for mac but is a start – BenGriffiths?]

This is a description of how it all works and can potentially be evolved into a book if the integration actually makes it into Rails proper.

What is Selenium?

Selenium is an open-source functional testing tool written by ThoughtWorks. It is the only open-source testing tool that supports a large variety of browsers and operative systems. It is implemented entirely in JavaScript and runs inside the web browser itself. Read more here: http://selenium.thoughtworks.com

Selenium seems to be the only viable open-source cross-browser testing solution on the market today. If there are other alternatives, please add your alternative to the section at the end.

Why integrate Selenium and Rails?

Offering an easy to use end-to-end automatic functional testing with cross-browser testing support out-of-the-box would be a huge differentiator for Rails.

Selenium is a bit painful to set up for a project and I would like a Rails application to come with Selenium support out of the box.

This is actually quite easy and has very little impact on Rails itself.

I also would like to make it possible to test the Ajax stuff in Rails as it badly suffers from cross-browser compatability issues.

Driven or FIT-style

There are two types of tests in Selenium: FIT and driven.

FIT tests are written in plain static HTML files consisting of a table with three columns. The first column should have a command or a verification and the other two are parameters. These files are then loaded into the Selenium test runner which is running inside a web browser. The test runner then performs the instructions contained inside the HTML files. Simple and intuitive. (The name FIT comes from Ward Cunninghams testing framework with the same name: http://fit.c2.com/)

Driven tests are quite a bit more complicated. They are written in Ruby (or any other supported languages) and enqueue commands to be executed by the Selenium test runner inside the web server. The runner polls the webserver for new commands and executes them.

Driven tests can set up test data, use fixtures and all other items from the existing testing framework.

Unfortunately the infrastructure is significantly more complex
to get to work than FIT-style tests, we might get it to work for
Webrick but FastCGI? would be incredibly complex.

I think we should initially focus on getting FIT-style tests to work and then move on to driven tests. I’m going to leave out driven tests from this discussion henceforth.

There seems to be a wonderfully clever hack to make driven work here:
http://moriq.tdiary.net/20050325.html

Unfortunately it’s in japanese. :-(

Packaging and installation
(TODO: stubs only)

Selenium as a Gem.

gem install selenium

Selenium test generator. Generate Selenium test automatically for each controller? Selenium tests are end-to-end so it doesn’t actually makes sense to structure around controllers. Should be structured around functional area or use-case/story.

The Rails gem should depend on the selenium gem.

Writing tests
(TODO: stubs only)

./script/generate selenium user_logs_in

Tests should be placed in test/selenium.

Generator should also generate or update the TestSuite?.html file. (TODO: what to do about ordering here? Many functional test suites end up depending on which order you run the tests. Can we do some really clever thing that appends the newly generated test to the end?)

Setting up test data

It’s usually not feasible to set up test data using the web GUi for each test. It could be a lengthy involved procedure that complicates the test itself, or it could even be that some data is just not possible to enter using the web GUI. In that case we can simply call the action of a special test data controller using the “open” command. This controller is not loaded

Example:
|open|/testdata||
|open|/item/view/1||
|assertTextPresent|Test item 1||

The Selenium test generator could automatically generate the controller and ensure that it’s not deployed on a production system. It could look something like:

# call this controller from your selenium tests to set up test data 
# using the 'open' command, the second column should contain the url
# eg. "open /testdata" for the default fixture
# eg. "open /testdata/clean" for a clean database
# eg. "open /testdata/specialfixture" for some specially tweaked fixture

# only load in development and test environment
if ['development', 'test'].index(RAILS_ENV) then

# TODO require the fixture stuff
class TestDataController
  fixture :items

  def index
    # default test data, the fixtures as above
  end

  def clean
    # completely empty database
    # TODO how to completely clean out all the data from the fixtures?
  end

  # add more test data fixtures by adding actions here 
  # and calling them from your selenium test 
end
end

Executing tests

Start Webrick with Selenium deployed:

./script/server —selenium

Starts server with Selenium deployed.

Go to http://localhost:3000/selenium/TestRunner.html

The GUI is pretty self-explanatory. Hit “All Tests” on the right hand side to load all the tests. Select “Walk” to run tests slowly for cool demo effects. Select “Step” to run one step at a time. The embedded frame at the bottom is the actual application under test.

With Apache we need some rewrite rules in .htaccess to put the Selenium test runner files under /selenium and the tests under /selenium/tests.

What about LightHTTPD?

Automatic testing (part of continuous integration)

rake selenium

Starts server with Selenium, starts web browser and runs all the tests.

Cross browser testing

rake selenium_cross_browser

As above but auto-detects installed browsers (This is tricky!) and executes tests for each browser. Can also manually specify browsers which would fail if a browser can’t be started.

Documentation
(TODO: stubs only)

Would require tutorial for introduction and writing simple tests and further tutorial with some more advanced usages (testing Ajax etc).

Selenium itself suffers badly from bad documentation. Maybe it would be easier to just write reference documentation for the Ruby driven approach and host it here. It’s not very complicated.

Alternatives to Selenium

If there are other alternatives to Selenium please add them here. Preferably they should meet some or most of the following criteria (in order of preference): open-source, supports multiple browsers (Safari, IE5.5+, Mozilla/Firefox), supports multiple operative systems (Linux, Windows, Mac OSX), tests can be written in Ruby. (These are all met by Selenium, although Safari support is a bit flaky at the moment.)

Watir. Open-source. IE only (5.5+). Windows only. Tests are written in Ruby. Instiki includes a Watir-based testsuite that may be worth reviewing.