Note: a Pivotal Blabs version of this article is available here.

Selenium RC Fu is a fantastic system for testing Ruby On Rails applications. It is the blending of xUnit testing with selenium. Selenium is an amazing system that operates your browser as if a human were sitting there moving the mouse, pressing buttons and keys.

Selenium RC Fu is also a remarkable example of the power of open source. It’s Selenium remotely controlled by rails and ruby. You can learn more about it by viewing the slides for Full-stack webapp testing with Selenium and Rails presented by my colleagues Alex Chaffee and Brian Takita at the SDForum Silicon Valley Ruby Conference.

Now that you are excited about Selenium RC Fu, by law I must inform you that this wonderful testing tool comes with some costs. First, this is the daisy cutter of testing — problems will be detected, but it won’t be too specific about those problems. A failed selenium test will likely only tell you some expected text was not present on the page — you have to do some digging to discover the real problem.

It’s also slow. To be fair, a lot of software is running to do this testing.

So use selenium testing sparingly. A good strategy is to restrict selenium testing to “happy path” testing. These happy path tests become a compliment to other more focused and faster unit and integration tests.

Getting Started With Selenium RC Fu

Selenium RC Fu is hosted at rubyforge.org in the “pivotal.rb” project. It’s not well documented and a little hard to find. The primary documentation is the README File.

Step 1 is to add it to your project:

    script/plugin install svn://rubyforge.org/var/svn/pivotalrb/seleniumrc_fu/trunk

Building Your Tests

Create your tests in app/tests/selenium. The basic structure mirrors the other test types:

    require File.dirname(__FILE__) + "/selenium_helper"
    class HappyPathsTest < MyProject::SeleniumTestCase

      def test_nav_bar
      	  ...
      end

      def test_about
      	  ...
      end

      # more tests
    end

Note that HappyPathsTest extends something called MyProject::SeleniumTestCase. Selenium RC Fu provides a sample selenium helper file that suggests this convention for name spacing your tests.

For the happy paths tests I simply wanted to go to a page and know that the page loads. A pattern I learned from my colleague Shifra is:

1. Pick something you know is on the target page, but not on the current page. Let’s call it ‘evidence’. Assert that evidence is not present on the current page.
2. go to the target page
3. assert the evidence is present on the target page.

This process insures that you are moving about as you expect. Here’s an example from my project:

      def test_tasks
        ...

        assert_element_not_present "xpath=//h1&91;text()='The Daily Planet']"
        click_and_wait "link=The Daily Planet"
        assert_element_present "xpath=//h1[text()='The Daily Planet']"

        ...
      end

I repeated this simple pattern for every path.

Selenium Functions/Assertions/Commands

Here are the functions and assertions I used in my tests:

• click_and_wait locator
• go_back
• wait_for_page_to_load
• type locator, text
• assert_text_present text
• assert_text_not_present text
• assert_equal value1, value2

Many of the functions require a ‘locator’ argument. This needs to identify a single element on the page. Often the name or id of an element is sufficient, but you may need to use an XPath. Check the docs for more information on element locators.

These functions were sufficient for me. Look for more in vendor/plugins/seleniumrc_fu/lib/seleniumrc_fu/selenium_dsl.rb.

Selenium RC Fu comes with some rake commands too (from rake -T selenium):

    rake selenium:restart_servant           # Stop and start the selenium servant (the server that launches browsers) on localhost
    rake selenium:run_server                # Run the selenium servant in the foreground
    rake selenium:server                    # Run the selenium remote-control server
    rake selenium:start_servant             # Start the selenium servant (the server that launches browsers) on localhost
    rake selenium:stop_servant              # Stop the selenium servant (the server that launches browsers) on localhost
    rake selenium:test                      # Run a selenium test file (default test/selenium/selenium_suite)
    rake selenium:test_with_server_started  # Run the selenium tests in Firefox

Running Your Tests

You can run your selenium tests individually or as part of a suite. Selenium RC Fu comes with a suite script in the vendor/plugins/seleniumnrc_fu/sample directory. Drop that in your test/selenium directory.

The selenium server must be running before running individual tests. You can start the server with the rake command rake selenium:server. Once the server is running you run your tests from the command line or via the suite.

You can also run the suite with the command rake selenium:test. This command is smart enough to start the server if it’s not already running. However, when the server is started by this mechanism its a little more work to stop it. You can stop the server with a simple ^c if it was started via the rake selenium:server command.

Make It So

That’s really all you need to build your tests but here are few more things that may help.

Site Diagram

A site diagram is handy for building your happy paths tests. There will be a lot of paths through the site and marking the paths on the diagram as you visit them is an easy way to track your progress. On the diagram you need to indicate both pages and the paths to those pages.

The later is really important. It’s the paths that you will be testing with your happy path selenium testing. Often there are buttons, links, or other controls such as check boxes that may touch the server and reload the current page, or just update the current page via ajax. Be sure to add these to your navigation diagram.

Here’s the diagram I used for my site.

Selenium IDE

You can code your tests by hand using Selenium RC Fu functions but it’s easier if you use Selenium IDE. This is a Firefox plugin that records your activity. That activity is available in various selenium dialects and can be pasted into your tests. Unfortunately, it does not have output for Selenium RC Fu. Nonetheless I still found it useful. I simply clicked through my application while Selenium IDE was recording, pasted the resulting selenium commands into my tests, then converted those commands to equivalent Selenium RC Fu commands. For example:

Output from Selenium IDE:

    @selenium.click "link=My Account"
    @selenium.wait_for_page_to_load "30000"
    assert @selenium.is_text_present("Update Account")

Becomes:

    click_and_wait "link=My Account"
    assert_text_present "Update Account"

Test Data

Your selenium tests will be exercising your application just like standard unit or integration tests. These tests require data and there are a variety of techniques to make data available to your tests. Building your happy path tests is easier if you have fixture data and that fixture data is loaded in your development environment via rake db:fixtures:load. If you use fixture scenarious, it is handy to have a scenario for selenium testing, which you would load with something like rake db:scenario:load SCENARIO=selenium.

With your fixtures loaded in your development environment you can see exactly what is present during testing and the selenium IDE will record exactly what is played back by selenium.

XPath Checker

Usually Selenium IDE can assign a locator that simply works but there are times when it can’t or won’t. Using XPath Checker you can right click on an element and it will display an XPath to that element. You can also experiment with varitions and XPath Checker will list all of the elements on the page that can be identified by that XPath.

That’s it. Let me know how it goes.

Note: a Pivotal Blabs version of this article is available here.

Recently I was about to check in some changes and did a last minute click through of the application. All of a sudden I’m staring at a stack trace. My tests were green and I had functional tests for the failing
controller/action.

Tests are like pants — they cover your backside while you focus on other things like adding features to your application. Suddenly I felt a breeze on my cheeks. Something was amiss.

I soon discovered the action and its associated tests had diverged over time. Some of the parameters were renamed in the action but not in the functional test. Since some of the work of the action was
conditional on the presence of certain parameters, that work was no longer being tested.

This exposed weaknesses in the tests and code, such as expected side effects in the tests that are never checked. If they had been checked the tests would have failed and the parameter name mismatch
would have been discovered.

Most functional tests provide specific parameters that should at least be examined during the processing of the action. Reporting unread parameters would strengthen those tests. It was conceivable to me that
some of the other functional tests had similar unused parameters. I wanted all of my functional tests to report all unused parameters.

The first step was to instrument the params hash. I wanted to track access to the params hash and report parameters that were not read during the processing of the action. I don’t know what all is done to params during the life cycle of a test. I’m only interested in access
from the time the action starts till it returns so I need to be able
to turn the tracking on and off at specific times.

It turns out that Rails uses a subclass of Hash called HashWithIndifferentAccess. I added my changes to HashWithIndifferentAccess in test/test_helper.rb:

    class HashWithIndifferentAccess
      def [](key)
        @accessed_keys ||= {}
        @accessed_keys[key] = true
        super
      end

      def start_logging
        @accessed_keys = {}
      end

      def end_logging
        @accessed_keys['action'] = true
        @accessed_keys['controller'] = true
        never_accessed = []
        self.each_key do |key|
          never_accessed << key unless @accessed_keys.include?(key)
        end
        raise "Some keys never accessed: #{never_accessed.join(', ')}" unless never_accessed.empty?
      end
    end

With these changes an exception will be raised if any first level keys are not read between start_logging and end_logging.

In each of my functional tests I added code similar to this (from account_controller_test.rb):

    class AccountController
      around_filter :check_params
      private
      def check_params
        params.start_logging
        yield
        params.end_logging
      end
    end

The around filter starts and ends the logging in the context of the action.

With these changes in place my tests no longer passed and my backside was warm and protected again.

I continue the quest to use the leopard provided ruby and gems. Unfortunately, I’ve made some mistakes. Perhaps I can save you from doing the same. Want to benefit from my experience without investing by reading the story? Skip to the recommended process at the end.

The Story

Part I: Anticipation

I upgraded to leopard recently. Before doing so I read about leopard’s new and improved ruby and rails support and some of the issues around this support. Since I had previously installed ruby and rails using Dan Benjamin’s excellent “Building Ruby, Rails, Subversion, Mongrel, and MySQL on Mac OS X” I anticipated that, at the least, I would need to make some adjustments when it was time to install or update gems so I wasn’t surprised when I got an error while running geminstaller on one of my projects. I’m not blaming geminstaller. I suspect the problem would have occurred if I had attempted my installation with a standard “sudo gem install” command.

Part II: Just Like I Had Good Sense

I decided to move forward by removing my installations of ruby and rubygems then using the leopard installation for future gem installs. The first step was to remove the installations of rubygems and ruby. Fortunately, I had saved the source directories that I used to install these features originally. To remove rubygems, I did the following:

$ cd /usr/local/src/rubygems-0.9.2
$ sudo rm `cat InstalledFiles`

This will remove the gem excutable, but not the gems themselves. I expected to remove the gem library when I removed my installation of ruby.To remove ruby I thought I could do something similar:

$ cd /usr/local/src/ruby-1.8.6
$ cat .installed.list | xargs -L 1 sudo rm -dfv

I ran that last command over and over again until it no longer deleted anything. I could have changed the rm command to ‘rm -rf’ but I wanted to see what got deleted. At some point I realized I was only deleting ‘ri’ stuff so it was time to do some more research.

Part III: The Awakening

Apparently you can specify an option when installing ruby from source to keep a list of all files installed. I didn’t know about this and didn’t do it. It would have been helpful.Instead I attempted to decipher the makefile to determine what gets done when you say ‘make install’. It turns out most of the work is done in a script called instruby.rb. I sifted through that script and attempted to devine where and what it moves into place when performing an install. I listed every directory and file, then removed the duplicates and came up with this set of commands for removing ruby:

sudo rm -rf /usr/local/lib/ruby
sudo rm /usr/local/bin/ruby
sudo rm /usr/local/bin/erb
sudo rm /usr/local/bin/irb
sudo rm /usr/local/bin/rdoc
sudo rm /usr/local/bin/ri
sudo rm /usr/local/bin/testrb
sudo rm /usr/local/share/man/man1/ruby.1

I expected that typing $ ruby -v would show the leopard version of ruby. Surprisingly, I saw something like “/usr/local/bin/ruby not found” (I don’t remember the exact message). Of course the issue was that the previous ruby had been hashed by the shell. Fixing it was simple: $ hash -r. Now I see:

$ ruby -vruby 1.8.6 (2007-06-07 patchlevel 36) [universal-darwin9.0]


Then I tried a few commands installed as gems. Here again there were errors. This time the solution was to remove the associated bin file from /usr/local/bin. At this point I realized that all gems with executables that I had previously installed had entries in /usr/local/bin that would likely generate errors.

Part IV: Drastic Measures

It was at this point I elected to remove the entire /usr/local and /opt/local directories. Keep in mind that I have a relatively new macbook and I can still fairly easily reinstall anything that I had previously removed from there.Now my gem installs are working correctly when I apply some of the new darwin solutions outlined below.

What would I do now, with the benefit of hindsight?

1. First, why change your existing ruby and gems installation? If it works, you’re done. Wait till the kinks are worked out before investing your time.
2. If you really want to use the leopard versions of ruby and rails, don’t install leopard… Not Yet!
3. Remove all of your existing gems, using “sudo gem uninstall [gem name here]. You don’t need them in their current location. Remove them first while you can use the gem utility. This will remove them both from the gem install directory and also from the /usr/local/bin directories (for gems that have bins).
4. Uninstall ruby using the commands listed above.
5. Now install leopard.
6. Update xcode from the leopard install disk.
7. Do not gem update --system (see this thread on apple’a ruby support discussion)

At this point you should be able to use rubygems to install and update gems in the new normal way. What’s that mean? I’ve encountered these issues:

1. The well documented architecture issue for some gems. Hint: try something like “sudo env ARCHFLAGS="-arch i386" gem install [gem name here]“.
2. The ‘gems’ command apparently attempts to update gems even when they don’t need updating. Why update a gem that doesn’t need updating? I don’t know, but if you try it will often fail with

Directory /Library/Ruby/Gems/1.8/doc/[some gem]/ri already exists, but it looks like it isn’t an RDoc directory. Because RDoc doesn’t want to risk destroying any of your existing files, you’ll need to specify a different output directory name (using the –op option).

Try adding the “–no-rdoc” option.

If you see “missing ruby header files” when installing a gem then you probably missed the “install xcode” recommendation above.

The East Bay Ruby Meetup for February was all about Capistrano.

Capistrano’s creator, Jamis Buck, says capistrano

…is a utility for executing commands in parallel on multiple machines, such as for automating the deployment of applications

Three of us talked about capistrano, deployment and scaling. My talk was titled “Bread and Butter Capistrano” because I wanted to emphasize that I use it as a tool to get my work done, find it very helpful, and use it all the time, but I haven’t spent a lot of time studying the tool itself.

Here’s the talk. Enjoy!

Bread and Butter Capistrano

If you’ve read this far, please consider rating me at Working With Rails.

I’ve been using a tool for user interface prototyping and testing called “DENIM“. DENIM is an application that allows you to sketch a user interface using a tablet connected to your pc or mac. It allows you to zoom in and out to create varying levels of detail as needed.

For example, often you will first sketch the general flow of the site. This will look like boxes with a textual name or title. You can then draw lines between these boxes indicating a flow from one box to the other. With DENIM, you can then select a box and zoom in. When you do this the box becomes a page. You can then add details to the page, such as a (sketched) forms, buttons and links. Now you can draw a line from one of these sketched buttons to another page and the sketched button becomes a live link. What does this mean? Well, DENIM allows you to save your prototype UI as html, and when you do it becomes a collection of clickable images — your sketches — embedded in html. When you click on one of the hot areas, which are blue (non hot areas are black ink on white, hot areas are blue ink on white), you jump to the target page.

Suddenly you have a live prototype. I’ve shown these live prototypes to customers and colleagues. At first people seem perplexed. They seem to think I’ve scribbled on a table, taken screen a shot and this is how I’m going to show them the proposed UI. Then they click on a link or two and suddenly they get it… and off they go exploring the whole site, spewing out suggested changes. After a few minutes you go back to your office and spend a few hours coming up with a new version. A couple iterations of this and you and the customer have a pretty good understanding of what is to be built.

But It’s Not All Rosy

This program feels abandoned. It has some bugs. The web site has a video of a future version, but the video is a few years old and the features in the video have not been released. I’ve learned to save my work often, and to save it as different versions. Occasionally your DENIM model will become corrupted and sometimes you cannot fix it, so you have to revert to an earlier state. DENIM creates snapshots for you but I lost trust in them early on and haven’t tried again. DENIM also seems to have a size or complexity limit. When the complexity of your UI exceeds some level the model can no longer be opened. I’ve also noticed that some machines can open some models while others cannot — perhaps it is a memory issue.Even with the bugs DENIM is a worthwhile tool to have in your bag. Please check it out.

Getting a cygwin terminal emulation to work properly with a remote unix or linux host has been challenging. The problems are primarily when interacting with emacs, but there are various other issues, such as unix curses applications (getting box drawing characters to display correctly).

I’ve recently switched back to the cygwin xterm as my primary terminal emulator. Previously I used rxvt. I switched to xterm for a variety of reasons (read “not well defined”), some of which are:

• I was dissatisfied with the colors in emacs. I started experimenting with colortheme.el, and noticed that the color themes as displayed on the color-theme web page were different from what I saw in my rxvt window. I don’t understand why.
• font issues have been frustrating. I don’t know if I’ll run into these problems with xterm or not. Specifically, I had to jump through hoops (read “use an rsvt specific term setting”) to get line drawing fonts working. That broke other things.

challanges

challenge: get delete working correctly in emacs on solaris via xterm

• discussion: delete should delete the character under the cursor. Delete is the same as backspace on solaris.
• status: solved! I added this to my .emacs:

(define-key function-key-map [delete] [deletechar])

This will likely break something else, especially since I share this .emacs with linux systems. Stay tuned.

challenge: get backspace working correctly in emacs in xterm

• discussion: backspace in emacs should delete the previous character. ^h brings up the emacs help system, “info”. Backspace keys often generate a ^h character.
• status: solved! I created an .Xdefaults with this line “xterm.*.backarrowKey: false“. Check out the discussion of this in
  Squishywishywoo: delete and backspace keys under X (emacs, xterm) under “xterm”.

challenge: get shift-arrow keys to work in rxvt

• discussion: I’m checking out windmove.el and its default key bindings use shift-some arrow key. The keys codes that emacs received were unrecognized.
• status: solved! I installed rxvt.el. I grabbed the 1.4 version and put it in the ~/emacs-lisp/term directory. Then I changed my $TERM type to “rxvt”.

challenge: get cycle-buffer f10 key working

• discussion: The above fix broke one of the keys that works cycle-buffer, f10.
• status: solved! I looked at the rxvt.el file but didn’t see a problem, not that I would recognize a problem if it was in front of my face. I then compared it to the xterm.el file. I decided I would rather go with the smaller and less complicated xterm.el file. So I made a copy of this from /usr/share/emacs/21.2/lisp/term/ and placed it in my ~/emacs-lisp/term directory. I then added key definitions, using the definitions I found in rxvt. With this I now have my long standing key definition problems fixed. Here are the lines I added to my cygwin ~/emacs-lisp/term/xterm.el file:


(define-key function-key-map “\\e[33~" [S-f9])
(define-key function-key-map “\\e[34~" [S-f10])
(define-key function-key-map “\\e[d" [S-left])
(define-key function-key-map “\\e[c" [S-right])
(define-key function-key-map “\\e[a" [S-up])
(define-key function-key-map “\\e[b" [S-down])
(define-key function-key-map “\\e[7~" [home])
(define-key function-key-map “\\e[7^" [C-home])
(define-key function-key-map “\\e[8~" [end])
(define-key function-key-map “\\e[8^" [C-end])


In the process I cleaned up (read “deleted”) my .Xdefaults and .Xresources files - they’re gone. Oh, and windmove.el ROCKS!

Here are some techniques to help you diagnose terminal emulation problems:

resources:

• The Linux keyboard and console HOWTO
• BackSpace and Delete Configuration for Linux - VT, xterm,
  bash, tcsh, netscape and more
• Squishywishywoo: delete and backspace keys under X - emacs, xterm
• Linux Backspace/Delete mini-HOWTO
• The Archive of VIDEO TERMINAL INFORMATION
• useful newgroup note #1

Cygwin is a collection of unix/linux style tools that run on Windows. I primarily use cygwin for xterms, ssh tools (ssh, scp), and rsync, but I sometimes use unix style tools to operate on the window system files.

The first thing you need to know is that cygwin is *not* unix or linux on windows — rather, it is a set of unix/linux tools that run on windows. Don’t expect a windows application to suddenly act like a unix application just because you see a familiar bash prompt.

I’ve used cygwin a long time and it has evolved quite a bit. It is simple and easy to install and use the basic tools. I’ve had trouble though when I’ve tried to run other applications, like mutt.

Here are my recommendations for installing and running cygwin.

Directory Structure

• Create a separate, permanent location for the setup application. I use c:/cygwin-setup. The setup application stores setup configuration information in this directory and things will run smoother if it is there the next time you run setup.
• I accept the default for the cygwin installation directory, c:/cygwin.

Packages

The cygwin setup application is similar in concept to unix/linux package management systems like Debian’s. It provides a windows gui interface listing packages. After you select packages it will download those selected packages and all other packages required for a successful install of the selected packages.

The setup application will also download updates to any already installed packages that have updates. This surprised me the first time I decided to install some new small package and noticed setup was updating everything else I had previously installed. I now have the habit of periodically updating my cygwin installation by running setup and simply clicking through all of the options taking all of the defaults.

I recommend you start small — don’t install everything all at once. When I get a new windows system I start by installing

• openssh (getting me ssh and scp)
• rsync
• x enabled emacs

These items depend on the cygwin core, which includes bash and basic unix file system utilities. The x enabled emacs depends on the x window system so that will be installed too.

Setup will download the appropriate packages and create and populate the directory structure. It will put an icon on the desktop to the cygwin console - which I never use.

I don’t like the cygwin console. Instead I use the xterm. However, as of this writing, I don’t think cygwin has a convenient way to start xterms. This is what I do:

• install the package that gives you x start up scripts (sorry! forgot what its called). This will place in your /usr/X11R6/bin directory a file called startxwin.bat.
• place two copies of startxwin.bat on your desktop — call one ’startxwin.bat’, call the other ’startxterm.bat’.
• edit the version name ’startxterm.bat’. Comment out the ‘run XWin line’.

Now, to run an x application (or to get the first xterm), simply double click on the startxwin.bat. This will start the x-server and 1 xterm. The x-server will continue running even when you kill the xterm window. (The x server adds the x icon to the system tray.) You only need one x-server to run which is why we created the file startxterm.bat.

To start subsequent xterms (once the x-server is running) simply double click on the startxterm.bat file.

Other Suggestions

Cygwin creates unix directory structure in the c:/cygwin directory. When you start an xterm you will be placed in a directory like /home/username. The windows path to that directory is c:/cygwin/home/username.

I used to recommend that you sym link this directory to your windows “My Documents” directory. In the long run I think this is more trouble than it is worth.

I do create sym links to windows “My Documents” and desktop, called simply “documents” and “desktop”. These links make it pretty convenient to find files in the windows file system.

You reference the windows drives via the “cygdrive” directory. For example, to get to c:/, go to /cygdrive/c.

For more info visit http://www.cygwin.com/.