Switching an old test set from webrat to capybara might not have been the smartest of moves, but scoped steps which work are very alluring.

A couple of snags I stumbled upon today, which may be handy to someone, relate to rack-test, sessions, and http request methods.

For a more comprehensive introduction see Tim Riley's post.

This is a quick one.

Cucumber's recent version bump includes a new and improved html report generator (for textmate users, at least) borrowing javascript from rspec. That is to say, when you run your features it will be more like when you run your specs, progress bar and so on. I like that now it shows undefined step snippets in the textmate window in a usable form, that's a timesaver.

The update has changed cucumber's rake tasks to have its @wip settings in your cucumber.yml profile instead of in the rake tasks. This is great, as it's nudged me away from editing the generated rake files and having upgrade pain.

A side-effect of putting these settings into the profiles is that anything other than rake tasks will also use these settings, including the Textmate bundle.

Since the Textmate bundle doesn't understand tags I just gave it a separate profile:

cucumber.yml:

default: --format progress --require features --color --tags ~@wip,~@pending --strict
wip: --format pretty --require features --color --tags ~@pending,@wip:4 --wip
textmate: --require features --format html

TM_ CUCUMBER_OPTS = '--profile textmate'

Which runs all the features you tell it to.

I released my first two real open source projects this week, both Ruby. Kind of nice, never thought it would happen, then it just did.

Vines is a (very small currently) bundle of useful general step definitions for Cucumber, it's not currently rails specific, and I may make rails-specific steps a different include so it can be used for regular ruby projects.

The only step currently available is very simple, but not provided by cucumber. From its own feature tests:

Feature: Should happen steps
  In order to assert errors in my steps
  As a developer
  I want to talk about a step's side effects or consequences

  Scenario: step raises
    Then an error should occur when I eval `raise`

  Scenario: step raises a certain type
    Then an ArgumentError error should occur when I eval `raise ArgumentError`

That's pretty simple stuff. There was and will be more there, but the code and dependencies had too many rough edges for now.

Anachronic went straight to plugin; in integration testing Paperclip I wanted to know whether deleting site images (my model wrapper for a paperclip image attachment) was cleaning up files correctly. I wanted to do it in Cucumber, and I wanted the steps to be simple.

The idea is very simple really:

When I add a site image
Then the images directory should have changed
When I delete the site image
Then the images directory should be back to how it started off

You could, of course, use git for this, but I didn't care about file contents, only whether the files still existed. I think it'd make the tests slow. You may also like to use some combination of ls and diff; have fun with those man pages.

Simple solution -- snapshot file and directory names recursively, then spot the difference between snapshots.

Wrote some specs, wrote some code, specs passed, put into the rails app as a plugin, wrote steps, green feature. Magic.

So, at last, something other than forks on my github page!

It quacks like a duck: it’s a duck.

Cucumber now comes with some rake tasks to help you out on your road to tag-happiness.

rake cucumber:ok will run any scenarios not tagged with @wip

rake cucumber:wip will run any scenarios tagged with @wip and if there are more than 2 of them, or any pass, it will complain. That's not as bad as it sounds, it can be handy info.

I added ~@pending tag to the rake tasks as I wrote some features to sort out later; I don't want them executing and don't want to delete or move them. Why record the ideas in another place, they'll just get lost.

Release versions are one way to handle your tags, but as Ian pointed out, really for most people (and perhaps me) it's excessive. I'd go further to say it's superfluous, clumsy and overly bureaucratic, all in all not very agile. Not bad for something I came up with!

While this sort of thing will take some iterating and is really down to how you arrange your work, the following simple scheme may provide a good foundation.

Tags: @stable

And that's it. Write your features, when they pass mark them as @stable, and when you want to do a regression test run it against all your @stable features.

cucumber --tags @stable -f progress features

You could even rake or alias it for added comfort.

Tags: @stable @wip @pending

In the above an untagged feature can be a work in progress, or left pending for later. While this is enough for regression testing, tagging these features as well has some workflow benefits. It's your metadata, why not use it.

Take for example the feature you quickly sketched out over lunch a week ago to flesh out later, where was it again? If it were tagged you could do a project search (TextMate) or equivalent to find the elusive text. Why waste Time looking when you can Search?™

Tag it your way

One of the nice things about these tags is you can build on this scheme quite easily, by domain, release number, external resource connectivity, or whatever you like.

This is inherently YMMV, and it should be interesting to see what other people find useful.

No, nothing to do with MS Office, promise.

This is a rehash of a conversation I had on the Cucumber mailing list / Google Group

If you are using the Paperclip plugin to handle user image submissions you may run into some head-scratching trouble when testing this with Cucumber. The following approach, while perhaps not ideal, tests the full stack, short of a browser.

The following example uses Machinist for random file selection and could be simplified to a single test image if desired.

Sample Images

Put some images into spec/fixtures directory with a common name, e.g. example1.jpg example2.gif example3.jpg and so on

Steps

My paperclip binding is called picture and it's on my service object. This is in my service steps file:

  def new_picture 
    @new_picture ||= Sham.image 
  end 

  When /^I provide an image file for "picture"$/ do 
    attach_file "service_picture", new_picture.path 
  end

And in my Sham.define block (in spec/blueprints.rb)

image do 
  images = [] 
  dirpath = File.join Rails.root, 'spec', 'fixtures' 
  Dir.new(dirpath).each { |file| images << File.new(File.join (dirpath, file)) if file =~ /^example\d/ } 
  images[rand(images.length)] 
end

Assertions?

Then /^I should see my photo$/ do 
  response.should have_selector("img[src*=''#{File.basename new_picture.path}]) 
end

or

response.body.should =~ /#{File.basename new_picture.path}/

and so on.

Note that paperclip will give you addresses resembling this:

  http://localhost:3000/system/pictures/2/index/example5.jpg?1258339851

so watch out for the secret token after the ? and don't just use == on the file path.

File Hangover

The gotcha is cleaning up the files afterwards, since we're not using the database for a file store. I took this approach (which is hacky but works, on unix) to put the paperclip test files into a different directory and delete the directory afterwards.

features/support.env

#remove paperclip files 
require 'paperclip' #make sure it's loaded, else cue intermittent weirdness 
module Paperclip::Interpolations 
  alias_method :org_attachment, :attachment 
  def attachment(att, style) 
    "CUKE/" + org_attachment(att, style) 
  end 
end 
After do 
  `rm -rf #{"#{RAILS_ROOT}/public/system/CUKE"}` 
end 
#end remove paperclip files

This wants to be an after block so that your database and file system stay in synch through your tests. Using an after all block (or equivalent) would mean the database having no pictures in it, but the directory having image files in it. While that may seem to work ok it's unnecessarily inconsistent.

While my last post highlighted spork as a way to speed up your rspec runs, unfortunately it doesn't currently seem to work out-of-the-box with cucumber. While I am sure this will be resolved in good time there is another approach to speeding up your feature runs & more besides: Tags!

Feature Spaghetti

At some point you will likely find yourself wanting to run a subset of your features rather than all of them, they may be spread across different files and directories, perhaps they're mixed within files but part of a different release. Recently I found myself having written up scenarios from an initial planning meeting which were broken up into 2 reasonably sized releases. The trouble with a straight cucumber features run is the output is a mess. It's cluttered with pending, undefined and even failing steps which you probably aren't interested in yet. Not only does this slow down the run but the terminal spam can be a real hindrance.

But wait, there's --help

cucumber --help, cukes.info and Joseph Wilk's talk at Rails Underground to the rescue!

While undefined stubs are nixed by -i or --no-snippets this absolute; you don't get the snippets you *do* want. Joseph's talk introduced me to cucumber's tagging functionality, which so far I had skirted around and avoided. While it's not yet in the Rspec Book there is a chapter on Cucumber to be added.

Meta Tags ain't just for SEO

In brief, Cucumber tags are really useful.

I broke my features up into domain tags (@services and @admin) and into releases for those written @0_2 @0_3 corresponding to my release plans.

They're easy to add: just add them to the top of your features or individual scenarios, see the wiki. If you're going to add them to an existing project this might be a big task, it can probably be automated. I found copy-paste sufficient for 10-15 feature files. If you know about it when you start out then it's painless as you just do it when you start writing the feature or allocate it to a release/iteration/etc.

Now when I'm working on a services feature I can just run pertinent features with

cucumber --tags @services -f progress features

This will also do the .-UF dance rather than printing out all the features.

Besides this I can test the features release by release, testing the current release goals. There doesn't seem to be much point checking 0.3 features when still working on 0.2.

cucumber --tags @0_2 features

No more fluff or failures for things I probably wrote earlier than I should have done.

But… But… What About The Customer?

The only downside I see to tagging is if you want to send the raw files to your customer for amendment. If you don't want them to see the tags you can try running them through cucumber and piping the result into text, or use cucumber's own file saving functionality. If your steps aren't all green you may have to fiddle with this a bit to get rid of error messages, undefined step stubs, etc.

I did something like this, as it was quicker than finding the right settings:

cucumber --quiet --tags @0_2 features > v0_2-features.txt

To diverge for a moment you're probably only interested in this if you're writing declarative features for the benefit of customer communication. If you're using cucumber purely for integration testing with "imperative" steps then you probably don't want to do this, as their eyes will glaze over. You may like to try some of the other tags shown by --help too, such as --expand for scenario outline tables.

For a comparison of imperative and declarative steps take a look at Ben Mabey's 2008 post. It's old and the technology has changed but the issue is identical. As Ben says, both approaches have their uses.

Forging Ahead

When moving forward with your project running one release at a time will become laborious. You will want to run several versions, and I don't suggest tagging features as @0_2 @0_3 etc, this would get old fast. An alternative is to run several tags at once

cucumber --tags @0_2,@0_3,@0_4 features

This sort of fiddling seems to lend itself to project-specific rake tasks, which I will be experimenting with some time soon.

Actually this doesn't work, it looks for scenarios with both tags, not either (intersection not union), so rake tasks or similar really will be handy for chaining feature runs.

One of the feelings I got from Rails Underground (best not to ask about the others!) is that while RSpec, Cucumber and family have laid tracks in the Rails toolkit and community, many still use TestUnit.

Alternatives are good in some senses, but make me feel like I'm missing out and worry about chasing the wrong rabbit. Seeing things bridged can be nice, and here's an example of that. This one is extra-nice because it also patched what was, to my eyes, a crack in the plaster.

Shoulda for RSpec

I've got some rspec view and controller macros I am considering pulling out into a plugin, but decided to look around for alternatives first. Remarkable is noted on some wiki or other, but I couldn't make much sense of it. rspec_on_rails_macros is a sensible looking alternative, but the origin has been out of development for some time and the forks are somewhat confusing. Looking around did have a positive consequence though -- they pointed out that Shoulda now has rspec support!

That may sound confusing, since Shoulda is another way of writing declarative unit tests (with a similar looking syntax to rspec), but the bonus is a raft of new matchers. So far only the model macros are available but more are promised to come.

Instant gratification:

-    it "should not be valid without a meta description" do
-      Service.make_unsaved(:meta_description => nil).should_not be_valid
-      Service.make_unsaved(:meta_description => '').should_not be_valid
-    end
-    
-    it "should not be valid without meta keywords" do
-      Service.make_unsaved(:meta_keywords => nil).should_not be_valid
-      Service.make_unsaved(:meta_keywords => '').should_not be_valid
-    end
-    
+    it { should validate_presence_of(:name) }
+    it { should validate_presence_of(:teaser) }

I'd Spork That

Next up is Spork, a must-have if you find your specs are running slowly.

You get another executable to add to your workbench, spork, this is a distributed ruby server which uses forking (unix only) to cut down start-up times. Pretty cool I think, get set up to a common starting point then clone it per run.

I find this config works for me, without having to restart the server when I change Machinist blueprints or helper files

require 'rubygems'; require 'spork'

Spork.prefork do
  ENV["RAILS_ENV"] ||= 'test'
  require File.dirname(__FILE__) + "/../config/environment"
  require 'spec/autorun'
  require 'spec/rails'
  require 'shoulda'
  require 'webrat'

  Dir[File.dirname(__FILE__) + "/spec_helpers/**/*.rb"].each do |file| 
    require file 
  end

  # Requires supporting files with custom matchers and macros, etc,
  # in ./support/ and its subdirectories.
  Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}

  Spec::Runner.configure do |config|
    config.use_transactional_fixtures = true
    config.use_instantiated_fixtures  = false
    config.fixture_path = RAILS_ROOT + '/spec/fixtures/'

    config.include Webrat::Matchers, :type => :views

    #machinist setup
    config.before(:all)    { Sham.reset(:before_all)  }
    config.before(:each)   { Sham.reset(:before_each) }
    #end machinist setup
  end
end

Spork.each_run do
  # This code will be run each time you run your specs.
  require File.expand_path(File.dirname(__FILE__) + "/blueprints")

  Spec::Runner.configure do |config|
    config.include ViewHelpers, :type => :views
    config.include ControllerHelpers, :type => :controller
    config.extend ControllerMacros, :type => :controller
  end
end

I went the route of adding --drb to the spec.opts file. This works with textmate and autospec. If you forget to run spork it'll run locally without the drb.

If you want something more (or actually) distributed for your tests take a look at testjour, or even some of the cloud computing services (which I've forgotten the names of!).

Authlogic is a handy plugin for shifting your user authentication, big or small, into an external dependency. Having it in gem (or submodule/plugin) form rather than generated code means as updates become available they are easily installed, since the codebase is separated from your own.

On a current green-field project I've switched from a home-rolled simple admin authentication solution to Authlogic. This took some fiddling as I don't get on well with Authlogic's style of documentation, but I expect that will improve with time and blogs.

I'm hesitant to give out security advice, but found the following useful. I kept hold of as many of my specs and features as was reasonable with the authlogic switch, while the hashing was now out of my hands there were still important criteria for how I wanted usernames, passwords and so on to behave at each MVC layer. In particular I don't want the attempted password to be sent back to the user as a field value when they fail to log in. While SSL offsets this somewhat I would rather have that data submitted once and not sent back. Also I don't see a use for the unencrypted password other than during validation and hashing at update or login.

A very simple fix for the view concern, found on slideshare.net is to set the view template value to an empty string.

<%= f.password_field :password, :value => "" %>

That's fine, but it's hardly in the MVC spirit. The model shouldn't be letting anything have the raw password. For example if you start adding more access formats you'd need to worry about it there. I think it's best to cut it off at source, with something like the following ActiveRecord hook in the model which acts_as_authentic

def after_save
  @password = nil
  @password_confirmation = nil
end

There is probably a better way to do this, but between that and the view value blanking my specs and features pass. If you're doing a lot before saving or leaving an unsaved model floating around bear this in mind.

Thoughts, comments & criticisms welcome.

While adding Textile to a resource following the outside-in BDD process described in the RSpec Book I found myself wanting to put a should_receive expectation on ActionView::Helpers::TextHelper#textilize

This left me somewhat confused, as I wasn't sure what to stub, and the book doesn't cover this. Google also drew a blank.

sticking a <%= inspect %> in the view and looking at the page source revealed an instance of ActionView::Base, which makes sense but was somewhat elusive. It also showed a @controller reference, which itself contains a @template reference. A consequence of this is that in your view spec you can get at the template with @controller.template

It turns out that within a view spec #template provides the relevant object. Thanks to Tim Riley for pointing this out.

For example:

it "should textalize the teaser and extra_content" do
  @service = mock_model(Service, :null_object => true)
  @service.stub!(:teaser).and_return 'a teaser'

  template.stub!(:textilize).and_return "textalised teaser"
  template.should_receive(:textilize).with @service.teaser

  render
  response.should contain("textalised teaser")
end

This kind of stubbing is useful as it checks that the view is calling the textilize method, part of the api, correctly, not whether RedCloth (or some other texitle gem) is doing its thing. Stubbing like this will, if nothing else, speed up the spec execution. Testing with input/output examples for the textilized data could go in the view spec, but api testing at that level seems out of place to me. Of course it should be tested at some level, e.g. with Cucumber. YMMV.

Well Uni is over now for me (ceremony pending) and life out of the city is a lot quieter.

I've decided to keep busy with a few events coming up this year, two rails specific and one on more general web design topics:

Rails Camp UK:

FOWD Tour - Leeds:

Take a look, there may be something you fancy.

I'm currently in the process of setting up a Rails 2.3 stack from scratch, and a few things have changed since the last couple of projects I worked on were tooled up.

One thing which has certainly changed for better is binding rspec, ZenTest and Growl together, previously I posted on some work-arounds for missing messages, adding images, and various other bits ad-hoc. This functionality is now all produced by installing the autotest-growl gem which may also be found on github. Take a look at the readme.rdoc

Of interest to me in particular were changes to ~/.autotest

Mine was quite something previously,

#!/usr/bin/env ruby
# Symlink this to ~/.autotest
require 'autotest/redgreen'
require 'autotest/fsevent'

AUTOTEST_IMAGE_PATH = File.dirname(File.symlink?(__FILE__) ? File.readlink(__FILE__) : File.expand_path(__FILE__))

module Autotest::Growl
  def self.growl title, msg, img, pri=0, stick=""
    system "growlnotify -n autotest --image #{img.inspect} -p #{pri} -m #{msg.inspect} #{title.inspect} #{stick}"
  end

  Autotest.add_hook :ran_command do |autotest|
    filtered = autotest.results.grep(/\d+\s.*examples?/)
    output = filtered.empty? ? "" : filtered.last.slice(/(\d+)\s.*examples?,\s(\d+)\s.*failures?(?:,\s(\d+)\s.*pending)?/)
    if output =~ /[1-9]\sfailures?/
      growl "Test Results", "#{output}", "#{AUTOTEST_IMAGE_PATH}/fail.jpg"
    elsif output =~ /pending/
      growl "Test Results", "#{output}", "#{AUTOTEST_IMAGE_PATH}/pending.jpg"
    else
      growl "Test Results", "#{output}", "#{AUTOTEST_IMAGE_PATH}/ok.jpg"
    end
  end
end

Autotest.add_hook :initialize do |autotest|
  %w{.git .svn .hg .DS_Store ._* vendor}.each {|exception| autotest.add_exception(exception) }
  false

now replaced with

require 'autotest/growl'
require 'autotest/fsevent' #osx specific file changed event notification
Autotest::Growl::show_modified_files = true #which changes prompted the autospec run
Autotest::Growl::remote_notification = true #networked growl, to work-around disappearing notifications
Autotest.add_hook :initialize do |at|
  %w{.git .svn .hg .DS_Store ._* log}.each {|exception|at.add_exception(exception)}
end

The FSEvent gem is well worth a look if you develop on OSX 10.5 (Leopard), it switches autotest from polling your hard drive (i.e. thrashing) to working with the OS's event notification system. Design patterns strike again!

With these updates I gave an old project a spin to see what would happen, and voila, the specs ran as they should, coloured and all.

A particular error I was receiving before doing this update was

script/autospec 
(Not running features.  To run features in autotest, set AUTOFEATURE=true.)
(Not running features.  To run features in autotest, set AUTOFEATURE=true.)
loading autotest/rails_rspec
/Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- autotest/redgreen (MissingSourceFile)
    from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `require'
  …
    from /usr/bin/autotest:19:in `load'
    from /usr/bin/autotest:19
Unable to find autotest.  Please install ZenTest or fix your PATH

The culprit being redgreen, which can be uninstalled when using the new ZenTest.

The DNS Problem

If you own an Apple Airport device or Timecapsule you may have come across the issue of it (unlike similar products from DLink and other specialists) not having an inbuilt DNS service hooked up to the DHCP requests (i.e. being able to resolve local computer names to lan ips the same way you might resolve a website to an ip address, by DNS).

This annoyed me greatly yesterday. Today Ray reminded me of Bonjour, Zeroconf, whatever you want to call it. This tech reminds me of the old windows networking protocol. It may be completely different, but it's amusing to see an abhorred concept coming back again.

Problems have solutions you know

Things worked one way out of the box (8.10), I was able to use ssh hostname.local from Ubuntu to my Mac right off. This is nice as with a default and a public key exchange this is a seamless remote login process.

The other way was not working, but not far out of reach. On my install avahi-demon was already present with the zeroconf plugin, both configured and running, it just didn't have anything to shout about.

This forum thread is recent and works, which is more than can be said for the Ubuntu documentation page for this particular subject.

A template ssh broadcast file is provided with the avahi install. Template so far as you can just copy-paste it into position and ssh will start being broadcast on the network.

sudo cp /usr/share/doc/avahi-daemon/examples/ssh.service /etc/avahi/services

sudo /etc/init.d/avahi-daemon restart

Service Browsing Apps

iStumbler comes with a Bonjour plugin, so you can use it to monitor what's available on your network visually.

For Gnome there is an application "Service Discovery Applet" available via Synaptic (which you can then add by right clicking the menu bar/panel, add, browse down to zeroconf discovery).

For my dissertation I'm working on a genetic algorithm solution to the guitar fingering problem. It's got just under 2 months to run now, and it's time to start wrapping it up.

Part of the process is getting results and evaluating them. I have running code now so it's time to start putting it through its paces.

The department provides students with access to a 'grid' computing setup, which is basically a batch processing system you access by ssh.

Unfortunately it's not a parallel architecture, but not having to play with threads is probably a good thing for me as they aren't something I am familiar with. The task I am addressing is very parallel, a favourite analogy of mine being natural selection occurring on a number of islands and the fittest individuals occasionally migrating between islands. Batch processing is sufficient to get similar effects, and I'll be toying with some ideas for that over the next week or two.

I thought I'd share some scripts and terminal output since they are pretty short and it's not something most people come across.

Scripting your application

Getting things running is deceptively simple. Make a script to run your program, I'm hoping to use Ruby later but went with (Ba)sh for now.

bash-3.2$ cat ./run_fingar
cd fingar
java -Xmx2g nruth.fingar.Run pop=400000 gens=500

The JRE arg is to set the maximum heap size to 2GB, rather than the default 128MB (though it should auto-configure itself to be higher because of something called Ergonomics that went into Java 5). Yes I do need that much memory, it's that kind of application. The grid notes have 3GB available so they can handle this, which is a bonus as it means I can increase population sizes greatly and see how they compare to smaller ones, and various other settings.

I/O

This is trivial really, write an app that uses Stdio and Stderr and when you run the job they will get piped into a specified log file. Mine just sit in the home directory for now. I will be changing that (with ruby, along with the rest of the statistical analysis scripting) to name by date, run parameters, code git revision, etc.

Submitting your job

Once I ssh into the distribution mechanism/scheduler I get presented with the usual home directory and terminal for my network account. There is a script to run which bolts on the grid commands I need, so I've made a wrapper for this & executing the above script to run my code. It's not ideal as I have to do this again to use qstat and so on, but that's just terminal script hackery and I am sure there is a work around (for example I have another script using source to load the settings into bash without having to look up the path every time).

bash-3.2$ cat ./run_fingar_et
#!/bin/bash
. /data/sungrid/default/common/settings.sh
qsub -cwd -l qp=LOW -P Basic -j y -o et_log run_fingar

qsub is the scheduling script for the grid, the parameters mean something along the lines of use the current working directory, queue priority low, basic profile (i.e. plebs not academics), pipe stdio stderr to et_log, and finally the command to run.

Watching and waiting

Once you submit there is a command qstat which will show you all the jobs you have pending, which is handy, as you know whether the results are ready or not.

bash-3.2$ ./run_fingar_et
Your job 7484 ("run_fingar") has been submitted
bash-3.2$ qstat
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 
-----------------------------------------------------------------------------------------------------------------
   7484 0.00000 run_fingar   **ntr       qw    03/24/2009 23:08:29                                    1        
bash-3.2$ qstat
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 
-----------------------------------------------------------------------------------------------------------------
   7484 0.56000 run_fingar   **ntr       r     03/24/2009 23:08:43 LOW.q@car2.expresstrain.dcs        1

I'm currently waiting for a more ambitious run to complete,

SimpleHandPositionModelGAEvolver: 
population size: 400000
locus crossover likelihood: 0.16
allele mutation likelihood: 0.02
generation 1 of 500

I started running into memory issues at 100k with the JRE defaults, so it'll be interesting to see how this goes. It's been running for an hour so far…