This is a quick one.

Cucumber's recent version bump includes a new and improved html report generator (for textmate users, at least) borrowing javascript from rspec. That is to say, when you run your features it will be more like when you run your specs, progress bar and so on. I like that now it shows undefined step snippets in the textmate window in a usable form, that's a timesaver.

The update has changed cucumber's rake tasks to have its @wip settings in your cucumber.yml profile instead of in the rake tasks. This is great, as it's nudged me away from editing the generated rake files and having upgrade pain.

A side-effect of putting these settings into the profiles is that anything other than rake tasks will also use these settings, including the Textmate bundle.

Since the Textmate bundle doesn't understand tags I just gave it a separate profile:

cucumber.yml:

default: --format progress --require features --color --tags ~@wip,~@pending --strict
wip: --format pretty --require features --color --tags ~@pending,@wip:4 --wip
textmate: --require features --format html

TM_ CUCUMBER_OPTS = '--profile textmate'

Which runs all the features you tell it to.

I released my first two real open source projects this week, both Ruby. Kind of nice, never thought it would happen, then it just did.

Vines is a (very small currently) bundle of useful general step definitions for Cucumber, it's not currently rails specific, and I may make rails-specific steps a different include so it can be used for regular ruby projects.

The only step currently available is very simple, but not provided by cucumber. From its own feature tests:

Feature: Should happen steps
  In order to assert errors in my steps
  As a developer
  I want to talk about a step's side effects or consequences

  Scenario: step raises
    Then an error should occur when I eval `raise`

  Scenario: step raises a certain type
    Then an ArgumentError error should occur when I eval `raise ArgumentError`

That's pretty simple stuff. There was and will be more there, but the code and dependencies had too many rough edges for now.

Anachronic went straight to plugin; in integration testing Paperclip I wanted to know whether deleting site images (my model wrapper for a paperclip image attachment) was cleaning up files correctly. I wanted to do it in Cucumber, and I wanted the steps to be simple.

The idea is very simple really:

When I add a site image
Then the images directory should have changed
When I delete the site image
Then the images directory should be back to how it started off

You could, of course, use git for this, but I didn't care about file contents, only whether the files still existed. I think it'd make the tests slow. You may also like to use some combination of ls and diff; have fun with those man pages.

Simple solution -- snapshot file and directory names recursively, then spot the difference between snapshots.

Wrote some specs, wrote some code, specs passed, put into the rails app as a plugin, wrote steps, green feature. Magic.

So, at last, something other than forks on my github page!

Release versions are one way to handle your tags, but as Ian pointed out, really for most people (and perhaps me) it's excessive. I'd go further to say it's superfluous, clumsy and overly bureaucratic, all in all not very agile. Not bad for something I came up with!

While this sort of thing will take some iterating and is really down to how you arrange your work, the following simple scheme may provide a good foundation.

Tags: @stable

And that's it. Write your features, when they pass mark them as @stable, and when you want to do a regression test run it against all your @stable features.

cucumber --tags @stable -f progress features

You could even rake or alias it for added comfort.

Tags: @stable @wip @pending

In the above an untagged feature can be a work in progress, or left pending for later. While this is enough for regression testing, tagging these features as well has some workflow benefits. It's your metadata, why not use it.

Take for example the feature you quickly sketched out over lunch a week ago to flesh out later, where was it again? If it were tagged you could do a project search (TextMate) or equivalent to find the elusive text. Why waste Time looking when you can Search?™

Tag it your way

One of the nice things about these tags is you can build on this scheme quite easily, by domain, release number, external resource connectivity, or whatever you like.

This is inherently YMMV, and it should be interesting to see what other people find useful.

No, nothing to do with MS Office, promise.

This is a rehash of a conversation I had on the Cucumber mailing list / Google Group

If you are using the Paperclip plugin to handle user image submissions you may run into some head-scratching trouble when testing this with Cucumber. The following approach, while perhaps not ideal, tests the full stack, short of a browser.

The following example uses Machinist for random file selection and could be simplified to a single test image if desired.

Sample Images

Put some images into spec/fixtures directory with a common name, e.g. example1.jpg example2.gif example3.jpg and so on

Steps

My paperclip binding is called picture and it's on my service object. This is in my service steps file:

  def new_picture 
    @new_picture ||= Sham.image 
  end 

  When /^I provide an image file for "picture"$/ do 
    attach_file "service_picture", new_picture.path 
  end

And in my Sham.define block (in spec/blueprints.rb)

image do 
  images = [] 
  dirpath = File.join Rails.root, 'spec', 'fixtures' 
  Dir.new(dirpath).each { |file| images << File.new(File.join (dirpath, file)) if file =~ /^example\d/ } 
  images[rand(images.length)] 
end

Assertions?

Then /^I should see my photo$/ do 
  response.should have_selector("img[src*=''#{File.basename new_picture.path}]) 
end

or

response.body.should =~ /#{File.basename new_picture.path}/

and so on.

Note that paperclip will give you addresses resembling this:

  http://localhost:3000/system/pictures/2/index/example5.jpg?1258339851

so watch out for the secret token after the ? and don't just use == on the file path.

File Hangover

The gotcha is cleaning up the files afterwards, since we're not using the database for a file store. I took this approach (which is hacky but works, on unix) to put the paperclip test files into a different directory and delete the directory afterwards.

features/support.env

#remove paperclip files 
require 'paperclip' #make sure it's loaded, else cue intermittent weirdness 
module Paperclip::Interpolations 
  alias_method :org_attachment, :attachment 
  def attachment(att, style) 
    "CUKE/" + org_attachment(att, style) 
  end 
end 
After do 
  `rm -rf #{"#{RAILS_ROOT}/public/system/CUKE"}` 
end 
#end remove paperclip files

This wants to be an after block so that your database and file system stay in synch through your tests. Using an after all block (or equivalent) would mean the database having no pictures in it, but the directory having image files in it. While that may seem to work ok it's unnecessarily inconsistent.

While my last post highlighted spork as a way to speed up your rspec runs, unfortunately it doesn't currently seem to work out-of-the-box with cucumber. While I am sure this will be resolved in good time there is another approach to speeding up your feature runs & more besides: Tags!

Feature Spaghetti

At some point you will likely find yourself wanting to run a subset of your features rather than all of them, they may be spread across different files and directories, perhaps they're mixed within files but part of a different release. Recently I found myself having written up scenarios from an initial planning meeting which were broken up into 2 reasonably sized releases. The trouble with a straight cucumber features run is the output is a mess. It's cluttered with pending, undefined and even failing steps which you probably aren't interested in yet. Not only does this slow down the run but the terminal spam can be a real hindrance.

But wait, there's --help

cucumber --help, cukes.info and Joseph Wilk's talk at Rails Underground to the rescue!

While undefined stubs are nixed by -i or --no-snippets this absolute; you don't get the snippets you *do* want. Joseph's talk introduced me to cucumber's tagging functionality, which so far I had skirted around and avoided. While it's not yet in the Rspec Book there is a chapter on Cucumber to be added.

Meta Tags ain't just for SEO

In brief, Cucumber tags are really useful.

I broke my features up into domain tags (@services and @admin) and into releases for those written @0_2 @0_3 corresponding to my release plans.

They're easy to add: just add them to the top of your features or individual scenarios, see the wiki. If you're going to add them to an existing project this might be a big task, it can probably be automated. I found copy-paste sufficient for 10-15 feature files. If you know about it when you start out then it's painless as you just do it when you start writing the feature or allocate it to a release/iteration/etc.

Now when I'm working on a services feature I can just run pertinent features with

cucumber --tags @services -f progress features

This will also do the .-UF dance rather than printing out all the features.

Besides this I can test the features release by release, testing the current release goals. There doesn't seem to be much point checking 0.3 features when still working on 0.2.

cucumber --tags @0_2 features

No more fluff or failures for things I probably wrote earlier than I should have done.

But… But… What About The Customer?

The only downside I see to tagging is if you want to send the raw files to your customer for amendment. If you don't want them to see the tags you can try running them through cucumber and piping the result into text, or use cucumber's own file saving functionality. If your steps aren't all green you may have to fiddle with this a bit to get rid of error messages, undefined step stubs, etc.

I did something like this, as it was quicker than finding the right settings:

cucumber --quiet --tags @0_2 features > v0_2-features.txt

To diverge for a moment you're probably only interested in this if you're writing declarative features for the benefit of customer communication. If you're using cucumber purely for integration testing with "imperative" steps then you probably don't want to do this, as their eyes will glaze over. You may like to try some of the other tags shown by --help too, such as --expand for scenario outline tables.

For a comparison of imperative and declarative steps take a look at Ben Mabey's 2008 post. It's old and the technology has changed but the issue is identical. As Ben says, both approaches have their uses.

Forging Ahead

When moving forward with your project running one release at a time will become laborious. You will want to run several versions, and I don't suggest tagging features as @0_2 @0_3 etc, this would get old fast. An alternative is to run several tags at once

cucumber --tags @0_2,@0_3,@0_4 features

This sort of fiddling seems to lend itself to project-specific rake tasks, which I will be experimenting with some time soon.

Actually this doesn't work, it looks for scenarios with both tags, not either (intersection not union), so rake tasks or similar really will be handy for chaining feature runs.

One of the feelings I got from Rails Underground (best not to ask about the others!) is that while RSpec, Cucumber and family have laid tracks in the Rails toolkit and community, many still use TestUnit.

Alternatives are good in some senses, but make me feel like I'm missing out and worry about chasing the wrong rabbit. Seeing things bridged can be nice, and here's an example of that. This one is extra-nice because it also patched what was, to my eyes, a crack in the plaster.

Shoulda for RSpec

I've got some rspec view and controller macros I am considering pulling out into a plugin, but decided to look around for alternatives first. Remarkable is noted on some wiki or other, but I couldn't make much sense of it. rspec_on_rails_macros is a sensible looking alternative, but the origin has been out of development for some time and the forks are somewhat confusing. Looking around did have a positive consequence though -- they pointed out that Shoulda now has rspec support!

That may sound confusing, since Shoulda is another way of writing declarative unit tests (with a similar looking syntax to rspec), but the bonus is a raft of new matchers. So far only the model macros are available but more are promised to come.

Instant gratification:

-    it "should not be valid without a meta description" do
-      Service.make_unsaved(:meta_description => nil).should_not be_valid
-      Service.make_unsaved(:meta_description => '').should_not be_valid
-    end
-    
-    it "should not be valid without meta keywords" do
-      Service.make_unsaved(:meta_keywords => nil).should_not be_valid
-      Service.make_unsaved(:meta_keywords => '').should_not be_valid
-    end
-    
+    it { should validate_presence_of(:name) }
+    it { should validate_presence_of(:teaser) }

I'd Spork That

Next up is Spork, a must-have if you find your specs are running slowly.

You get another executable to add to your workbench, spork, this is a distributed ruby server which uses forking (unix only) to cut down start-up times. Pretty cool I think, get set up to a common starting point then clone it per run.

I find this config works for me, without having to restart the server when I change Machinist blueprints or helper files

require 'rubygems'; require 'spork'

Spork.prefork do
  ENV["RAILS_ENV"] ||= 'test'
  require File.dirname(__FILE__) + "/../config/environment"
  require 'spec/autorun'
  require 'spec/rails'
  require 'shoulda'
  require 'webrat'

  Dir[File.dirname(__FILE__) + "/spec_helpers/**/*.rb"].each do |file| 
    require file 
  end

  # Requires supporting files with custom matchers and macros, etc,
  # in ./support/ and its subdirectories.
  Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}

  Spec::Runner.configure do |config|
    config.use_transactional_fixtures = true
    config.use_instantiated_fixtures  = false
    config.fixture_path = RAILS_ROOT + '/spec/fixtures/'

    config.include Webrat::Matchers, :type => :views

    #machinist setup
    config.before(:all)    { Sham.reset(:before_all)  }
    config.before(:each)   { Sham.reset(:before_each) }
    #end machinist setup
  end
end

Spork.each_run do
  # This code will be run each time you run your specs.
  require File.expand_path(File.dirname(__FILE__) + "/blueprints")

  Spec::Runner.configure do |config|
    config.include ViewHelpers, :type => :views
    config.include ControllerHelpers, :type => :controller
    config.extend ControllerMacros, :type => :controller
  end
end

I went the route of adding --drb to the spec.opts file. This works with textmate and autospec. If you forget to run spork it'll run locally without the drb.

If you want something more (or actually) distributed for your tests take a look at testjour, or even some of the cloud computing services (which I've forgotten the names of!).

Authlogic is a handy plugin for shifting your user authentication, big or small, into an external dependency. Having it in gem (or submodule/plugin) form rather than generated code means as updates become available they are easily installed, since the codebase is separated from your own.

On a current green-field project I've switched from a home-rolled simple admin authentication solution to Authlogic. This took some fiddling as I don't get on well with Authlogic's style of documentation, but I expect that will improve with time and blogs.

I'm hesitant to give out security advice, but found the following useful. I kept hold of as many of my specs and features as was reasonable with the authlogic switch, while the hashing was now out of my hands there were still important criteria for how I wanted usernames, passwords and so on to behave at each MVC layer. In particular I don't want the attempted password to be sent back to the user as a field value when they fail to log in. While SSL offsets this somewhat I would rather have that data submitted once and not sent back. Also I don't see a use for the unencrypted password other than during validation and hashing at update or login.

A very simple fix for the view concern, found on slideshare.net is to set the view template value to an empty string.

<%= f.password_field :password, :value => "" %>

That's fine, but it's hardly in the MVC spirit. The model shouldn't be letting anything have the raw password. For example if you start adding more access formats you'd need to worry about it there. I think it's best to cut it off at source, with something like the following ActiveRecord hook in the model which acts_as_authentic

def after_save
  @password = nil
  @password_confirmation = nil
end

There is probably a better way to do this, but between that and the view value blanking my specs and features pass. If you're doing a lot before saving or leaving an unsaved model floating around bear this in mind.

Thoughts, comments & criticisms welcome.

I'm currently in the process of setting up a Rails 2.3 stack from scratch, and a few things have changed since the last couple of projects I worked on were tooled up.

One thing which has certainly changed for better is binding rspec, ZenTest and Growl together, previously I posted on some work-arounds for missing messages, adding images, and various other bits ad-hoc. This functionality is now all produced by installing the autotest-growl gem which may also be found on github. Take a look at the readme.rdoc

Of interest to me in particular were changes to ~/.autotest

Mine was quite something previously,

#!/usr/bin/env ruby
# Symlink this to ~/.autotest
require 'autotest/redgreen'
require 'autotest/fsevent'

AUTOTEST_IMAGE_PATH = File.dirname(File.symlink?(__FILE__) ? File.readlink(__FILE__) : File.expand_path(__FILE__))

module Autotest::Growl
  def self.growl title, msg, img, pri=0, stick=""
    system "growlnotify -n autotest --image #{img.inspect} -p #{pri} -m #{msg.inspect} #{title.inspect} #{stick}"
  end

  Autotest.add_hook :ran_command do |autotest|
    filtered = autotest.results.grep(/\d+\s.*examples?/)
    output = filtered.empty? ? "" : filtered.last.slice(/(\d+)\s.*examples?,\s(\d+)\s.*failures?(?:,\s(\d+)\s.*pending)?/)
    if output =~ /[1-9]\sfailures?/
      growl "Test Results", "#{output}", "#{AUTOTEST_IMAGE_PATH}/fail.jpg"
    elsif output =~ /pending/
      growl "Test Results", "#{output}", "#{AUTOTEST_IMAGE_PATH}/pending.jpg"
    else
      growl "Test Results", "#{output}", "#{AUTOTEST_IMAGE_PATH}/ok.jpg"
    end
  end
end

Autotest.add_hook :initialize do |autotest|
  %w{.git .svn .hg .DS_Store ._* vendor}.each {|exception| autotest.add_exception(exception) }
  false

now replaced with

require 'autotest/growl'
require 'autotest/fsevent' #osx specific file changed event notification
Autotest::Growl::show_modified_files = true #which changes prompted the autospec run
Autotest::Growl::remote_notification = true #networked growl, to work-around disappearing notifications
Autotest.add_hook :initialize do |at|
  %w{.git .svn .hg .DS_Store ._* log}.each {|exception|at.add_exception(exception)}
end

The FSEvent gem is well worth a look if you develop on OSX 10.5 (Leopard), it switches autotest from polling your hard drive (i.e. thrashing) to working with the OS's event notification system. Design patterns strike again!

With these updates I gave an old project a spin to see what would happen, and voila, the specs ran as they should, coloured and all.

A particular error I was receiving before doing this update was

script/autospec 
(Not running features.  To run features in autotest, set AUTOFEATURE=true.)
(Not running features.  To run features in autotest, set AUTOFEATURE=true.)
loading autotest/rails_rspec
/Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- autotest/redgreen (MissingSourceFile)
    from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `require'
  …
    from /usr/bin/autotest:19:in `load'
    from /usr/bin/autotest:19
Unable to find autotest.  Please install ZenTest or fix your PATH

The culprit being redgreen, which can be uninstalled when using the new ZenTest.

Those of you who use github and have a vinyl or octopus fetish may find these of interest:

I'm hoping to get one for my laptop. Best be quick as they stop taking requests on the 11th of March!

Well not exactly. Further to my previous attempt at making a workbench launcher I found that rubyosa is not the only way to interact with osx applications through ruby. rb-appscript is an alternative derived from appscript, which boasts 5 years production use amongst other things.

Necessity may well be the mother of all creation but I have oft found laziness and irritation to be close relations. As a new user I'm still settling into a Mac and Rails working environment. I'm using TextMate and Terminal.app with several tabs in the same window.

I soon grew tired of tapping this kind of sequence into my macbook:

gem server
⌘T
cd dev/workspace/projectname
script/server
⌘T
cd dev/workspace/projectname
script/autospec
⌘T
cd dev/workspace/projectname
mate .

It seemed sensible that there would be some way to script this. In the end Applescript turned out to be the way forward - the problem was too osx-centric to do with a bash script in any way I could think of or find.

The first WIN for us to come out of the Rails/Merb merge is that the awesome provides API in merb (which Dan gave a compelling introduction to at RailsCamp4), has made its way to rails-3 (along with some tweaks to the API).

This is a win for us because our response_for plugin provides similar functionality (for rails 2 apps). So that's one less plugin for me to maintain.

Can't wait for RARB!

I'm upgrading one of our apps to rails 2.2 and along with that, some plugin dependencies.

In this instance, I needed to monkey patch a method in attachment_fu to make it use the new i18n framework in Rails.

I got to thinking that it would be great if I could be reminded when this monkey patch is no longer necessary (ie. if I upgrade attachmnet_fu, and if it has made that method compatible with i18n). This was easily achieved, using rspec's pending.

pending: it's alive!

(you may want to skip straight to the pending monkey-patch stuff)

You may not know that it you pass a block to pending, it will still run that block, but it will expect it to fail. If the block passes, then rspec will tell you that the example is now working.

Here's an example: Nick is expanding on some specs for some code that I wrote.

it "fred@fang should not be a valid email" do
  @user.email = 'fred@fang'
  @user.should have(1).error_on(:email)
end

The above spec fails because currently, I don't have a complete email regexp. For whatever reason, I tell Nick that I'll look into that at some later point. What do we do next? We could open a ticket, or write a TODO, but what about that spec?

Using pending, we can add some live doco to our app:

it "fred@fang should not be a valid email" do
  pending "a better email regexp" do
    @user.email = 'fred@fang'
    @user.should have(1).error_on(:email)
  end
end

The above code will be shown as PENDING as long as the example fails. When someone gets around to writing a better regexp, the code will pass, and the rspec will fail the build telling you that the example is no longer pending.

Use pending to monitor your monkey-patches

So, we can use this neat feature of rspec to tell us when a monkey-patch is no longer required:

• Write a spec that exhibits the desired behaviour
• This spec should fail
• Wrap the example(s) in pending block(s)
• (optional, as your app specs might already test this) Write a spec that passes on the monkey-patched code.

In the example given above, all I had to do was spec out the behaviour that attachment_fu should exhibit, then wrap that in an informative pending block.

require File.dirname(__FILE__) + '/../spec_helper'

describe "attachment_fu and i18n" do
  describe "with an invalid attachment" do
    before do
      @model = mock('an attachment_fu model')
      @model.extend Technoweenie::AttachmentFu::InstanceMethods
      @model.stub!(:errors).and_return(mock('errors', :add => nil))
    
      # setup an invalid condition on the model
      @model.stub!(:attachment_options).and_return(:size => (2..3))
      @model.stub!(:size).and_return(1)
    end

    it "#attachment_attributes_valid? should not call default_error_messages" do
      pending "if this passes remove Asset#attachment_attributes_valid?" do
        ActiveRecord::Errors.should_not_receive(:default_error_messages)
        @model.send :attachment_attributes_valid?
      end
    end
  end
end

This means that if I upgrade to a version of attachment_fu that fixes the above problem, my specs will tell me my monkey-patch is no longer required.

One of the projects to come out of RailsCamp 4 (photos) (well apparently it started on the flight from Birsbane) is Dr Nic's super awesome autocompletions - tabtab.

It's basically context sensitive autocompletion for your ruby (and shell) world (and you can add your own completions). Think git-bash-autocompletion - for everything! To get it installed:

sudo gem install tabtab
install_tabtab

Finally, add source ~/.tabtab.bash to the bottom of your ~/.profile

Here's an example of its use - using the github gem to checkout forks of one of my projects.

Adding your own completions

tabtab has a snazy ruby dsl to add completions. I tried adding them for the CI tool garlic and it took a whole minute!.

This was due to the fact you tabtab will parse your help file (if you tell it to) for the command line options, and the fact that I already had an abstraction in garlic to introspect the command-line/rake-tasks. If you don't have the latter it would take maybe 2 whole minutes.

After I pushed garlic, and on reinstall, I just had to run install_tabtab to have the garlic tabtab def automagically appear in my ~/.tabtab.bash.

Check out the comprehensive README. Thanks Dr Nic!

GitHub has a useful page on how to install gems directly from github.

A recent OS X example is

$ sudo gem install spicycode-rcov
Building native extensions.  This could take a while...
Successfully installed spicycode-rcov-0.8.1.5.0
1 gem installed
Installing ri documentation for spicycode-rcov-0.8.1.5.0...
Installing RDoc documentation for spicycode-rcov-0.8.1.5.0...