Red Green Repeat Adventures of a Spec Driven Junkie

Testing 'hello world' with Aruba

Ever wonder: how can I test a ‘hello world’ program?

I do. Every time I look at old code I wrote which doesn’t have tests, I ask myself: “Why didn’t I start testing earlier?!” One reason I have come up with: testing ‘hello world’ is really tough.

Code

All code for this article is available at:

https://github.com/a-leung/aruba_starter

Testing ‘puts’ in RSpec

The main reason testing hello world is tough is in ruby, puts is always used in the program and only recently I was able to figure out how to test ‘puts’ in RSpec, and it was a feature of RSpec3:

expect{hello_world}.to output("hello world").to_stdout

That’s a pretty advanced line of code in RSpec, even for me who has been working in specs for a few years. But that doesn’t really solve the ‘hello world’ program problem, because to make the above test pass, one solution would be:

1
2
3
4

# hello_world_method.rb
def hello_world
  puts 'hello world'
end

BUT, every hello world program in Ruby will always be:

# hello_world.rb
puts 'hello world'

No more, no less. So, how to test THAT?

I have found one solution: Aruba.

Aruba

This is a ruby gem for acceptance testing, Cucumber, which tests at a higher level than RSpec, but Aruba works for all major Ruby test frameworks: cucumber, RSpec, mini test.

More info can be found at Aruba’s homepage

Setup Aruba for RSpec testing

Project Setup

requirements:

  • Ruby 2.1.5
  • RSpec 3.5.4

note: A newer version of Ruby will also work. I prefer Ruby 2.1.5 as some native extensions Aruba uses haven’t fully been ported to the current version of Ruby.

Install Ruby

To install ruby, I recommend using rvm and running command:

u@system:~/aruba-starter$ rvm install ruby-2.1.5

Install RSpec

u@system:~/aruba-starter$ gem install rspec

Install Aruba

u@system:~/aruba-starter$ gem install aruba

Setup RSpec

u@system:~/aruba-starter$ rspec --init

Setup Aruba

u@system:~/aruba-starter$ aruba init --test-framework rspec

config spec_helper.rb file

The spec_helper.rb has a lot of boilerplate, but the most important lines to add are:

require 'aruba'

and in the RSpec.configure block, include line:

config.include Aruba::Api

spec_helper.rb

The spec_helper.rb file should look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

require 'aruba'

# default rspec config
RSpec.configure do |config|
  config.expect_with :rspec do |expectations|
    expectations.include_chain_clauses_in_custom_matcher_descriptions = true
  end

  config.mock_with :rspec do |mocks|
    mocks.verify_partial_doubles = true
  end

  config.include Aruba::Api
end

Test for Hello World

Now that Aruba is setup, this is the test for hello world:

1
2
3
4
5
6
7
8
9
10
11

describe 'test with aruba' do
  it 'tests hello_world.rb' do
    setup_aruba
    directory = "path/to/aruba_starter/src"
    run "ruby #{directory}/hello_world.rb"

    stop_all_commands
    sleep(1)
    expect(last_command_output).to eq("hello world\n")
  end
end

Let’s step through this line by line:

setup_aruba

This preps Aruba to start working its magic. (Really, I don’t know exactly what it does, but if it is not included, the rest doesn’t work.)

directory = "path/to/aruba_starter/src"

This line supports the next line by telling aruba where to find the hello_world.rb file.

run "ruby #{directory}/hello_world.rb"

This line is almost the same line which is used to run all hello world programs on the command line. The run() method tells Aruba to execute this code on the command line (in it’s own magical space). Isn’t that cool?

stop_all_commands

This tells Aruba to stop any running commands, like ones executed with the run() method.

sleep(1)

This is needed because there needs to be time for output to be received by Aruba. Otherwise, there is no output for the next line:

expect(last_command_started.output).to eq("hello world\n")

last_command_started refers directly to run(Ruby hello_world.rb), and .output is the output on the command line.

So, this is basically how one can test a hello world program:

1
2

# hello_world.rb
puts 'hello world'

using Aruba and RSpec. Whew.

Other languages?!

But wait, on Aruba’s website, it lists that it can also test other command line programs, like ones written in bash and Python. How can one test those too?

Using this RSpec and Aruba, there needs to be only one change to test python’s version of hello world:

run("python #{directory}/hello_world.py")

And the same for a bash script:

run("#{directory}/hello_world.sh")

So, the full test would look like this for Python:

1
2
3
4
5
6
7
8

it 'tests python hello world' do
  setup_aruba
  directory = '/home/acl/development/article_code/aruba_starter/src'
  run("python #{directory}/hello_world.py")
 stop_all_commands
  sleep(1)
  expect(last_command_started.output).to eq("hello world\n")
end

And this for bash:

1
2
3
4
5
6
7
8

it 'tests shell hello world' do
  setup_aruba
  directory = '/path/to/aruba_starter/src'
  run("#{directory}/hello_world.sh")
 stop_all_commands
  sleep(1)
  expect(last_command_started.output).to eq("hello world\n")
end

note: the shell program needs to be made executable using this command: chmod +x hello_world.sh

Yes, with this RSpec and Aruba setup, one can test literally any command line program. Isn’t that great??

Getting input??

Well, getting input to the programs being tested would be greater, since many command line programs under test have input, so let’s test a simple adder, which takes the form of:

1
2
3

Enter number: x
Enter another number: y
The sum of x and y are: z

So, the test could now be:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

it 'tests the adder' do
  setup_aruba
  directory = '/path/to/aruba_starter/src'
  number_1 = 1
  number_2 = 2
  number_3 = 3
  adder_output = "Enter number:\n"\
                 "Enter another number:\n"\
                 "The sum of #{number_1} and #{number_2} is: #{number_3}\n"
 run("ruby #{directory}/adder.rb")
  stop_all_commands
  sleep(1)
  expect(last_command_started.output).to eq(adder_output)
end

And one solution can be:

1
2
3
4
5
6

# adder.rb
puts "Enter a number:"
number_1 = gets.chomp.to_i
puts "Enter another number:"
number_2 = gets.chomp.to_i
puts "The sum of #{number_1} and #{number_2} is: #{number_1 + number_2}"

But, running the test will still fail. Why? Because there is no input!

To give input into a program being run by Aruba, use the type command. Incorporating that into the test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

it 'tests the adder' do
  setup_aruba
  directory = '/path/to/aruba_starter/src'
  number_1 = 1
  number_2 = 2
  number_3 = 3
  adder_output = "Enter a number:\n"\
                 "Enter another number:\n"\
                 "The sum of #{number_1} and #{number_2} is: #{number_3}\n"
  run("ruby #{directory}/adder.rb")
  type "#{number_1}"
  type "#{number_2}"
  stop_all_commands
  sleep(1)
  expect(last_command_started.output).to eq(adder_output)
end

And Ta-dah, the test passes!

Some limitations

  • Can’t have “interactive” tests, which checks the output intermittently. So far, Aruba can only check the output once.
    • https://github.com/cucumber/aruba/issues/407
  • Long running programs times out (i.e. Increase the sleep time to be longer)

Overall

Aruba is a great tool to have in any toolbox, not just working with ‘hello world’ programs, but any legacy code or coding any command line program.

Go Forth and (finally!) test Hello World!