Red Green Repeat Adventures of a Spec Driven Junkie

Using Selenium with Ruby

Why?

Selenium-WebDriver with Ruby is of the most powerful tools I have used to test web servers through the browser and automate any kind of repetitive web tasks. It is also how I discovered outside in testing.

Selenium and Ruby is a great way to automate client tasks for a web server. Debugging is easier as well, because you can see the script run in the browser, step by step.

I love using Ruby in general. At the same time, a good resource for using Ruby and Selenium together are very sparse, also I just want to have a reference for myself in the future to share with others.

How to use Selenium with a Ruby script?

I will explain how to get Selenium running with a basic Ruby script. This setup will be independent of any framework. All that is required is a web browser that can connect with Selenium and the Ruby gem for Selenium. I want to have a bare bone script to be able to use in almost any environment which can run Ruby, Selenium, and Firefox.

Getting started:

  • Make sure you have a recent version of Ruby installed. I will use Ruby 2.1.5 (managed by rvm)

    rvm install ruby-2.1.5

    note: A newer version of Ruby is fine, my personal preference is to use a stable version of Ruby and 2.1.5 has been a favorite.

  • Install the Selenium-Webdriver gem

    gem install selenium-webdriver

    The version used in this document is: 2.53.3.

  • FireFox web browser

    Most versions of Firefox will have support for Selenium. The version of Firefox used for this article is: 46.0.1

First step: test connections

I always want to run a simple test to make sure each component can talk to each other. Here, we want to make sure the Ruby script can load the Selenium library and launch Firefox. Ruby’s irb is a great way to test this using these two lines:

1
2
3
4
2.1.5 :001 > require 'selenium-webdriver'
 => true
2.1.5 :002 > browser = Selenium::WebDriver.for :firefox
 => #<Selenium::WebDriver::Driver:0x..f96ee1c90b443ac24 browser=:firefox>

Problem solving tips if Firefox is not starting up

  • Double check with the Github issues list for Selenium

  • At the time of writing, there were issues with FireFox 47 and Selenium WebDriver 2.53

  • The best way to fix this is to use an older version of Firefox, which can be found in this support article or Mozilla’s FTP archive.

  • Try using an older version of FireFox with Selenium if the above script is not working.

  • Note: if Firefox is not installed, an exception will be thrown:

2.1.5 :002 > browser = Selenium::WebDriver.for :firefox
Selenium::WebDriver::Error::WebDriverError: Could not find Firefox
binary (os=macosx).
Make sure Firefox is installed or set the path manually with
Selenium::WebDriver::Firefox::Binary.path=

How to load a web address using Selenium

The next most important step is telling the browser (programmatically) how to load a web address so it can go to the site. There are many ways of doing this, but the way I usually do it is once the browser has been launched, run these commands.

2.1.5 :004 > browser
 => #<Selenium::WebDriver::Driver:0xf0cec284e895d64 browser=:firefox>
2.1.5 :005 > browser.navigate.to 'http://redgreenrepeat.com/2016/07/29/selenium-ruby/'
 => ""

Problem solving tips if the address did not load

  • make sure the browser is connected to a Firefox window. If the browser window was closed by accident, restart it by initializing another instance:
  2.1.5 :006 > browser2 = Selenium::WebDriver.for :firefox
  => #<Selenium::WebDriver::Driver:0xe0ced172e885e32 browser=:firefox>
  2.1.5 :007 > browser2.navigate.to 'http://redgreenrepeat.com/2016/07/29/selenium-ruby/'
  
  • If the browser instance was closed before, an exception will be thrown:
  2.1.5 :005 > browser.navigate.to 'http://redgreenrepeat.com/2016/07/29/selenium-ruby/'
  Errno::ECONNREFUSED: Connection refused - connect(2) for "127.0.0.1"
  port 7055
  
  • Double check address. :-D

    The address to be used in this page’s examples:

    http://redgreenrepeat.com/2016/07/29/selenium-ruby/

How to select page elements

Now that the browser is setup to load a page, the next function to use: selecting page elements.

This is probably one of the most frequently used calls with Selenium, because there are so many elements in a web page to interact with. To interact with any of them, they have to be selected. There are a few ways to select elements in a web page, it all depends on how the web page is setup. This article will cover selecting by: ID tag, class, and XPath.

Selecting elements that have an ID tag associated with them

The easiest way to select an element with Selenium is when it has an ID tag associated with it. By definition, ID tags are all unique to a page and Selenium can target specific elements easily. To do that, use the .find_element(:id, "<element targeted>") call. Quick demonstration on this page!

Element with ID Tag

1
2
2.1.5 :006 > browser.find_element(:id, 'test')
 => #<Selenium::WebDriver::Element:0x2f75f4922d8517c6 id="{903e82a4-f591-4948-8991-885a821ce5c6}">

Problem solving tips if selecting by ID does not work

  • Check that an item is selected using: element.present?? (?) If there are more than one item on a page, the first one will be selected.

  • If the ID does not appear on the page source, an error will be thrown:

1
2
3
4
5
2.1.5 :007 > browser.navigate.to "http://redgreenrepeat.com/2016/07/29/selenium-ruby/"
 => ""
2.1.5 :008 > browser.find_element(:id, 'test2')
Selenium::WebDriver::Error::NoSuchElementError: Unable to locate
element: {"method":"id","selector":"test2"}

Selecting elements by their class

If the element to be selected does not have a unique ID tag that can be targeted by .find_element(:id, <id>), but it has a class associated with it, Selenium can select all the elements which have the same class in an array.

As long as the element occurs in the same order on the page, this can be a reliable technique to select elements and filter through them in an array.

.find_elements(:class_name, <class>)

Note that the function call is: .find_elements (plural and not singular.)

1
2
3
2.1.5 :008 > browser.find_elements(:class_name, 'test_class')
 => [#<Selenium::WebDriver::Element:0x6b628b2285b77b38 id="{20972ab1-1e4e-5e4e-a9a9-b80132c71a03}">,
     #<Selenium::WebDriver::Element:0x53176c4d870a7976 id="{6212c0e5-9451-834e-85f6-109b738814aa}">]

Selecting elements by their ‘Xpath’

The final way I will talk about selecting an element is by using an element’s Xpath.

Using the Xpath is a way to select an arbitrary element on a web page that has no class or ID associated with it. Now I have never heard of ‘Xpath’ until using Selenium, but every element on any web page has a unique ‘Xpath’ with it.

For me, I need to use another tool to look up an element’s Xpath, and the tool I use is Firefox’s FirePath extension. The FirePath extension can be downloaded here and Firebug can be downloaded here (current versions used when writing this document: 2.0.17)

To use it, it’s basically the same as inspecting any element on the webpage. Instead of selecting ‘inspect element’, select: ‘inspect element with firebug’

  1. Right click on page to get the context menu.
  2. Select: “Inspect element with Firebug”
  3. In the tab menu, select: “FirePath”

To get the element’s Xpath, in the FirePath console:

  1. Find the element in the document tree by expanding and looking for its highlight
  2. Right click over the element’s DOM tag (i.e. <p>)
  3. Select: “Copy Xpath”

For example, to get the path to the paragraph for the code below, the Xpath is:

browser.find_elements(:xpath, "html/body/div/main/div[1]/article/div[9]/pre/code")

Selecting elements by: ID, class, and Xpath

Three ways to select elements on a web page have been presented. These are the three most common ways I use to select elements on a page. There are a few more ways to select an element on a web page through Selenium, the documentation is a great resource.

Interacting with selected element

After an element has been selected, interacting with the element is essential to making any script do useful things. The ways I usually interact with page elements are: click or enter text.

Click interaction

One of the most used interactions on the web is the ‘click’. Click this link, click this button, click this field. Performing a ‘click’ in Selenium is exactly the same as a user performing the click with their mouse.

To peform a click on a selected element, just send the .click message. (i.e. element.click) For example:

1
2
3
4
5
6
7
8
2.1.5 :011 > browser = Selenium::WebDriver.for :firefox
 => #<Selenium::WebDriver::Driver:0x..fea93fd18efb1608 browser=:firefox>
2.1.5 :012 > browser.navigate.to 'http://redgreenrepeat.com/2016/07/29/selenium-ruby/'
 => ""
2.1.5 :013 > element = browser.find_element(:id, 'test_click')
 => #<Selenium::WebDriver::Element:0x493ee5b5e5030460 id="{3489fd8e-3afa-0842-a17a-3c6abdef5177}">
2.1.5 :014 > element.click
 => "ok"

Entering text

Another interaction that is very useful to have in a script to test a web page is to enter text. Any text field like username, password, search, etc. can have text entered into it once the element is selected.

To enter text on a selected element (make sure the element can receive text), send the .send_keys message to the element. (i.e. element.send_keys('hello world')) For example:

  • Enter text - simulate a user entering text

1
2
3
4
5
6
2.1.5 :024 > browser.navigate.to 'http://redgreenrepeat.com/2016/07/29/selenium-ruby/'
 => ""
2.1.5 :025 > element = browser.find_element(:id, 'test_input')
 => #<Selenium::WebDriver::Element:0x..fe3e2d3833718b314 id="{3b28ab8b-d327-ee4b-91a6-d15e5f9fdfea}">
2.1.5 :026 > element.send_keys 'hello world'
 => ""

Interacting with elements

The two most common ways I interact with elements in Selenium is to use the .click and .send_keys events to click and type in text for an element. There are a few more ways to interact with elements to be found in the Selenium documentation.

Reading values of elements

The last function I use frequently with Selenium is reading an element’s value. This is essential when using the script to make sure the page has a correct known value (or even to compare values within the page.)

When I build out scripts for commerce websites, I want to simulate a user shopping. On checkout, I want to make sure the cart sums the items properly, applies tax, etc. Reading values from a web page make scripting this out simple.

To read the object’s value, send message: .text on the selected element. (i.e. element.text) For example:

Contents of test_read element


1
2
3
4
5
6
2.1.5 :009 > browser.navigate.to 'http://redgreenrepeat.com/2016/07/29/selenium-ruby/'
 => ""
2.1.5 :010 > element = browser.find_element(:id, 'test_read')
 => #<Selenium::WebDriver::Element:0xe9ef59dfcb30f58 id="{07900097-19fb-9b4f-bc1f-a693bbde14b7}">
2.1.5 :011 > element.text
 => "Contents of test_read element"

Problem solving tips

  • Make sure the element has been selected.

    Selenium will throw an exception if the item cannot be found and will return nil for the object.

1
2
3
4
5
6
7
  2.1.5 :012 > invalid_element = browser.find_element(:id, 'invalid')
  Selenium::WebDriver::Error::NoSuchElementError: Unable to locate
  element: {"method":"id","selector":"invalid"}
    ...
  2.1.5 :012 > invalid_element
   => nil
  

Single page applications

One type of web page I found that are trickier to work with in Selenium are single page apps which use front end frameworks such as AngularJS and EmberJS.

The reason for this difficulty is a script in Selenium would expect the whole page to be rendered once it loads, but a single page app does not render the page until the browser’s JavaScript function renders it.

For single page applications, the workaround I have found is to use Ruby’s sleep(1) method. This method waits a specified amount of (wall) time, which allows the browser’s JavaScript engine to render the page.

Conclusion

Selenium’s API in Ruby has been presented. This is something I use frequently to test out web servers from the outside in and want to be able to simulate a user with a script. The Selenium API is great, but searching for examples is tricky, especially since so many other programming languages can interface with it.

Items covered in this article:

  • Getting Selenium going with Ruby and FireFox
  • Selecting elements by ID, class, and Xpath
  • Interacting with elements by click and entering text
  • Reading element values
  • Workaround for single page applications

Where to next?

Well, there’s more of the Selenium API to explore and mixing in Ruby’s language features (I.e. Looping, etc.) will take the script very far. It’s quite useful tool to have when working with web servers.

Great documentation can be found at:

http://docs.seleniumhq.org/docs/03_webdriver.jsp

Go forth and script the browser!

update (Aug 6, 2016) - A comment from Redditor /u/notorious1212 informed me about Watir, which looks pretty awesome. I will have to check it out in the future!

update (Sep 21, 2017) - Another great resource is by the Guru99, which is very in-depth and goes beyond this article, way beyond:

https://www.guru99.com/selenium-tutorial.html