Introduction

Within the software industry, once a product reaches the stable manual testing phase, every organization usually thinks of automated testing to save on the costs involved in manual testing. Since testing cost is an important factor for any project, organizations have started preferring open source test automation tools (which have reached a stage where they now rival the commercial ones) instead of investing in costly commercial testing tools. With no licensing costs, open source automation testing tools provide competitive features for automating the testing of software applications as well as Web portals.

A variety of open source automation testing tools is available for almost all types of testing such as functional, Web, UAT, regression, performance etc. Because of the extent to which these open source tools have matured, it’s time to think about them and have them in your QA automation kit. There are also various open source tools available to support the different testing types such as White Box Testing (Unit Testing, for e.g., using JUnit) and Black Box Testing (system/regression testing, for e.g., using Selenium, Sahi, Watir, TestMaker, LogiTest, TestGen4J, FitNesse etc).

The scope of this blog is to cover the basic flow for one of these open source Web testing automation tools – Selenium.

Why Selenium?

Selenium is probably the best option for automated testing of Websites today. It is becoming increasingly popular and it is the first choice of automation testers as well as organizations for automating the testing of Web-based applications for both the GUI as well as the functionality. Selenium can also be used as a unit testing tool for JavaScript. We at Xoriant have been working on Selenium for automating the testing of Web sites developed using AJAX for the pharmaceuticals and travel industries. The following graph shows the popularity of Selenium along with other open source automation testing tools.

Selenium Test Package

Selenium is a package of various test components which consists of the following three major tools. Each one has a specific role in aiding the development of test automation for a Web application.

1. Selenium IDE – A Firefox extension to record test cases and suites.
2. Selenium RC – Used to run tests on different browsers and systems.
3. Selenium Grid – Runs multiple instances of Selenium RC at once.
4. Qualitia and Tellurium – A wrapper for the Selenium engine.

Selenium Modes:

Bases on the components Selenium has following three modes for executing the test cases and test suites:

• Record-Playback mode (Selenium IDE)‏:

In this mode only Selenium IDE is used to record the test scenarios in terms of test cases in firefox. This is a great way to get started to writing tests and group them together to form the test suite. The recorded tests can be exported to many programming languages so that we can tweak them and put them in the testing framework. The test cases and test suites can be replayed back to check the verifications and validations or sent to Selenium RC or Grid for further tweaking.

• Selenium Remote Control (RC) Mode

In this mode Selenium starts multiple browsers (one at a time) and then runs the recorded test-cases which are saved in your language of choice. This helps to enhance the test cases with looping and programming techniques to cover all the required test scenarios and checks.

• Test Runner Mode

In this mode the test cases are recorded ad replayed in the form of HTML tables. This is just one more facility to execute the Selenium IDE as well as RC test cases. This helps to check the test results reports in better manner if not formatted already.

Testing AJAX with Selenium

Selenium IDE – Recording and updating a script

Selenium IDE is the FireFox Add-on provided by the Selenium group. This is very simple and easy to use add-on so that non-programmers can record and create the test scripts for automating web components. These automated test scripts are used as Selenium RC test cases by choosing the language code. i.e. Selenium IDE makes easier to create Selenium RC test cases. (Install Selenium IDE and Selenium RC from Selenium download page).

Selenium IDE is used for:

• Recording and updating the test cases or write them manually in table tab.
• Creating Test Suite by grouping the test cases under one group
• Exporting Test Cases/Suites the supported language and save for Selenium RC to enhance them
• Finding reference of every API and Selense commands
• Debugging test cases by toggling breakpoints through the commands

How commands generated in IDE for AJAX pages:

1. Almost all of the web pages are developed with AJAX technology since web2.0 evolution. While recording the page elements, Selenium IDE locates the page elements/objects using XPath/DOM/CSS. XPaths are found automatically, but you may have to update the specific elements in Xpath manually by looking into the source of the page to make them unique. Firebug is the helpful firefox Add-on for this purpose.
2. Every command has maximum two arguments. The first is usually target element pointed by its ID, name, link, DOM, CSS or XPath and the second is the value to that target element if exists. The selenium IDE commands when saved in the programming language of our choice for selenium RC purpose are called as Selenese.
3. The common commands are open, click, type, assert, verify. Commands beginning with ‘assert’ or ‘verify’ are for checks and verifications. If verify command is used and it fails, it fails the test case and continue to run the remaining tests. If assert command is used and it fails, the test case is aborted and stops to run the remaining tests.
4. For synchronizing the flow of test execution Selenium has commands ending with ‘AndWait’, it tells Selenium to wait for the page to load after the action has been done. However, it fails when using AJAX calls. In this case you have to use ‘WaitFor’ commands.

Selenium IDE – Most commonly used commands

The following are the most commonly used commands in the Selenium IDE:

open: Opens a page using a URL.

click: Clicks the element/object on the page.

clickAndWait: Performs a click operation, and optionally waits for a new page to load.

verifyTitle: Verifies the expected title and continues to run if it fails.

assertTitle: Verifies an expected page title and stops the execution if it fails.

verifyTextPresent: Verifies that the expected text is present somewhere on the page.

verifyElementPresent: Verifies an expected UI element, as defined by its HTML tag.

verifyText: Verifies that the expected text and its corresponding HTML tag are present on the page.

waitForPageToLoad: Pauses execution until an expected new page loads.

waitForElementPresent: Pauses execution until an expected UI element, as defined by its HTML tag, is present on the page. Used with AJAX calls.

Get more details about Selenium IDE here: Selenium IDE documentation.

If you need a full reference of all Selenium commands, read the Selenium commands.

Conclusion

In the first part of this blog, we have covered the basics of a typical test setup required for Selenium for automated testing of a Web-based application.

In the next part, we shall see automated testing in action – i.e., a practical example of how to actually use the Selenium IDE and RC to execute the tests. The reader is advised to check out the links highlighted in this blog to build the foundation needed for part two.

Introduction

This is the second in a three-part blog on Selenium – a free and open source automated software testing tool.

In part one of this blog, we saw the basics of Selenium, its test packages, test case execution modes and some of the commonly used commands in the Firefox-based Selenium IDE. We had also seen the basics of testing AJAX with Selenium, with an overview of how commands are generated in Selenium IDE for AJAX pages.

In this blog, we shall have an overview of AJAX testing using the Selenium IDE.

Commands available in a test script recorded using the Selenium IDE

To know about the commands available in the Selenium IDE, download the IDE and add it as an extension to the Firefox browser, after which, it will be available from under the Tools menu in Firefox.

In order to record test cases using the IDE:

1. Start the Firefox browser.
2. Go to the Tools menu of the browser and select Selenium IDE to launch the IDE.
3. In the Base URL field of the IDE, enter the URL of a Website on which user actions are to be recorded for automated testing purposes.
4. Click the red Record button on the top right side below the Base URL field.
5. Record the script for the test cases/test scenarios as per the test plan.
6. A Base URL field to enter the URL of the site to be tested.
7. A Record button to record the tests.
8. A Replay button with two options – one for replaying a particular test-case and one for replaying the entire test-suite.
9. A Pause/Resume button to pause or resume the tests under execution.
10. A Step button for step-wise debugging of the test cases.
11. A collapsible test case list area, test case tab area and test case detail area with Table and Source modes.
12. Text columns such as Command, Target and Value.
13. Log, Reference, UI-Element and Roll-up tabs at the bottom for reference.
14. Table mode, and
15. Source mode.

Selenium IDE has the following components to record and replay test scripts:

Let’s see a simple example of the test script generated using the Selenium IDE. The script displayed in the IDE has two modes:

Table Mode

This mode displays three column lists – Command, Target and Value, as shown in the previous screenshot. The Command option has recorded commands for the events as per the test cases while recording. The Target option has the target to be tested i.e., text, XPath, DOM and CSS. The Value option has the value for the command and the target for each event. We can also have text in the command line as a comment. The Table mode has three text fields, one each for the command, target and value. The Command field is an auto-complete dropdown used to enter and update/enhance the commands. The Target field is used to enter and update/enhance the value of the XPath/DOM/CSS object. The Value field is used to enter the specific value required for the commands recorded.

Source Mode

By default, the recorded script is displayed as HTML under the Source mode. However, the script can be exported in any supported/required target language for the Selenium RC framework of your interest. To do this, go to Options > Format and select the supported language for your framework from the available (configured) options, as shown in the following screenshot.

However, Selenium-IDE does not directly support:

• Conditional statements and iterations.
• Error handling, particularly unexpected errors.
• Database testing.
• Test case grouping and re-execution of failed tests.
• Test case dependency.
• Logging and reporting of test results.
• Capturing screenshots of test failures.

Although these tasks are not supported by Selenium directly, all of them can be achieved by using programming techniques with a language-specific Selenium RC client library – a subject that we shall look into in more detail in the third and concluding part of this blog.

Basics of Selenium RC

Selenium RC starts a Web server that provides APIs for client libraries as well as Selenium Grid – a tool which transparently distributes your tests over multiple machines allowing parallel execution of the test cases increasing the overall testing efficiency. Selenium-RC uses the full power of programming languages to create more complex tests like reading and writing files, querying a database, and emailing test results.

Selenium RC comes in the following two parts:

1. A server which automatically launches and kills browsers, and acts as a HTTP proxy for Web requests from them, and
2. Client libraries for your favorite computer programming language.

In order to execute the Selenese code in the programming language environment, first install the environment for that language. We will be using Java, so install Eclipse for Java, install JUnit and then install Selenium RC. The detailed steps are outlined below:

1. Download and unpack the Selenium RC archive from Selenium RC.
2. Set up a programming project for the Java client driver, as listed below:

• Download Selenium-RC from the Selenium HQ downloads page.
• Extract the file selenium-java-client-driver.jar.
• Open your desired Java IDE (Eclipse, NetBeans, IntelliJ, Netweaver, etc.)
• Create a new project.
• Add the selenium-java-client-driver.jar files to your project as references.
• Add to your project classpath the file selenium-java-client-driver.jar.
• From Selenium-IDE, export a script to a Java file and include it in your Java project, or write your Selenium test in Java using the selenium-java-client API. The API is presented later in this blog. You can either use JUnit, or TestNg to run your test, or you can write your own simple main() program. These concepts are explained later in this section.
• Run Selenium server from the console.
• Execute your test from the Java IDE or from the command-line.
  • Run console and go to the selenium-server folder.
  • Type: java -jar selenium-server.jar on the above path.

3. Install the Selenium-RC Server:

You may configure the Selenium server based on the java -jar selenium-server.jar –help help file. Now, save the sample code from our example above in Java format in your workspace directory and open it in Eclipse. A snapshot of the code that can be exported in Java for JUnit and TestNG looks like the one shown below:

package com.example.tests;
import com.thoughtworks.selenium.*;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.util.regex.Pattern;

public class Sample extends SeleneseTestCase {
@Before
	public void setUp() throws Exception {
	//code to start browser test
	}
@Test
public void testSample() throws Exception {

//Open the url

//code to verify the logo on the page

// code to get the auto-complete list
  	selenium.click("query");
	selenium.waitForPageToLoad("30000");
	selenium.type("query", "san");
	selenium.waitForPageToLoad("30000");
	selenium.keyDown("query", "40");
	selenium.mouseDown("query");
	selenium.waitForPageToLoad("30000");

//Select the item from the list

selenium.click("//div[@id='destination_selections']/ul/li[8]");
	selenium.waitForPageToLoad("30000");

@After
// code to stop the test case and browser
}

The autocomplete feature of AJAX

In the above example, we have automated the auto-complete dropdown scenario, which works like this:

Enter the prefix of the name of a city in the text field, e.g., “san” (as in “San Francisco”). The auto-complete list of cities is then displayed for “san”. Click on one of the items from the dropdown list. The clicked item will be given as an input to the field to display the next page. Verify a few items on the page to check whether the page displayed is indeed for the input which was given.

Conclusion

In part two of this blog, we saw how to test AJAX components in Selenium with a code example. We also saw the autocomplete feature of AJAX – a useful tool for testing textbox fields and dropdowns on Web pages.

In the third and concluding part, we shall see some advanced features of Selenium RC along with selense, and  wrappers for the Selenium engine.

Introduction

This is the third and concluding part of a three-part blog on Selenium – a free and open source automated software testing tool.

In part one of this blog, we saw the basics of Selenium, its test packages, test case execution modes and some of the commonly used commands in the Firefox-based Selenium IDE. We also had an overview of how commands are generated in the Selenium IDE for AJAX pages.

In part two, we had an overview of AJAX testing using the Selenium IDE.

In this part, we shall see some of the more advanced features that Selenium offers, such as Selenium RC, and wrappers for the Selenium engine.

Selenium commands vs. Selenese commands

Commands generated in the Selenium IDE Table mode are known as Selenium commands which are simple, type/click-type commands, whereas the commands generated when the script is exported in a user-selected programming language for Selenium RC are known as Selenese commands.

Using XPath/DOM/IDs for automating UI tests with exact position of objects

Selenium is capable of capturing the test items/objects both with and without the XPath values. Capturing the item with its absolute XPath value helps to test the exact position of the object on the page, whereas without the XPath value, the item can be located anywhere on the page. The items which are captured from the dropdown/combo lists always have an XPath value associated with them. These values are in the //div[n]/div[]/h2, //div[n]/ul/li format. In the following subsections, we will analyze the command flow with XPath to select some items from the auto-complete lists.

Synchronizing the script with the mouse and keyboard related commands

While recording, the generated script only has commands related to the user’s mouse clicks and keystrokes, but these commands are not sufficient and fail when the recorded script is replayed. Only clicked and typed commands are not sufficient to recognize the auto-complete list items displayed on the screen. In order to get an item selected from the auto-complete list and execute the script successfully, we need to modify the script with some keyboard and mouse related commands such as keypress, keydown, mousedown, mouseup and mousedownAndWait etc. The flow of script execution is synchronized with such commands.

A sample script generated while recording a test case is:

click query
type san
click //div[@id='destination_selections']/ul/li[8]

This script can be modified with some keyDown and mouse events to synchronize the selection of items from the auto-complete list so that the script executes successfully:

clickAndWait query
typeAndWait query san
keyDown query 40
mouseDownAndWait query
clickAndWait //div[@id='destination_selections']/ul/li[8]

A user possessing knowledge of the basic XPath concepts will easily understand that the position of the item is in ‘li’ of ‘ul’ of ‘div’ with id equal to ‘destination_selections’. This simulates a click on the 8^th ‘li’ item in ‘/ul/’ which is “San José del Cabo” for our example. However, we can generalize it further using regular expressions so that it will select the item based on the given ‘li’ id.

Some of the basic commands along with their Java Selenese equivalent commands are listed next.

Commands for synchronization of the script execution

clickAndWait commands with XPath:
clickAndWait //div[@id='destination_selections']/ul/li[8]

The equivalent Java Selenese for this command is:

selenium.click("//div[@id='destination_selections']/ul/li[8]");
selenium.waitForPageToLoad("30000");

Another example of wait command:

waitfor commands with XPath:
waitForText  //div[@id='leftColumn']/div[3]/div/h2 Best hotels, facts and information in San José del Cabo

The equivalent Java Selenese for this command is:

for (int second = 0;; second++) {
if (second >= 60) fail("timeout");
	try {
if ("Best hotels, facts and information in San José del cabo".equals(selenium.getText("//div[@id='leftColumn']/div[3]/div/h2")))
break; } catch (Exception e) {}
	Thread.sleep(1000);
		}

Commands for verification and validation – verify commands with XPath:

verifyText  //div[@id='leftColumn']/div[3]/div/h2 Best hotels, facts and information in San José del Cabo

The equivalent Java Selenese for this command is:

verifyEquals("Best hotels, facts and information in San José del Cabo", selenium.getText("//div[@id='leftColumn']/div[3]/div/h2"));

This is how simple and easy it is to understand the script converted in the Selenese (JUnit) code. For further automation tasks, we need to apply regular expressions, parameterization, and enhance the script with conditional loops for functional logic and this is the actual challenging task for the QA engineer.

Selenium Grid

Selenium Grid manages several Selenium RCs through API commands which are called Selenese commands. It is a tool which dramatically speeds up functional testing of Web applications by leveraging the user’s existing computing infrastructure. It allows the user to easily run multiple tests in parallel, on multiple machines, in a heterogeneous environment. If the user wishes to test Selenium Grid, additional information may be obtained from here. Setting up Selenium RC on different systems and connecting them using Selenium Grid is a pretty complex task, but it can be made easier by buying services that run in clouds. An example of such a service is http://saucelabs.com.

Wrappers for Selenium: Qualitia™ and Tellurium

Writing UI module-based Web automation scripts in Selenium is not yet supported. To achieve this, there are some wrappers available for the Selenium engine. Qualitia and Tellurium are two popular such wrappers used for Selenium and a few similar other tools. While presently acting as wrappers for the Selenium test units, Qualitia and Tellurium seem to be developing into independent test tools going forward.

Qualitia™

Qualitia™ is a thick client application based on .NET. It has seamless integration and the ability to work as a wrapper with market leading test automation tools such as Rational Functional Tester, QTP, and Selenium. It also has common test case management methodologies which helps the Selenium engine to manage the test cases and execute them as suites as well as scenarios. The fully loaded and easy to use graphical user interface helps to access the functionalities, building test cases, execution, error reporting, diagnosis and analysis. Qualitia can generate both XML based HTML reports.

Tellurium

Because Selenium does not support the ‘UI module-based’ Web automated testing, Tellurium is used as a wrapper for Selenium unit tests. Tellurium is a portable software testing framework for Web applications that runs on top of Selenium. The statement Tellurium is just a wrapper for Selenium was true only till version 0.6.0. Tellurium will be an independent testing framework while still acting as wrapper to Selenium unit tests (for backward compatibility) from version 0.7.0 onwards. Tellurium focuses on Groovy objects for runtime locator mapping, and then uses Selenium RC under the hood to drive the tests. The diagram below explains how Tellurium acts as a wrapper along with the test execution flow. The Tellurium framework is a separate topic altogether and is not within the scope of this blog.

Using Tellurium with Selenium RC and Selenium Core

Best practices for using Selenium

• Make sure that all tests are source controlled.
• Make use of nightly runs for most of the tests.
• Try to use one test per function, and not one test per page.
• Always use separate environment specific variables.
• Automate only the fixed functions, keep the other ones manual.
• Try to abstract out and reuse the functionality.
• Maximize the use of parameterization in your tests.
• Use ids wherever possible (XPaths make the tests brittle).
• In order to generate unique ids, use timestamps.
• Use the right level of granularity.
• Use data-driven tests.
• Use JSP/PHP pages to generate the tests.

Benefits of Selenium

• Free and open-source tool.
• Helps automate testing on AJAX/CSS applications.
• Supports testing on multiple browsers.
• Contains record and playback facility.
• Best for GUI-intelligent field selection (uses IDs, names, or XPaths as needed).
• Auto-complete feature available in IDE for all common commands.
• User-friendly features for debugging and setting breakpoints.
• Available in multiple languages of the user’s choice; can save tests as HTML, Ruby scripts, or several other formats.
• Support for Selenium user-extension .js files for explicitly looping add-ons.
• Option to automatically assert the title of every page.
• Mimics actual user experience.
• Continuous integration:
  • Runs Selenium tests as part of the build.
  • Can generate HTML reports and publish them to the entire team.
  • Helps catch bugs ASAP.
• Large user community.

Limitations of Selenium

Problems in low level testing:

• Performance issues (HTMLUnit, WebTest).
• Reporting issues (WebTest).
• CSS/AJAX needs tricky coding for correct and efficient usage.
• Data dependency needs to be avoided.
• Slow when testing edge cases.
• Slow for testing fine-grained features (since set-up and tear-down are called for each test).

Limitations applicable to Selenium Core:

• Hard to automate and generate alerts.
• Cannot go with “Same Origin Policy”.
• File operations such as file uploads are difficult to handle.

Common problems in Web applications testing:

• Record/replay feature is not reusable and is fragile.
• Test case management is not flexible.
• UI changes make tests breakable and need rework.

Conclusion

In this concluding part of the blog on Selenium, we saw several advanced features of the Selenium suite, such as Selenium RC and Selenium Grid. We also had an overview of some of the wrappers available for Selenium, which help the user in performing tasks not supported by the Selenium core itself. We finally ended the blog with a quick overview of Selenium’s advantages and disadvantages.

Overall, the popularity of Selenium is increasing by the day since many software applications are slowly migrating from the traditional desktop platform to the Web platform. The open-source advantage of Selenium along with its large present user base will ensure that it remains one of the most popular tools for testing Web-based applications for years to come.