In this blog I will be introducing FitNesse tool, an automated testing tool for user acceptance testing. This blog will give an overview of the tool which is a Wiki built on top of Fit framework and used for automating Acceptance test cases.

Existing User Acceptance Testing Processes

• Manual Acceptance testing: User exercises the system manually using his system knowledge.
• Acceptance testing with “GUI Test Drivers” (at the GUI level): These tools help the developer perform functional/acceptance testing through user interface such as a native GUI or web interface.
• Table-based Approach for acceptance testing: Starting from a user story (or use case or textual requirement), the customer enters in a table (spreadsheet application, html, etc) the expectations of the program’s behavior. The customer can manually insert inputs in the system and compare outputs with expected results.

Why FitNesse?

The Existing Manual User Acceptance Test process as described above is time consuming, prone to errors and incurs cost. Manual testing becomes exhaustive because of the large test data and numerous test cases. This process can be improved by automating the test cases for acceptance testing phase. FitNesse is the only Open source automation tool available for automating the Acceptance test cases/scenarios. This tool is Simple and has Robust Test Frameworks and also provides simple and user friendly result reporting.

What is FitNesse?

• FitNesse is an Open source integrated testing framework for automating the User Acceptance Testing and Regression Testing process.
• FitNesse creates a feedback loop between customers and programmers and allows customers and testers to use tools like MS Office Excel to give examples on how a program should behave.
• FitNesse automatically checks those examples against the actual program, thus building a simple and powerful bridge between the business and software engineering worlds.
• FitNesse enables customers, testers and programmers to learn what their software should do, and to automatically compare that to what it actually does.
• FitNesse is used by agile web teams to create comprehensive test suites that make system-level assertions to test in the TDD process.

FitNesse is basically

• Open source User accepting Testing tool written in Java 1.4
• Tests written in the form of HTML tables in wiki pages
• Makes use of two frameworks – Fit and Slim frameworks
• Test tables references fixture classes that connects to the business logic, Fixture classes/code are written in either – Java, .Net, Ruby, Python, C#

Benefits of FitNesse

Reusability of Test Cases based on dataset – Same Set of Test Cases can be used to test positive as well as negative scenarios with different dataset.

Reusability of Framework and Fixtures – As the Fixture classes are developed using Java, they can be reused for similar scenarios and fixture tables. Frameworks are also reusable for similar types of scenarios and functionalities.

Unit Testing and Regression Testing – FitNesse can be used and implemented for Unit Testing as well as Regression testing. Generation of fixture classes and test table can be incorporated as a process in the development life cycle. This gives advantage of using the same test cases for unit testing which will save time required for next levels of testing by finding the defects at early stages which will reduce testing cost.

Script Maintenance Activity – The developer, who works on change request can also modify the test cases in Fitnesse as a part of process. This will help in upgrading the test scripts quickly as per the new features in the application.

Multiple Test Suites – Once all the test cases are created in Fitnesse, different Test Suite can be created based on the selection of test cases that are required for execution. This will help in creating multiple test suites like- Unit Test, Regression Test and user Acceptance Test.

Independent Test Cases – The Test Cases in Fitnesse are independent of each other. During script execution, if a specific test case is not correctly executed it will not affect other test cases in the suite.

Reliability – Automation Process is always more reliable than the Manual Process

FitNesse Testing Process/Framework

1. Write unit/regression/acceptance tests:  These tests are written in a form of executable tables with Wiki syntax.
2. Write fixture code:  After the customers define the acceptance tests in wiki tables, the developers work on implementing the test cases with fixtures which communicates with business logic.
3. Run acceptance tests:  Acceptance tests are executed through the FitNesse Test runner.  The Test runner parses the test tables to find the key words and link them together to construct the class name of the fixture code which communicates with the business logic and returns the actual data.
4. View test results:  After running the acceptance tests on the FitNesse server, the results are shown in the form of the original test table with the appropriate value cells colors that represent the acceptance testing results.

The Table styles

There are two types of test-systems/frameworks available in FitNesse- Fit and Slim. The test/wiki pages have wiki text written in wiki syntax, the input and expected output is written in the form of tables. The Fit and Slim Test System has it’s own particular kind of Test Table styles. First Row- first cell always refers Fixture Name. Subsequent cells are used to pass the input parameters and expected values.

The Most Common Fit Table/Fixture Styles

Column Fixture	This is the style you may end up using most: rows of data represent inputs and expected outputs.
Row Fixture	This is good for testing queries that should return an exact set of values (Order-independently).
Action Fixture	This style allows you write a script that emulates a series of events (Such as controls manipulated on a user interface).
Comment Tables	Sometimes you want a tabular comment that is not executed as a test.

The Slim Table styles

The first cell of a slim table tells you what kind of table it is. Here are the table types so far:

Decision Table	This is similar to Fit Column Fixture which supplies inputs and outputs for decision.
Query Table	Supplies the expected results of a query. This is similar to the Fit Row Fixture
Subset Query Table	Supplies a subset of the expected results of a query.
Ordered query Table	Supplies the expected results of a query. The rows are expected to be in order. This is similar to the Fit Row Fixture
Script Table	A series of actions and checks. Similar to Do Fixture.
Table Table	The design which you want it to be!
Import	Add a path to the fixture search path.
Comment	A table that does nothing.
Scenario Table	A table that can be called from other tables.
Library Table	A table that installs fixtures available for all test pages

Advantages of FitNesse

• Fitnesse is webserver – requires no extra configuration or Setup
• FitNesse is a wiki –You can easily create and edit the testcases on wiki pages
• Tests can be written before the code so that this approach can support TDD-Agile approach
• Requirements Engineering –
  • Easy to describe the Requirements themselves for existing frameworks using tables
  • Good tool for requirements engineering
  • Easy for the developer to glue the requirements with business logic using Fixtures
• FitNesse is light weight and Open Source framework that makes it easy for software teams to:
  • Collaboratively define Acceptance Tests- wiki with table of inputs and expected outputs
  • Write the Fixture code to communicate this tests with Business logic
  • Run those tests and analyze the results
• FitNesse has features adapted to system tests, like:
  • CommandLineFixture available to execute special cases
  • DoFixture and Ordered Query table executing some methods in order
• FitNesse is perfectly suited for regression testing
• The biggest advantage Fitnesse has over other integration testing frameworks is that the entry point into the code stack is not the (web) frontend – fixtures are written in java/C# and call out to whatever part of the codebase you are testing. This allows us to test separate layers as well as testing the entire code stack working together.

Limitations of FitNesse

1. No capture & replay possible
2. Cannot explicitly test a web frontend
3. Web interface can make setting up expectations time consuming
4. Learning curve to understand the framework is steeper if we have to modify the framework.

This was a brief introduction to the FitNesse tool, in the subsequent blogs we shall see how to use FitNesse tool to Create Test pages/wiki pages, Write Fixture classes, etc along with code samples

Introduction

This is the third and concluding part of a three-part blog on Selenium – a free and open source automated software testing tool.

In part one of this blog, we saw the basics of Selenium, its test packages, test case execution modes and some of the commonly used commands in the Firefox-based Selenium IDE. We also had an overview of how commands are generated in the Selenium IDE for AJAX pages.

In part two, we had an overview of AJAX testing using the Selenium IDE.

In this part, we shall see some of the more advanced features that Selenium offers, such as Selenium RC, and wrappers for the Selenium engine.

Selenium commands vs. Selenese commands

Commands generated in the Selenium IDE Table mode are known as Selenium commands which are simple, type/click-type commands, whereas the commands generated when the script is exported in a user-selected programming language for Selenium RC are known as Selenese commands.

Using XPath/DOM/IDs for automating UI tests with exact position of objects

Selenium is capable of capturing the test items/objects both with and without the XPath values. Capturing the item with its absolute XPath value helps to test the exact position of the object on the page, whereas without the XPath value, the item can be located anywhere on the page. The items which are captured from the dropdown/combo lists always have an XPath value associated with them. These values are in the //div[n]/div[]/h2, //div[n]/ul/li format. In the following subsections, we will analyze the command flow with XPath to select some items from the auto-complete lists.

Synchronizing the script with the mouse and keyboard related commands

While recording, the generated script only has commands related to the user’s mouse clicks and keystrokes, but these commands are not sufficient and fail when the recorded script is replayed. Only clicked and typed commands are not sufficient to recognize the auto-complete list items displayed on the screen. In order to get an item selected from the auto-complete list and execute the script successfully, we need to modify the script with some keyboard and mouse related commands such as keypress, keydown, mousedown, mouseup and mousedownAndWait etc. The flow of script execution is synchronized with such commands.

A sample script generated while recording a test case is:

click query
type san
click //div[@id='destination_selections']/ul/li[8]

This script can be modified with some keyDown and mouse events to synchronize the selection of items from the auto-complete list so that the script executes successfully:

clickAndWait query
typeAndWait query san
keyDown query 40
mouseDownAndWait query
clickAndWait //div[@id='destination_selections']/ul/li[8]

A user possessing knowledge of the basic XPath concepts will easily understand that the position of the item is in ‘li’ of ‘ul’ of ‘div’ with id equal to ‘destination_selections’. This simulates a click on the 8^th ‘li’ item in ‘/ul/’ which is “San José del Cabo” for our example. However, we can generalize it further using regular expressions so that it will select the item based on the given ‘li’ id.

Some of the basic commands along with their Java Selenese equivalent commands are listed next.

Commands for synchronization of the script execution

clickAndWait commands with XPath:
clickAndWait //div[@id='destination_selections']/ul/li[8]

The equivalent Java Selenese for this command is:

selenium.click("//div[@id='destination_selections']/ul/li[8]");
selenium.waitForPageToLoad("30000");

Another example of wait command:

waitfor commands with XPath:
waitForText  //div[@id='leftColumn']/div[3]/div/h2 Best hotels, facts and information in San José del Cabo

The equivalent Java Selenese for this command is:

for (int second = 0;; second++) {
if (second >= 60) fail("timeout");
	try {
if ("Best hotels, facts and information in San José del cabo".equals(selenium.getText("//div[@id='leftColumn']/div[3]/div/h2")))
break; } catch (Exception e) {}
	Thread.sleep(1000);
		}

Commands for verification and validation – verify commands with XPath:

verifyText  //div[@id='leftColumn']/div[3]/div/h2 Best hotels, facts and information in San José del Cabo

The equivalent Java Selenese for this command is:

verifyEquals("Best hotels, facts and information in San José del Cabo", selenium.getText("//div[@id='leftColumn']/div[3]/div/h2"));

This is how simple and easy it is to understand the script converted in the Selenese (JUnit) code. For further automation tasks, we need to apply regular expressions, parameterization, and enhance the script with conditional loops for functional logic and this is the actual challenging task for the QA engineer.

Selenium Grid

Selenium Grid manages several Selenium RCs through API commands which are called Selenese commands. It is a tool which dramatically speeds up functional testing of Web applications by leveraging the user’s existing computing infrastructure. It allows the user to easily run multiple tests in parallel, on multiple machines, in a heterogeneous environment. If the user wishes to test Selenium Grid, additional information may be obtained from here. Setting up Selenium RC on different systems and connecting them using Selenium Grid is a pretty complex task, but it can be made easier by buying services that run in clouds. An example of such a service is http://saucelabs.com.

Wrappers for Selenium: Qualitia™ and Tellurium

Writing UI module-based Web automation scripts in Selenium is not yet supported. To achieve this, there are some wrappers available for the Selenium engine. Qualitia and Tellurium are two popular such wrappers used for Selenium and a few similar other tools. While presently acting as wrappers for the Selenium test units, Qualitia and Tellurium seem to be developing into independent test tools going forward.

Qualitia™

Qualitia™ is a thick client application based on .NET. It has seamless integration and the ability to work as a wrapper with market leading test automation tools such as Rational Functional Tester, QTP, and Selenium. It also has common test case management methodologies which helps the Selenium engine to manage the test cases and execute them as suites as well as scenarios. The fully loaded and easy to use graphical user interface helps to access the functionalities, building test cases, execution, error reporting, diagnosis and analysis. Qualitia can generate both XML based HTML reports.

Tellurium

Because Selenium does not support the ‘UI module-based’ Web automated testing, Tellurium is used as a wrapper for Selenium unit tests. Tellurium is a portable software testing framework for Web applications that runs on top of Selenium. The statement Tellurium is just a wrapper for Selenium was true only till version 0.6.0. Tellurium will be an independent testing framework while still acting as wrapper to Selenium unit tests (for backward compatibility) from version 0.7.0 onwards. Tellurium focuses on Groovy objects for runtime locator mapping, and then uses Selenium RC under the hood to drive the tests. The diagram below explains how Tellurium acts as a wrapper along with the test execution flow. The Tellurium framework is a separate topic altogether and is not within the scope of this blog.

Using Tellurium with Selenium RC and Selenium Core

Best practices for using Selenium

• Make sure that all tests are source controlled.
• Make use of nightly runs for most of the tests.
• Try to use one test per function, and not one test per page.
• Always use separate environment specific variables.
• Automate only the fixed functions, keep the other ones manual.
• Try to abstract out and reuse the functionality.
• Maximize the use of parameterization in your tests.
• Use ids wherever possible (XPaths make the tests brittle).
• In order to generate unique ids, use timestamps.
• Use the right level of granularity.
• Use data-driven tests.
• Use JSP/PHP pages to generate the tests.

Benefits of Selenium

• Free and open-source tool.
• Helps automate testing on AJAX/CSS applications.
• Supports testing on multiple browsers.
• Contains record and playback facility.
• Best for GUI-intelligent field selection (uses IDs, names, or XPaths as needed).
• Auto-complete feature available in IDE for all common commands.
• User-friendly features for debugging and setting breakpoints.
• Available in multiple languages of the user’s choice; can save tests as HTML, Ruby scripts, or several other formats.
• Support for Selenium user-extension .js files for explicitly looping add-ons.
• Option to automatically assert the title of every page.
• Mimics actual user experience.
• Continuous integration:
  • Runs Selenium tests as part of the build.
  • Can generate HTML reports and publish them to the entire team.
  • Helps catch bugs ASAP.
• Large user community.

Limitations of Selenium

Problems in low level testing:

• Performance issues (HTMLUnit, WebTest).
• Reporting issues (WebTest).
• CSS/AJAX needs tricky coding for correct and efficient usage.
• Data dependency needs to be avoided.
• Slow when testing edge cases.
• Slow for testing fine-grained features (since set-up and tear-down are called for each test).

Limitations applicable to Selenium Core:

• Hard to automate and generate alerts.
• Cannot go with “Same Origin Policy”.
• File operations such as file uploads are difficult to handle.

Common problems in Web applications testing:

• Record/replay feature is not reusable and is fragile.
• Test case management is not flexible.
• UI changes make tests breakable and need rework.

Conclusion

In this concluding part of the blog on Selenium, we saw several advanced features of the Selenium suite, such as Selenium RC and Selenium Grid. We also had an overview of some of the wrappers available for Selenium, which help the user in performing tasks not supported by the Selenium core itself. We finally ended the blog with a quick overview of Selenium’s advantages and disadvantages.

Overall, the popularity of Selenium is increasing by the day since many software applications are slowly migrating from the traditional desktop platform to the Web platform. The open-source advantage of Selenium along with its large present user base will ensure that it remains one of the most popular tools for testing Web-based applications for years to come.

Introduction

This is the second in a three-part blog on Selenium – a free and open source automated software testing tool.

In part one of this blog, we saw the basics of Selenium, its test packages, test case execution modes and some of the commonly used commands in the Firefox-based Selenium IDE. We had also seen the basics of testing AJAX with Selenium, with an overview of how commands are generated in Selenium IDE for AJAX pages.

In this blog, we shall have an overview of AJAX testing using the Selenium IDE.

Commands available in a test script recorded using the Selenium IDE

To know about the commands available in the Selenium IDE, download the IDE and add it as an extension to the Firefox browser, after which, it will be available from under the Tools menu in Firefox.

In order to record test cases using the IDE:

1. Start the Firefox browser.
2. Go to the Tools menu of the browser and select Selenium IDE to launch the IDE.
3. In the Base URL field of the IDE, enter the URL of a Website on which user actions are to be recorded for automated testing purposes.
4. Click the red Record button on the top right side below the Base URL field.
5. Record the script for the test cases/test scenarios as per the test plan.
6. A Base URL field to enter the URL of the site to be tested.
7. A Record button to record the tests.
8. A Replay button with two options – one for replaying a particular test-case and one for replaying the entire test-suite.
9. A Pause/Resume button to pause or resume the tests under execution.
10. A Step button for step-wise debugging of the test cases.
11. A collapsible test case list area, test case tab area and test case detail area with Table and Source modes.
12. Text columns such as Command, Target and Value.
13. Log, Reference, UI-Element and Roll-up tabs at the bottom for reference.
14. Table mode, and
15. Source mode.

Selenium IDE has the following components to record and replay test scripts:

Let’s see a simple example of the test script generated using the Selenium IDE. The script displayed in the IDE has two modes:

Table Mode

This mode displays three column lists – Command, Target and Value, as shown in the previous screenshot. The Command option has recorded commands for the events as per the test cases while recording. The Target option has the target to be tested i.e., text, XPath, DOM and CSS. The Value option has the value for the command and the target for each event. We can also have text in the command line as a comment. The Table mode has three text fields, one each for the command, target and value. The Command field is an auto-complete dropdown used to enter and update/enhance the commands. The Target field is used to enter and update/enhance the value of the XPath/DOM/CSS object. The Value field is used to enter the specific value required for the commands recorded.

Source Mode

By default, the recorded script is displayed as HTML under the Source mode. However, the script can be exported in any supported/required target language for the Selenium RC framework of your interest. To do this, go to Options > Format and select the supported language for your framework from the available (configured) options, as shown in the following screenshot.

However, Selenium-IDE does not directly support:

• Conditional statements and iterations.
• Error handling, particularly unexpected errors.
• Database testing.
• Test case grouping and re-execution of failed tests.
• Test case dependency.
• Logging and reporting of test results.
• Capturing screenshots of test failures.

Although these tasks are not supported by Selenium directly, all of them can be achieved by using programming techniques with a language-specific Selenium RC client library – a subject that we shall look into in more detail in the third and concluding part of this blog.

Basics of Selenium RC

Selenium RC starts a Web server that provides APIs for client libraries as well as Selenium Grid – a tool which transparently distributes your tests over multiple machines allowing parallel execution of the test cases increasing the overall testing efficiency. Selenium-RC uses the full power of programming languages to create more complex tests like reading and writing files, querying a database, and emailing test results.

Selenium RC comes in the following two parts:

1. A server which automatically launches and kills browsers, and acts as a HTTP proxy for Web requests from them, and
2. Client libraries for your favorite computer programming language.

In order to execute the Selenese code in the programming language environment, first install the environment for that language. We will be using Java, so install Eclipse for Java, install JUnit and then install Selenium RC. The detailed steps are outlined below:

1. Download and unpack the Selenium RC archive from Selenium RC.
2. Set up a programming project for the Java client driver, as listed below:

• Download Selenium-RC from the Selenium HQ downloads page.
• Extract the file selenium-java-client-driver.jar.
• Open your desired Java IDE (Eclipse, NetBeans, IntelliJ, Netweaver, etc.)
• Create a new project.
• Add the selenium-java-client-driver.jar files to your project as references.
• Add to your project classpath the file selenium-java-client-driver.jar.
• From Selenium-IDE, export a script to a Java file and include it in your Java project, or write your Selenium test in Java using the selenium-java-client API. The API is presented later in this blog. You can either use JUnit, or TestNg to run your test, or you can write your own simple main() program. These concepts are explained later in this section.
• Run Selenium server from the console.
• Execute your test from the Java IDE or from the command-line.
  • Run console and go to the selenium-server folder.
  • Type: java -jar selenium-server.jar on the above path.

3. Install the Selenium-RC Server:

You may configure the Selenium server based on the java -jar selenium-server.jar –help help file. Now, save the sample code from our example above in Java format in your workspace directory and open it in Eclipse. A snapshot of the code that can be exported in Java for JUnit and TestNG looks like the one shown below:

package com.example.tests;
import com.thoughtworks.selenium.*;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.util.regex.Pattern;

public class Sample extends SeleneseTestCase {
@Before
	public void setUp() throws Exception {
	//code to start browser test
	}
@Test
public void testSample() throws Exception {

//Open the url

//code to verify the logo on the page

// code to get the auto-complete list
  	selenium.click("query");
	selenium.waitForPageToLoad("30000");
	selenium.type("query", "san");
	selenium.waitForPageToLoad("30000");
	selenium.keyDown("query", "40");
	selenium.mouseDown("query");
	selenium.waitForPageToLoad("30000");

//Select the item from the list

selenium.click("//div[@id='destination_selections']/ul/li[8]");
	selenium.waitForPageToLoad("30000");

@After
// code to stop the test case and browser
}

The autocomplete feature of AJAX

In the above example, we have automated the auto-complete dropdown scenario, which works like this:

Enter the prefix of the name of a city in the text field, e.g., “san” (as in “San Francisco”). The auto-complete list of cities is then displayed for “san”. Click on one of the items from the dropdown list. The clicked item will be given as an input to the field to display the next page. Verify a few items on the page to check whether the page displayed is indeed for the input which was given.

Conclusion

In part two of this blog, we saw how to test AJAX components in Selenium with a code example. We also saw the autocomplete feature of AJAX – a useful tool for testing textbox fields and dropdowns on Web pages.

In the third and concluding part, we shall see some advanced features of Selenium RC along with selense, and  wrappers for the Selenium engine.

Introduction

Within the software industry, once a product reaches the stable manual testing phase, every organization usually thinks of automated testing to save on the costs involved in manual testing. Since testing cost is an important factor for any project, organizations have started preferring open source test automation tools (which have reached a stage where they now rival the commercial ones) instead of investing in costly commercial testing tools. With no licensing costs, open source automation testing tools provide competitive features for automating the testing of software applications as well as Web portals.

A variety of open source automation testing tools is available for almost all types of testing such as functional, Web, UAT, regression, performance etc. Because of the extent to which these open source tools have matured, it’s time to think about them and have them in your QA automation kit. There are also various open source tools available to support the different testing types such as White Box Testing (Unit Testing, for e.g., using JUnit) and Black Box Testing (system/regression testing, for e.g., using Selenium, Sahi, Watir, TestMaker, LogiTest, TestGen4J, FitNesse etc).

The scope of this blog is to cover the basic flow for one of these open source Web testing automation tools – Selenium.

Why Selenium?

Selenium is probably the best option for automated testing of Websites today. It is becoming increasingly popular and it is the first choice of automation testers as well as organizations for automating the testing of Web-based applications for both the GUI as well as the functionality. Selenium can also be used as a unit testing tool for JavaScript. We at Xoriant have been working on Selenium for automating the testing of Web sites developed using AJAX for the pharmaceuticals and travel industries. The following graph shows the popularity of Selenium along with other open source automation testing tools.

Selenium Test Package

Selenium is a package of various test components which consists of the following three major tools. Each one has a specific role in aiding the development of test automation for a Web application.

1. Selenium IDE – A Firefox extension to record test cases and suites.
2. Selenium RC – Used to run tests on different browsers and systems.
3. Selenium Grid – Runs multiple instances of Selenium RC at once.
4. Qualitia and Tellurium – A wrapper for the Selenium engine.

Selenium Modes:

Bases on the components Selenium has following three modes for executing the test cases and test suites:

• Record-Playback mode (Selenium IDE)‏:

In this mode only Selenium IDE is used to record the test scenarios in terms of test cases in firefox. This is a great way to get started to writing tests and group them together to form the test suite. The recorded tests can be exported to many programming languages so that we can tweak them and put them in the testing framework. The test cases and test suites can be replayed back to check the verifications and validations or sent to Selenium RC or Grid for further tweaking.

• Selenium Remote Control (RC) Mode

In this mode Selenium starts multiple browsers (one at a time) and then runs the recorded test-cases which are saved in your language of choice. This helps to enhance the test cases with looping and programming techniques to cover all the required test scenarios and checks.

• Test Runner Mode

In this mode the test cases are recorded ad replayed in the form of HTML tables. This is just one more facility to execute the Selenium IDE as well as RC test cases. This helps to check the test results reports in better manner if not formatted already.

Testing AJAX with Selenium

Selenium IDE – Recording and updating a script

Selenium IDE is the FireFox Add-on provided by the Selenium group. This is very simple and easy to use add-on so that non-programmers can record and create the test scripts for automating web components. These automated test scripts are used as Selenium RC test cases by choosing the language code. i.e. Selenium IDE makes easier to create Selenium RC test cases. (Install Selenium IDE and Selenium RC from Selenium download page).

Selenium IDE is used for:

• Recording and updating the test cases or write them manually in table tab.
• Creating Test Suite by grouping the test cases under one group
• Exporting Test Cases/Suites the supported language and save for Selenium RC to enhance them
• Finding reference of every API and Selense commands
• Debugging test cases by toggling breakpoints through the commands

How commands generated in IDE for AJAX pages:

1. Almost all of the web pages are developed with AJAX technology since web2.0 evolution. While recording the page elements, Selenium IDE locates the page elements/objects using XPath/DOM/CSS. XPaths are found automatically, but you may have to update the specific elements in Xpath manually by looking into the source of the page to make them unique. Firebug is the helpful firefox Add-on for this purpose.
2. Every command has maximum two arguments. The first is usually target element pointed by its ID, name, link, DOM, CSS or XPath and the second is the value to that target element if exists. The selenium IDE commands when saved in the programming language of our choice for selenium RC purpose are called as Selenese.
3. The common commands are open, click, type, assert, verify. Commands beginning with ‘assert’ or ‘verify’ are for checks and verifications. If verify command is used and it fails, it fails the test case and continue to run the remaining tests. If assert command is used and it fails, the test case is aborted and stops to run the remaining tests.
4. For synchronizing the flow of test execution Selenium has commands ending with ‘AndWait’, it tells Selenium to wait for the page to load after the action has been done. However, it fails when using AJAX calls. In this case you have to use ‘WaitFor’ commands.

Selenium IDE – Most commonly used commands

The following are the most commonly used commands in the Selenium IDE:

open: Opens a page using a URL.

click: Clicks the element/object on the page.

clickAndWait: Performs a click operation, and optionally waits for a new page to load.

verifyTitle: Verifies the expected title and continues to run if it fails.

assertTitle: Verifies an expected page title and stops the execution if it fails.

verifyTextPresent: Verifies that the expected text is present somewhere on the page.

verifyElementPresent: Verifies an expected UI element, as defined by its HTML tag.

verifyText: Verifies that the expected text and its corresponding HTML tag are present on the page.

waitForPageToLoad: Pauses execution until an expected new page loads.

waitForElementPresent: Pauses execution until an expected UI element, as defined by its HTML tag, is present on the page. Used with AJAX calls.

Get more details about Selenium IDE here: Selenium IDE documentation.

If you need a full reference of all Selenium commands, read the Selenium commands.

Conclusion

In the first part of this blog, we have covered the basics of a typical test setup required for Selenium for automated testing of a Web-based application.

In the next part, we shall see automated testing in action – i.e., a practical example of how to actually use the Selenium IDE and RC to execute the tests. The reader is advised to check out the links highlighted in this blog to build the foundation needed for part two.

Overview

Today, non brick-and-mortar companies such as Google, Amazon and Yahoo! generating revenues in billions of dollars is quite well-known. What may not be so well-known, though, is the level of investment these companies have to make in ensuring that their systems and servers are always available to their customers 24×7, and to make sure that their data centres are able to handle the amount of traffic needed to be processed to keep their business going.

Little wonder then, that testing forms an important part of managing their business infrastructure.

Introduction

This blog is intended to explain to the reader the most important factors that are involved in the testing of high-traffic Web portals. For such Websites, security and performance become the key factors in testing along with other concepts such as usability, functionality and navigation.

Unit Testing

Unit testing must be performed by the developers against their components prior to integration. Custom unit tests only exercise functionality but neglect performance, which becomes a bottleneck later; hence component performance needs to be analyzed during their unit tests. This is usually done using the following tools:

• Memory profilers
• Code coverage and coding standard tools

Usability Testing

Usability testing helps to create and maintain a user-friendly and user-centric portal that a visitor will find easy to use and will want to visit again. Some of the usability practices tailored for high traffic Websites are listed below:

• User friendly layout: Use language and concepts which are familiar to the user. Try to present information in a sequential and logical order.
• Maintain consistent conventions: Concepts which are similar should be shown using similar terminologies and graphics. Details of maintaining uniform conventions for aspects such as layout, formatting, typefaces, labelling, etc should not be ignored.
• Minimize users’ memory usage: Try not to force visitors to remember important information across multiple documents.
• Using a flexible and efficient design: Try to arrange screens in a way such that frequently required data is found easily, providing instructions and directions wherever necessary.
• Keeping an aesthetic yet simple design: Do not present or display information which may be distracting or irrelevant.
• Arrange a progressive level for details: Information should be organized such that generic information is presented first, followed by more details, as requested or searched for.
• Provide navigational feedback. Moving between related topics should be made easy. The user should be allowed to find out where he/she presently is on the website (by providing a site-map) and it should be easy to return to any previously visited point on the site.

Globalization

For high traffic portals, there is increasing demand for supporting local languages as the end user base is increasing from every corner of the world. Globalization testing is the process of testing the Website which has to work uniformly across multiple regions and cultures. There are two aspects to an international website: world readiness (globalization), and localization. World readiness refers to the process of designing, coding and testing the website such that it can be easily localized for different regions. Localization involves translating and customizing the products for different regions. Ensuring world-readiness is different from testing localized program versions, and it is broader than just functionality testing. It also includes realizing the implications of globalization, plus verifying that those implicit requirements are met throughout all the design and development steps.

AJAX Bridging

It is well known that AJAX applications can only connect back to the website from which they have originated and this is a security measure in AJAX. For instance, JavaScript code developed with AJAX and obtained from a certain site cannot launch a connection to another site. The AJAX service bridge was developed to enable a third-party Website connection in this way.

This is how it works: Inside the bridge, first, a host provides a Web service acting as a proxy and allowing traffic forwarding between the JavaScript code running on the client and the third-party Web site. Since a bridge can be considered as a ‘Web service to Web service’ type of connection, an attacker can use this to gain access to sites with restricted access. Hence, it is required to check whether the website is vulnerable to attacks or not.

HTML/AJAX Injection

The heart of AJAX is XMLHttpRequest which allows for synchronous server communications and browser updates. The browser can be updated with just simple HTML (DOM), XML, or another structured data format. These XMLHttpRequest calls are just normal HTTP requests. Check if every request for authentication checks for the behavior of application after introduction of HTML injection. The same security vulnerabilities and controls apply to AJAX websites.

• AJAX often requires additional or stronger controls because they are usually complex, bidirectional, and asynchronous.
• AJAX applications often have weak authentication, session management, and error handling.

Cross Site Scripting

Additionally, check the Web application for XSS (Cross site scripting). Any HTML, such as <HTML>, or any script such as <SCRIPT> should not be accepted by the application. If it is, the application can be prone to an attack by Cross Site Scripting.

Cross-Browser Compatibility

Cross-browser compatibility generally is a big issue. It is magnified a few times when you consider high traffic Websites with AJAX. Browsers either do not support AJAX or render AJAX differently. More stress on browser compatibility testing of high traffic Websites comes when you consider different browsers with different versions such as Internet Explorer, Netscape, Firefox, Safari, Opera and Chrome.

Security Testing

Because AJAX is still a recent technology, several of its security issues are still to be fully understood. AJAX must not be deployed in security-sensitive applications, such as credit card verification portals. Some of the security concerns to be considered while designing the test cases while using AJAX are listed below:

• It increases the available attack surface and requires many more inputs to be secured.
• It exposes the internals of the application.
• It enables access to third-party resources for the client without any built-in security and encoding mechanisms.
• It has no mechanisms to avoid failures in protecting authentication information and sessions.
• The line between client and server-side code is highly blurred, which results in security mistakes.

Testing Search Engine Optimization

Testing teams of such high traffic portals need to deeply understand the basics of search engine optimization (SEO). This is all the more important, especially because Website technologies like AJAX and Flash are not search engine friendly. Listed below are some important SEO items that the testing teams should be aware of:

• Ensure that static URLs are used widely and IP addresses are used only when absolutely necessary.
• Ensure that there is no excessive use of tables which is considered a bad SEO practice.
• Ensure that non-compliant HTML is not used.
• Ensure that images have ‘ALT’ and ‘Title’ attributes.
• Ensure proper use of ‘Meta’ tags and keywords.
• Ensure the presence of an ‘Admin’ screen to dynamically update keywords based on feedback from search engines.
• Check hyperlink updates for use of domain name instead of IP address.
• Avoid excessive use of JavaScript based menus which have a negative impact on SEO.
• Ensure that all searchable text is available to search engines.
• Test the Website on all search engines for which the site is optimized.

Special Test Cases for Websites Heavily Loaded with AJAX and Flash

• Absence of an IDE for AJAX makes unit testing and debugging a discouraging task. Hence, excessive dependency is always there on the regression testing teams who need to factor this and design the test suite more elaborately and cover each single unit for all its possible functionalities.
• High traffic Websites with AJAX will not work if JavaScript is disabled. Make sure that the user is prompted to enable JavaScript and even if the user doesn’t do so, the site should not become totally inaccessible with JavaScript turned off.
• The ‘Back’ button on the browser may not work until developers provide that functionality. Testing teams need to ensure that this functionality is available on pages where there is a high probability of users hitting the ‘Back’ button.
• It may be difficult to bookmark a particular section of an AJAX site. Testing teams must ensure that pages users may want to bookmark are free from this limitation.
• The Flash based interface must be tested on different speeds such as modem, broadband, LAN etc.
• The audio and video synchronization for flash movies and animations must be checked.
• Finally, the Flash/AJAX application on the machine with minimum configuration must be tested.

Bandwidth Dependency

Both AJAX and Flash have a heavy bandwidth dependency and do not work well on low bandwidths. Hence, testing the response on varying bandwidths to determine and ensure the comfort band is an important requirement. This adds a whole new dimension to testing such sites, and demands additional discipline, time and resources.

Performance Testing

For such high traffic Web portals, thousands of requests are usually being made to the server and during particular events and times, this count can go in millions. With such a high volume of traffic, server performance may become an issue and hence to ensure satisfactory performance, the following criteria should be met during performance testing:

• It must be ensured that your site’s performance requirements and goals represent the needs of your users.
• Load tests must be designed such that they replicate the actual workload at both normal and peak times.
• Performance testing must be conducted using parameters such as data types, distributions and volumes which are similar to those used in actual business operations under actual production.

• Whether the system can handle multiple concurrent requests performing the same transactions must be checked.
• To properly identify the bottlenecks during performance testing, using a good and optimised profiling tool is also important.
• Automated testing tools such as LoadRunner, Keynote and JMeter should also be used for evaluating various performance aspects.
• Time-critical transactions in the performance tests must be included.
• Even under peak traffic conditions, it must be ensured that the user can transfer large volumes of data.
• The performance tests must also be conducted while regular system processes, such as a batch job execution or a virus definition update download, are in progress.
• The performance must be measured under various load levels and mixed scenarios.
• It has to be validated that all of the correct data was displayed and saved during your performance tests.
• An important testing technique, known as Soak Testing, also has to be executed. This involves testing a system with a significant load extended over a significant period of time, to discover how the system behaves under sustained use.

Summary of the Factors Important in Testing High-traffic Web Sites

Standard testing methodologies must be adapted to the unique nature of Website testing environments. The following table lists some of the key aspects that testing teams should consider while designing test strategies for high traffic Web sites:

Type of Check	Parameters to be Validated
Validation checks	Validating HTML, input forms, check for broken links, validating CSS (such as fonts, colors, spacing etc)
Security checks	Testing cookies
Performance checks	Testing bandwidth dependencies
Browser compatibility checks	Using different browsers such as Internet Explorer, Netscape, Firefox, Safari, Opera and Chrome
Other checks	Testing the sever side interface, configuration, recovery and accessibility

Phases in Testing Applications Running on such High-traffic Web Portals

To ensure that the application is able to withstand the high volume of traffic, it must go through the following phases during specific points in the software development lifecycle:

• Unit testing
• Integration testing
• Functional testing
• End-to-end testing
• Distributed testing
• Usability Testing
• Globalisation testing
• Exploratory testing
• Regression testing
• Compatibility testing
• Conformance testing
• Fault tolerance and security testing
• Load and stress testing
• Performance testing

Summarizing, we have seen in this blog that testing high-volume portals requires careful planning at every stage, starting from the site’s design right up to considerations such as the end user’s bandwidth availability.

This blog will focus on the best practices to be followed for Internationalization & Localization testing of web based applications.

Introduction

Internationalization testing is the process of testing software which is supposed to work uniformly across multiple regions and cultures. There are two of the major aspects of international software – world readiness (globalization) and localization. World readiness refers to the process of designing, coding and testing the product such that it can be easily localized for different regions. Localization involves translating and customizing the product for different regions.

Ensuring world readiness is different from testing localized versions of the product, and it is broader than just testing the functionality. It also includes realizing the implications of globalization, and verifying that those implicit requirements are met throughout all the design and development steps.

Internationalization testing

The goal of Internationalization testing is to detect potential problems in software globalization. It makes sure that the code can handle all international support without breaking functionality that would cause either data loss or display problems. Globalization testing checks proper functionality of the product with any of the locale settings using every type of international input possible. There are two basic testing requirements which ensure that the product works well in all regions for which it was developed:

• Testing to check if the product is locale aware (globalization).
• Testing the localized versions of the product.

The following diagram depicts the breakdown of the product with respect to internationalization and localization testing:

Best Practices in Internationalization Testing

• Feature based Testing

The product may be marketed with some features applicable and others not applicable to specific cultures. While testing, it is important to ensure that these features are turned on or off as required while switching locales.

• Testing for Localized content

Localized content includes both text and graphic artifacts on the user interface. Textual content to be localized includes the static text on controls such as menus, buttons, etc. and user specific messages. Language experts are required for testing the localized content in each language that the product supports.

• Testing for culture awareness

User locale settings define items such as number, date and currency formatting. Input locales should be handled and the user be allowed to input data in that language.

In many cases, data would need to be converted from one encoding format to another. Correct handling and understanding of encoding formats is required in such cases. If coded incorrectly, in addition to operational errors, it can also result in loss of data. For example, converting from a multi-byte character set to a single or double-byte character set can result in data loss if not handled properly. User locales do not impact conversion of data from one encoding format to another.

• Testing rendering

Internationalization testing should check whether an internationalized product properly displays all supported scripts in accordance with the linguistic characteristics associated with them. The characteristics include bi-directionality, character reordering, contextual shaping, combining characters and special rules in terms of word breaking, line breaking and text justification.

• Testing user interface

Internationalization testing should cover whether the user interface, on which the fonts are rendered, supports the different fonts and character widths as well as the varying spacing requirements introduced by translation. For example, it is a well known fact that German text occupies more space to convey the same information when compared to English. Thus, the user interface should be adaptable to accommodate user messages in both languages.

Following are the recommended guidelines for user interface testing:

1. Check if textbox labels appear on top of the textboxes in order to handle text with larger widths. If it is still required to have the label beside a textbox, enough room should be left for the text to grow.
2. Check if dialog boxes expand while localizing. Additional space (ideally, 30%) should be left between the end of messages and the edge of dialog boxes for further expansion.
3. Check the text embedded in images and icons. Having text in images will make localization difficult and require the efforts of a user interface designer/tester.
4. Buttons and labels should have a larger width to accommodate for longer text in non-English languages.
5. Images embedded within resource files of forms should be moved out as separate image files.
6. Mirroring: Mirroring is the process of localizing the user interface to handle right to left (RTL) languages such as Arabic, Hebrew, Farsi, etc. To give a perfect RTL look and feel to an application’s UI, both the text and the UI elements need to be laid out from right to left once they are translated into RTL languages. A complete discussion on Mirroring is not within the scope of this paper.
7. Graphical images: Graphics can be difficult and expensive to translate. Hence it is preferable to have graphics which are universally acceptable.
8. Following are some of the key user interface elements that need to be localized:
   1. Menus
   2. Messages
   3. Dialog boxes
   4. Images
   5. Toolbars
   6. Status bar

• Storage testing

Internationalization testing should also cover data storage and retrieval in multilingual applications, which warrants careful attention. The key sources of storage include files and databases.

• Testing File Transactions

While reading from a file which contains Unicode characters, it is important to specify the encoding. If the encoding is not explicitly specified, a default encoding may be used (UTF-8 in case of .NET framework) resulting in misinterpretation of data. For example, if a text file stored in UTF-16 is read without specifying the encoding, .NET will interpret the UTF-16 characters as UTF-8 resulting in unintelligible input. Please note that ASCII and UTF-8 are interoperable if the file contains only English characters having code point less than 128

• Database Testing

Testing should cover the support of Unicode characters in databases; special data types are used which are defined for this purpose. For example, SQL Server defines nchar, nvarchar and ntext data types to allow you to store Unicode text. (The n prefix for these data types comes from the SQL-92 standard for National (Unicode) data types). Use of nchar, nvarchar and ntext in SQL Server 2000 is the same as char, varchar, and text, respectively, except that:

• Unicode supports a wider range of characters.
• More disk space is needed to store Unicode characters.
• The maximum size of nchar and nvarchar columns is 4000 characters and not 8000 characters as for char and varchar.

Unicode constants are specified with a leading N (for example, N<a Unicode string>). The following example shows how to insert Unicode values (Russian) into an nvarchar column.

Once you have defined a Unicode compatible column in the database, inserting and retrieving multilingual data is the same as working with regular (ASCII) data. In database migration, care should be taken while mapping the source and target data columns. If the source data type is nchar, and the target data type is char, data could be lost upon migration.

In summary, there are several aspects to be kept in mind while designing software for a global audience. These aspects cover not only coding and testing, but also the interface and layout of the software as well as factors such as the file formats and data encoding to be used.