Archive for the ‘Testing’ Category

HOW TO MANAGE TEST CASES AND BUGS USING JIRA

HOW TO MANAGE TEST CASES AND BUGS USING JIRA

My determination of writing this blog is to describe how we can use JIRA as a TC (Test cases) and bug management with the help of add-on JIRA Cli. Also, this document will help in analyzing the benefits & drawbacks of using JIRA as a test case management.

Previously, I had to depend on an Excel spreadsheet for logging test cases and JIRA for tracking bugs in my project. I needed a tool by which we can accomplish both test cases and bugs with using a single tool. Now I am using JIRA for Test cases and bug management in spite of some constraints as outlined in the later part of this blog.

Modification in JIRA

Modifications are required to use existing flow of JIRA as a Test Case  management with the help of a JIRA administrator. Below are the steps to modify the existing JIRA workflow:

  • Create/Add new Issue type
  • Create/Add new Status
  • JIRA default workflow
  • Test case work flow after modification
  • Status in Test case workflow

Here are additional detail explaining each of the steps below:

How to create issue type?

Add a new issue type in existing flow and name it “Test case.” This should be a top- level (standard) task (for this go to Administration > Issues > Issues Types and use  Add Issue Type option).

How to create status?

Add new status (for this go to Administration > Status > Status Types and use Add    Status Type option)

JIRA default workflow

JIRA workflow is the set of status and transitions through which an issue goes during its lifecycle.

JIRA default workflow 

Test case work flow after modification

After modification with the help of JIRA administrator below is the Test case workflow with different status:

Test case work flow  

Status in Test case workflow:

TC awaiting review: Default status when a Test Case is created and awaiting for Review from the QA lead.

TC Template: An approved TC that has not yet been scheduled yet.

TC-Ready to Run: TC that has been scheduled.

In Progress: TC that has been in-progress state

TC-Failed: After execution when TC fail status update to TC-Failed

TC-Passed: After execution when TC Pass status update to TC-Passed

TC-Invalid: A test that has become obsolete or cannot be run

Closed: This is the final status of the TC.

About JIRA CLI

One of the major challenge was how to import the TCs in JIRA because designing test cases one by one in JIRA was time consuming. A tool was needed by which I can upload the TC in JIRA from file (.csv, Excel, etc.). Finally, I found Add-on JIRA CLI (free tool ~Till version 2.6.0) to import the Test case’s from .csv

How to import Test case using JIRA CLI?

-Download jira-cli-2.6.0 from this link

-Save the Test cases file with name import.xls and convert into .csv format then copy import.Csv file under downloaded folder (jira-cli-2.6.0) .csv format given below

JIRA
~In above import.csv file Description column consist  merge data of (Prerequisites, Steps, and Expected Results)

-Under downloaded folder (jira-cli-2.6.0) there is a file with name import.properties

-Map column name for which we want to import the data in the properties file

field.Type = Type

field.Component = Components

field.Project = Project

field.Objective = Summary

field.Description = Description

File (~import.properties file format as seen above)

-Run the following single line command in command prompt

c://Jira-cli-3.8.0>jira.bat –server <URL of JIRA> -user kailash.pathak – password “xxxxxxx” –action runFromCsv –file import.csv –propertyFile import. Properties –common “–project SSPQA”               – continue

(~Here “SSPQA” Is Project Key)

-Test case data is imported in JIRA against the particular components successfully

Process for Test case creation and allocation in JIRA

Below are the steps for test case creation and allocation in JIRA:

Test Case creation steps:

  • Create a new project with the help of JIRA administrator.
  • Create sprint Task, sub task for QA e.g. Design Test case and Execute Test case of particular module    
  • Create Components against the Project with the help of JIRA administrator
  • Create Test Case In Excel
  • Convert TC in .csv file with giving name as import.csv (its mandatory csv with this  name) for to import in JIRA. csv file format attached above
  • Upload .csv into JIRA using JIRA CLI using the (above mentioned steps)

Create Test Case in excel sheet in the format which is acceptable by JIRA CLI (see below for example)

JIRA 

Test Case Allocation steps :

We can allocate Test Cases for execution using bulk change functionality. During the bulk change we can assign the user name to the Test Cases.

First search the TCs then assign user name in bulk by clicking on Assign to me link see below screen shot for more detail

JIRA-KP-3 

 

JIRA-KP-4
Test Case Execution

Once the TCs are allocated next step is to execute the Test Cases in JIRA with updating their status with Pass, Fail etc.

Prerequisites:

  • We are executing TC in three browsers (Chrome, IE, and FireFox)
  • We are executing each of TC in two cycles (Cycle 1, Cycle 2)

After uploading and allocating the test cases next step is to execute the test cases. Follow these steps to execute it:

  • Bring all Test cases we want to execute in “IN PROGRESS” state using Bulk change functionality
  • Execute TCs one by one

In case of Test case FAIL

  • In case of a FAIL a New Transition comments screen (See screen shot below) open
  • Enter actual result.
  • Enter Sprint<Sprint Number>-Cycle< Test Cycle No. >-<Browser Name> <Fail>

E.g. Sprint26-Cycle1-IE-Fail as Comment

  • Click on the Fail button
  • Particular test case is Fail
  • Report bug for Fail Test Case
  • Link Fail Test Case with Bug Number (Explain in next point)

JIRA-KP-5  

Linking of Fail Test case with Bug ID

To link the test case with Bug ID Open Particular Failed TC and Click on “Link” from menu following screen open to link the Test Case with Bug ID after clicking on link button Fail TC link with the Bug ID

JIRA-KP-6 

In case of Test case PASS

If Test Case Pass in first time or in second cycle in that case update the comments with Pass status and with respective Cycle Below the screenshot when TC pass in second cycle

JIRA-KP-7 

*Note:

Entered above comments help us to pull out the report.

In case of Test case INVALID

In case of invalid test case that has become obsolete or cannot be run. We will update the status as shown below:

*Note:

Same transition screen open to enter comments in case of other status (Closed, TC-Template, TC-Ready to Run etc.)

Report in JIRA of Test Cases

We can generate execution report using different criteria (We can use the entered comments during execution of TC as search criteria).

Search Criteria:

Sprint Wise Report

    1. Complete Sprint wise report (Sprint26-*)

    Example:

Enter Sprint26-* in the Comment window after selecting Project, Components as shown in the screen below:

JIRA-KP-10 

End Result:

This search criteria displays all executed TCs (Pass, Fail, Invalid etc.)  with a consolidated list as shown below:

We can convert this report data (in Two Dimensional and Pie Chart illustrations as well)

See two dimensional and pie chart report for the above documented criteria

JIRA-KP-11 

*Note:

For each of the below mentioned search criteria we can create a similar report

2. Sprint with cycle wise report Sprint26-Cycle1-*

End Result:

This search criteria display all executed TC of Cycle1

JIRA-KP-12We can convert this report data in (Two Dimensional and Pie Chart

3.Sprint with Cycle and Browser wise (Firefox, IE, Chrome) report

  • Search by criteria Sprint26-Cycle1-FF-*
  • Search by criteria Sprint26-Cycle1-IE-*
  • Search by criteria Sprint26-Cycle1-Chrome-*

End Result:

This search criteria displays all executed TC of Cycle1 for different browser

We can convert this report data in (Two Dimensional and Pie Chart)

4. Sprint with (Cycle, Browser and Test case Pass) wise report

Search by criteria Sprint26-Cycle1-IE-Pass

End Result:

This search criteria display all Passed TC of Cycle1 for Browser IE

We can convert this report data in (Two Dimensional and Pie Chart)

5. Sprint with (Cycle, Browser and Test case Fail) wise report

Search by criteria Sprint26-Cycle1-IE-Fail

End Result:

This search criteria display all Failed TC of Cycle1 for Browser IE

We can convert this report data in (Two Dimensional and Pie Chart)

*Note:

Similarly we can generate report for Cycle2 #

JIRA – Pros

  • Bulk clone is possible
  • Possibility to update the status of multiple test cases at once
  • Custom searches can be saved as filters and shared through products/users
  • Test plan can be easily create in JIRA by putting together all test cases to be executed in that iteration (Sprint)
  • Easily linking between Test Cases and Bugs
  • Provides an advanced search mechanism and a way to create custom search queries

JIRA – Cons:

  • The “look and feel” is the same as for bugs not as for a normal test case management tool (like Test Link or Quality Center)
  • Initial set up is bit time consuming but once design user can easily manage the Test Cases

 Conclusion

  • With this blog, we have understood how we can use JIRA as a single tool to create, execute Test Cases and bug management.
  • We have also given a brief overview of add-on JIRA Cli and its uses when it comes to importing the test case in JIRA.
Advertisements

How to create test suite using Junit and eclipse in selenium

How to create test suite using Junit and eclipse in selenium

There are some scenarios where we need to run multiple test cases. Either we can run those test cases independently or together. But there are some real time cases where we need to run our test cases in a particular order. In this case we would prefer Test Suite to combine the test cases together and decide their orders and run those.

Below are the steps

1. Create a Test Suite class where we create the Test Suites which will call all the test cases in a particular order.


import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;  
import junit.textui.TestRunner;

public class TestSuite1 extends TestCase {

public static Test suite()  
{  
TestSuite suite = new TestSuite();  

suite.addTestSuite( TestCase1.class);  
suite.addTestSuite( TestCase2.class);
suite.addTestSuite( TestCase3.class);  
return suite;  
}  

public static void main(String arg[])
{
TestRunner.run(suite());

}
}

Step 2. Create your first test case


import org.openqa.selenium.server.RemoteControlConfiguration;
import org.openqa.selenium.server.SeleniumServer;
import com.thoughtworks.selenium.*;
public class TestCase1 extends SeleneseTestCase{

Selenium selenium;
public static final String MAX_WAIT_TIME_IN_MS="60000";
private SeleniumServer seleniumServer;

public void setUp() throws Exception {

RemoteControlConfiguration rc = new RemoteControlConfiguration();
rc.setSingleWindow(true);
seleniumServer = new SeleniumServer(rc);
selenium = new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com/");
seleniumServer.start();
selenium.start();

}
public void testgoogling() {
selenium.open("/");
selenium.type("q", "Niraj");
selenium.click("btnG");
selenium.waitForPageToLoad(MAX_WAIT_TIME_IN_MS);
assertTrue(selenium.isTextPresent("Niraj"));

}

public void tearDown() throws InterruptedException{
selenium.stop(); 
seleniumServer.stop();
}
}

Step 3. Create your second test case

import org.openqa.selenium.server.RemoteControlConfiguration;
import org.openqa.selenium.server.SeleniumServer;
import com.thoughtworks.selenium.*;
public class TestCase2 extends SeleneseTestCase{
Selenium selenium;
public static final String MAX_WAIT_TIME_IN_MS="60000";
private SeleniumServer seleniumServer;
public void setUp() throws Exception {

RemoteControlConfiguration rc = new RemoteControlConfiguration();
rc.setSingleWindow(true);
seleniumServer = new SeleniumServer(rc);
selenium = new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com/");
seleniumServer.start();
selenium.start();
}

public void testgoogling() {
selenium.open("/");
selenium.type("q", "Niraj Kumar");
selenium.click("btnG");
selenium.waitForPageToLoad(MAX_WAIT_TIME_IN_MS);
assertTrue(selenium.isTextPresent("Niraj Kumar"));

}

public void tearDown() throws InterruptedException{
selenium.stop(); 
seleniumServer.stop();
}
}

Step 4. Create your third test case

import org.openqa.selenium.server.RemoteControlConfiguration;
import org.openqa.selenium.server.SeleniumServer;
import com.thoughtworks.selenium.*;
public class TestCase3 extends SeleneseTestCase{
Selenium selenium;
public static final String MAX_WAIT_TIME_IN_MS="60000";
private SeleniumServer seleniumServer;
public void setUp() throws Exception {

RemoteControlConfiguration rc = new RemoteControlConfiguration();
rc.setSingleWindow(true);
seleniumServer = new SeleniumServer(rc);
selenium = new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com/");
seleniumServer.start();
selenium.start();
}

public void testgoogling() {
selenium.open("/");
selenium.type("q", "http://www.automationtricks.blogspot.com");
selenium.click("btnG");
selenium.waitForPageToLoad(MAX_WAIT_TIME_IN_MS);
assertTrue(selenium.isTextPresent("http://www.automationtricks.blogspot.com"));

}

public void tearDown() throws InterruptedException{
selenium.stop(); 
seleniumServer.stop(); 
}
}

Step 5. Run your Test Suite
Go to your Test Suite class and right click on that and run as Junit Test.

This will run the TestCase1 then TestCase2 then TestCase3

If you want to execute your test cases in some specific order then you call them in that order like.

suite.addTestSuite( TestCase3.class);  
suite.addTestSuite( TestCase2.class);
suite.addTestSuite( TestCase1.class);  

Above will run the TestCase3 first then TestCase2 then TestCase1.

Upgrading Selenium Standalone Server

Upgrading Selenium Standalone Server

I have started writing this blog from 3 months. When I have started writing the latest Selenium Standalone Server version available for download was 2.31.0.

Now I would like to check if there are any latest version available by following the below steps:

1. Open www.seleniumhq.org
2. Click on ‘Download’ tab to navigate to the downloads page as shown below:

3. In the Download page, view the latest version of the Selenium Standalone Server as shown below:

4. I’ve observed that there is a new version 2.32.0 for Selenium Server available for me to download  (currently I and using 2.31.0 version). So I would like to upgrade it to the latest version by following the below steps.

Upgrading to the Latest Selenium Server:

Pre-requisite: Ensure that Selenium Standalone Server is not running. If it is running, just restart the Eclipse IDE to stop it.

1. Download the Selenium Servers new version 2.32.0 and place the downloaded file in any folder as shown below:

2. In Eclipse IDE, Right click on the Project you are working with and select ‘Properties’ option as shown below:

3. Ensure that ‘Properties for Project’ dialog is displayed, select the older version of Selenium Server and click on ‘Remove’ button as shown below:

4. Ensure that the older version of Selenium Server is removed and click on ‘OK’ button as shown below:

5. Expand the libraries folder, right click on the older version of Selenium Server and click on ‘Delete’ option as shown below:

6. Click on ‘OK’ Button on the ‘Confirmation’ dialog as shown below:

7. Ensure that the older version got deleted from the libraries folder
8. Copy the latest version as shown below:

9. Paste the copied latest version into the libraries folder as shown below:

10. Ensure that the latest Standalone Server 2.32.0 got pasted into the libraries folder as shown below:

11. Right click on the Project and select Properties option as shown below:

12. Ensure that ‘Properties for Project’ dialog is displayed and click on ‘Add Jars’ button as shown below:

13. Ensure that ‘Jar Selection’ dialog is displayed, explain the Project -> libraries folder, select the latest selenium standalone version and click on ‘OK’ button as shown below:

14. Ensure that the latest Selenium Server version got added in the ‘Libraries’ tab and click on ‘OK’ button as shown below:

15. Select ‘External Tools Configuration’ option from the Run options as shown below:

16. Ensure that ‘External Tools Configuration’ dialog is displayed, change the version details in the ‘Argument’  text box and click on ‘Apply’ Button as shown below:

17. Click on ‘Run’ Button to start the ‘Selenium Server’ as shown below:

18. Ensure that the Selenium Server is running as shown below:

19. Run the ‘Class16.java’ file by selecting the ‘JUnit Test’ option as shown below:

20. Ensure that the Test got passed without any errors as shown below:

Watch the Below Video:

Click Here to watch the Video

Download this Project:

Click here to download this project and import into Eclipse IDE  on your machine.

Only one test should be written inside a test method

Only one test should be written inside a test method

1. Before going forward, lets once again view the code we have written in our previous post below: (i.e. The code with more than one tests inside a single test method)

2. As its not a good idea to write more than one tests inside a single test method, lets Refactor the Selenium Automation code by writing all the 3 tests into a separate test methods as shown below:

3. Start the Selenium Standalone Server
4. Save and Run the ‘Class16.java’ file by selecting the ‘JUnit Test’ option
5. In this case, Observe that the Firefox Browser has launched and closed for 3 times instead of once,  for performing the above three tests as shown in the below video.

Watch the Below Video:

Click Here to watch the video.

Download This Project:

Click here to download this project and import into Eclipse IDE  on your machine.

Why did the firefox launched and closed separately for each test method in the code ?

As we’ve three tests and we’ve written test method for each of our tests, hence we got 3 test methods in the Selenium Automation code. But in this case ‘chapter1’ page is common for 3 of the tests and hence its not required to close the application and launch it for performing the remaining tests. This has happened because of @Before and @After JUnit annotations.

The code inside the @Before and @After annotations will be executed once for each test method in our Selenium Automation code. Since we’ve three test methods in our code,  the methods under the @Before and @After got executed thrice (i.e. once for each test method).

How to resolve this problem ?

Suppose if we want to run the methods under the @Before and @After annotations only once, we’ve to replace these annotations with @BeforeClass and @AfterClass. (i.e. The method under @BeforeClass will be executed before executing all the test methods inside the class and The method under @AfterClass will be executed after executing all the test methods inside the class.

So to conclude the executing process will be as shown in the below screenshots:

@Before and @After:


@BeforeClass and @AfterClass:

You can research for implementation for the @BeforeClass and @AfterClass Annotations as well for confirmation.

How to Start IE9 using Selenium RC in Windows Vista or Windows 7

Selenium RC – How to Start IE9 using Selenium RC in Windows Vista or Windows 7

Issue:

When I started to work on the Selenium RC, it was running the tests against IE6 and that makes sense if the tests were written 3 years ago and code was never updated since then. Now the challenge here is to run all the tests of the projects I was working on, in Windows 7 machine instead of WinServer 2003 and running the tests against IE9 instead of IE 6.

Solution:

There are couple of simple solutions you can try to get these things done and working properly in your new environment.

1. Check your Server version. If you are using very old selenium version then its the time to get the latest Selenium Server from the Selenium Website.

2. Command Prompt. You will also have to make sure that the command prompt you are working on to run the Selenium Server is opened as “Run as Administrator“. This is one of the reasons that you might not be able to open IE latest version with Windows Vista and Windows 7 i.e. IE9.

3. Disable Protected Mode. This is an alternative trick to solve the issue and run the tests against IE9. You will have to go to IE | Internet Options | Security | Enable Protected Mode and then Uncheck the box and click on Apply and Ok. That is it and it will resolve the issue. Please go to my other blog where I have mentioned everything step by step how to do this.

Now we are good to go and run our tests against IE7, IE8, IE9 with either Windows Vista or Windows7.

How to do database testing using selenium rc

How to do database testing using selenium rc?

There are no selenium specific commands to do the database testing. But, you have the work around with native language. Here is the example how you can make database testing in selenium. The below mentioned commands will help you to begin with.

Before you use the below code You should have created the DSN name for MS SQL Server DB or Oracle DB using ODBC drivers. Here the name that we have used for the database DSN name is QT_Flight32. And the table in that database we are using is Orders. Hope the remaining steps you can easily understand.

//KEEP THE NAME OF THE PACKAGE
package com.src.test;

//IMPORT THE SQL PACKAGE
import java.sql.* ;

//DECLARE THE CLASS
class JDBCODBCTestCase
{

//WRITE THE MAIN METHOD TO CONNECT TO THE DATABASE
public static void main( String args[] )
{

try{

// LOAD THE DATABASE DRIVER
Class.forName( “sun.jdbc.odbc.JdbcOdbcDriver” ) ;

// ESTABLISH THE CONNECTION TO THE DATABASE
Connection conn = DriverManager.getConnection( “jdbc:odbc:QT_Flight32” ) ;

// GET A STATEMENT FOR THE CONNECTION
Statement stmt = conn.createStatement() ;

// PREPARE THE SQL STATEMENT
String strSQL = “SELECT * FROM Orders”;

// EXECUTE THE SQL QUERY AND STORE IN RESULTS SET
ResultSet rs = stmt.executeQuery( strSQL ) ;

// LOOP THE RESULT TILL IT REACHED END
// rs.getString(1) – FIRST COLUMN
// rs.getString(2) – SECOND COLUMN ETC
while( rs.next() )
System.out.println( rs.getString(1) + rs.getString(2)+ rs.getString(3)+ rs.getString(4)) ;

// CLOSE THE RESULT, STATEMENT AND CONNECTION
rs.close() ;
stmt.close() ;
conn.close() ;

}
// HANDLE THE SQL EXCEPTION
catch( SQLException se ) {

System.out.println( “SQL Exception:” ) ;

// PRINT TILL ALL THE ECEPTIONS ARE RAISED
while( se != null ) {

System.out.println( “State : ” + se.getSQLState() ) ;
System.out.println( “Message: ” + se.getMessage() ) ;
System.out.println( “Error : ” + se.getErrorCode() ) ;

se = se.getNextException() ;

}

}
//CATCH THE CLASS EXCEPTION
catch( Exception e ) {

System.out.println( e ) ;

}
}

}

Using *safari for Running our Tests on Safari Browser

Using *safari for Running our Tests on Safari Browser

After a lot of research I’ve found that only the following combination works for running our tests on Safari Browser:

Mac OS + Safari Browser -> This will work without any problems.
Windows 7 + Safari Browser -> This may run the tests but the final result will fail though the test executed without any fails. Hence not suggested.
What happened when I tried Running Selenium RC Tests on Windows 7 OS + Safari Browser:
1. I disabled the Pop-Up blocker in Safari Browser as  shown below:
2. Copied the Safari Folder path in C:/Program files as shown below:
3. Click on ‘Start’ Button, Right Click on the ‘Computer’ option in the start Menu and select ‘Properties’ option as shown below:
4. Click on ‘Advanced System Settings’ option and ensure that the ‘System Properties’ dialog is displayed.  In the ‘Advanced’ tab of ‘System Properties’ dialog select ‘Environment Variables’ button as shown below:
5. Ensure that ‘Environment Variables’ sub dialog is displayed, select the Path variable and click on ‘Edit’ button as shown below:
6. Ensure ‘Edit User Variable’ sub dialog is displayed and add a semi colon at the end of the existing statements followed by pasting the above copied path in step 2 into the Variable Value field and click on ‘OK’ button as shown below:
7.  Click on ‘OK’ buttons on the remaining ‘Environment Variables’ and ‘System Properties’ dialog.
8.  In Eclipse IDE, I’ve replaced *iexplore with *safari and Ran the Selenium RC Test using JUnit Test option. Unfortunately the Test didn’t Run.
9. So I have then replaced *safari with *safariproxy as shown below:
 10. Start the Selenium Standalone Server
11. Run the Test using JUnit Test option and observe that the Test Ran this Time in Safari.
12. Though the Selenium Test executed successfully, the Test has resulted as Fail though there are no failures as shown below (i.e. com.thoughtworks.selenium.SeleniumException: ‘undefined’ is not an object is displayed as error )
 Watch the below video:

Click here to watch the video.

Download this Project:

Click here to download this project and import into Eclipse IDE  on your machine.

NOTE : After lots of research activity its found that Running Selenium RC Tests on Safari Browser of MAC OS X machine will work without any errors. I am not going to explain this practically on MAC machine. Please try yourself.

%d bloggers like this: