Execute a Manual Test
Execute a Single Automated Test
Execute Multiple Automated Tests

Executing Tests

Test execution refers to the act of walking through the test steps and verifying the outcome against the expected result. Manual Execution is where a human (tester) personally executes each test step and visually compares the results. Automated Execution refers to a computer program that automatically executes test steps and compares results by looking at objects and data on the screen without human interaction. Crystal Test supports both manual and automated execution. The execution type is determined by the View dropdown on the Test Case List page. The View dropdown determines not only the way the test is executed, but also which test cases appear in the Test Case List.

Note: Before you can execute automated tests, you will need start the Automation Test Engine, Selenium Hub, and Selenium Node on your machine.

View Displayed Test Cases Execution Type
Automated Only test cases that have either been directly automated or are children of automated test cases are displayed. Automated
Manual Only test cases that have not been directly automated and are not children of automated test cases. This view removes automated children to ensure that we do not inflate our testing efforts by testing cases that will be covered by automation. Manual
All All test cases are listed regardless of whether they have been automated or not. This is the only view in which automated children can be tested manually. Manual

Execute a Manual Test

To manually execute a test:
  • Click the Testing button on the menu bar. This will open the Test Case List page.
  • Verify that All or Manual is selected in the View dropdown on the left column.
  • Click the Test button next to the test case you wish to test. This will open the Insert Test Result page

Execute Manually
  • Manually walk through and execute each test step exactly as it is listed.
  • When all steps are complete, compare the state of the application to the expected results.
  • If the state of the application is exactly as listed in the expected results, the test passed, otherwise the test failed.
For information on entering test results, see Test Results.

Execute a Single Automated Test

When an automated test is started from Crystal Test, the test is marked as ‘In Queue’. From that point the Automation Test Engine takes over and makes sure that only the allowed number of automated tests are running simultaneously. The test will be started as soon as a slot is available. Since the automation engine is a separate entity and there is no way to determine the length of time it may take to complete the test, you will need to refresh the screen to get status updates on your automated test. For more information see Automation Test Engine

Note: Crystal Test comes with an example script that allows a user to automatically log into Gmail. Until you create your own automated scripts, use this script to familiarize yourself with automation. To use this script, open the Crystal Test database and ExampleLogin table. Change the username and password fields in the first row to an account you are familiar with. Then execute the EXAMPLE-1 test case.
To automatically execute a test:
  • Click the Testing button on the menu bar. This will open the Test Case List page.
  • Verify that Automated is selected in the View dropdown on the left column.
  • Click the Test button next to the test case to be executed. The test is now either queued or in progress.
  • If you are running locally, your browser window will open and you can watch the test execute, if you are executing on the test grid, you will not be able to see the test running.
  • Results will be emailed to you but you may also refresh the screen periodically to see the result.

Execute Automatically

Execute Multiple Automated Tests

The ability to execute several or all automated test cases at one time is the most important thing that gives Crystal Test and automation it’s Return on Investment (ROI). You only need to create the script one time and while the system is executing your tests, you are available for other things.

When the Test Case List is in automated view, two new icons appear in the Icon Menu; the Add to Queue and Remove from Queue buttons. These buttons are used to add or remove multiple test cases from In Queue status. Once a test has been started and is set to ‘In Progress’, is not possible to remove it.

Add to Queue and Remove from Queue

To execute multiple automated test cases:
  • Click the Testing button on the menu bar. This will open the Test Case List page.
  • Verify that Automated is selected in the View dropdown on the left column.
  • Check the checkbox next to each test case you wish to add to the automation queue. If you wish to run all automated test cases, check the ‘All’ checkbox in the Test Case List header.
  • Click the Add to Queue button in the Icon Menu.
  • If you are running locally, your browser window will open and you can watch the tests execute, if you are executing on the test grid, you will not be able to see the tests running.
  • Results will be emailed to you but you may also refresh the screen periodically to see the result.

Last edited Apr 2, 2014 at 6:33 PM by jacquelinewalton, version 3