Teneo Developers
Table of Contents
Was this page helpful?

Share to

Auto-test results panel

Test results panel

Results from Auto-tests are displayed in the Auto-test page in the backstage of Teneo Studio Desktop.
If a test was performed earlier, when opening the page, the results from the last Auto-test are automatically displayed. If no test was executed previously, then test results populates the window only when the user starts the first Auto-test; learn how to run an Auto-test here.

Auto-test results panel

The top area of the test results view includes a visual summary of the currently displayed test results (below Solution). The summary includes details about the number of tested Triggers, Transitions and Links as well as a percentage for the pass rate.

Action panel

Selecting any of the examples of User Intent in the results view will populate the right-side panel with information related to the selected User Intent example and the Trigger it belongs to.

In case of a failed test, the Trigger which stole the example (blocking the expected result) will have a red line on its left side and is displayed in the top under Triggered. The test example will be displayed next in the section Expected Result - Should trigger.

For both failed tests and tests with warning, an All Matches section is available displaying information about all the Triggers which matched the User Intent example.

In the Action panel, the user can double-click any of the Triggers to open the Flow it belongs to. Furthermore, three icons are available above the All Matches section which allows the user to open the Flow, Trigger Ordering or obtain further information by clicking the Info button.

Action panel

Filter options

The filters are available below the test result summary.

Filter section

Test results can be filtered by using the following options:

  • Text filter (applies to Flow names, examples of User Intent, and the test result message)
  • Filter buttons to show/hide the following test items:
    • Succeeded (green checkmark)
    • Passed with warning (yellow triangle)
    • Failed (red warning)
    • Non-testable items (clipboard)

When many test results are available, the list of successful results will automatically be hidden (the Succeeded button will automatically be disabled). This happens in the following two scenarios:

  • There are more than 100 results and at least one is a failed result. In this case only failed results are displayed.
  • There are more than 500 results, but none of them are a failed result.

In both of the above cases, the succeeded results can always be displayed by clicking the respective button.

Test passed (with warning)

When Positive examples of User Intent pass the Auto-test with warning, it means that although the example matched the correct trigger and triggered it, there are one or more triggers in the same Order group which also match the positive example and potentially could trigger it.

In these cases, it is recommended to create order relations between the Triggers for which the warning is displayed, because even though these currently do not cause the Auto-test to fail, an order relation will help avoid conflicts or Auto-test failures in the future.

Failed tests

The most common reasons for a failed test are:

  • The Matches are not covering the positive examples of User Intent
  • Ordering problem (the User Intent example is stolen by another Trigger with a higher rank in Ordering).

When the text The example did not match this trigger is displayed next to the failed example of User Intent, it indicates that the example is not matched by the Matches defined in the trigger and therefore was not triggered.

The example was not matched

Another trigger, ranked below the current one, could still trigger the example (if covered by its Matches), but as this was most likely not the intention, the user is encouraged to open the failed Trigger and either edit the example of User Intent or update the defined Matches.

When one or more examples of User Intent are triggered by one or more different Flows, in most cases this is due to Ordering and can be solved by either changing the Order group of one of the triggers or by creating a relation between the two triggers.

Issue related to Ordering

Ordering, Order groups and relations
Defining Trigger properties
Matches