Teneo Developers
Table of Contents
Was this page helpful?

Share to

Auto-test results panel

Test results panel

The results from the Auto-test are displayed in the Auto-test page. If no test has been run earlier, the test results will be populated into the window once the user has started an Auto-test. If a test has been performed earlier, when opening the Auto-test page, the results from the last Auto-test are displayed.

For information on how to run an Auto-test, please look here.

Auto-test results panel

The top area of the test results view includes a visual summary of the currently displayed test results (below Solution). The summary includes details about the number of tested Triggers, Transitions and Links as well as a percentage for the pass rate.

Action panel

Selecting any of the examples of User Intent in the results view will populate the right-side panel with information related to the selected User Intent and the Trigger it belongs to.

In case of a failed test, the Trigger which stole the User Intent (blocking the expected result) will have a red line on its left side and is displayed in the top under Triggered. The test example will be displayed next in the section Expected Result - Should trigger.

For both failed tests and tests with warning, an All Matches section is available displaying information about all the Triggers which matched the User Intent.

In the Action panel, the user can double-click any of the Triggers to open the Flow it belongs to. Furthermore, three icons are available above the All Matches section which allows the user to open the Flow, Trigger Ordering or obtain further information by clicking the Info button.

Action panel

Filter options

The filters are available below the test result summary.

Filter section

Test results can be filtered by using the following options:

  • Text filter (applies to Flow names, examples of User Intent, and the test result message)
  • Filter buttons to show/hide the following test items:
    • Succeeded (green checkmark)
    • Passed with warning (yellow triangle)
    • Failed (red warning)
    • Non-testable items (clipboard)

When many test results are available, the list of successful results will automatically be hidden (the Succeeded button will automatically be disabled). This happens in the following two scenarios:

  • There are more than 100 results and at least one is a failed result. In this case only failed results are displayed.
  • There are more than 500 results, but none of them are a failed result.

In both of the above cases, the succeeded results can always be displayed by clicking the respective button.

Test passed (with warning)

When Positive Intents pass the Auto-test with warning, it means that although the Intent matched the correct trigger and triggered it, there are one or more triggers in the same Order group which also match the positive intent and potentially could trigger it.

In these cases, it is recommended to create order relations between theTriggers for which the warning is displayed, because even though these currently do not cause the Auto-test to fail, an order relation will help avoid conflicts or Auto-test failures in the future.

Failed tests

The most common reasons for a failed test are:

  • The Matches are not covering the positive User Intents
  • Ordering problem (the example Intent is stolen by another Trigger with a higher rank in Ordering).

When the text The example did not match this trigger is displayed next to the failed User Intent, it indicates that the Intent is not matched by the Matches defined in the trigger and therefore was not triggered.

The example was not matched

Another trigger, ranked below the current one, could still trigger the Intent (if covered by its Matches), but as this was most likely not the intention, the user is encouraged to open the failed Trigger and either review the user intent or update the defined Matches.

When one or more User Intents are triggered by one or more different Flows, in most cases this is due to Ordering and can be solved by either changing the Order group of one of the triggers or by creating a relation between the two triggers.

Issue related to Ordering

Ordering, Order groups and relations
Defining Trigger properties
Matches