The Class Performance view (Teneo backstage: Solution tab > Optimization > Class Performance) provides the user of Teneo Studio with a way of checking the performance of the Machine Learning model used in the solution and analyzes which Classes are conflicting with one another.
The evaluation of the machine learning model in Teneo is performed using Cross Validation.
The Class Performance view provides two visualizations for model performance analyses:
- Confidence Threshold Graph which gives an estimation of the behavior of the model for different confidence thresholds.
- Class Performance Table which shows the per-class performance of the ML model.
Common view controls
There are some controls which are common to all of the views and are always on display:
- View selector allows the user to change between the two Class Performance views.
- Launch Cross Validation (Run) button initiates the Cross Validation process. Once the process has started, a progress indicator will be shown in the Last CV result control.
- Last CV results shows the date and time of the last execution of the CV process if none is running and a progress indicator if there is one running. In the latter case, a cancel button will be shown by the progress bar, letting the user cancel the process at any time. A history of the last executions is also provided as a hint that is shown when the mouse hover over this control.
- Execution history selector all the views allow the user to compare between the last successful run and the previous CV runs. This control let's the user select with which run to compare. If it is empty, just the result of the last successful run are shown.
- None of the views will present any data until the Class Performance has been launched at least once successfully.
- It is not possible to run Cross Validation for solutions with less than 10 training data examples in any of their classes.
- The CV process is generally slow, in the order of minutes, and its duration depends mainly on the total number of training data examples of the solution. This duration should never exceed 1 hour; if it does, the process will be marked as failed. Note that this interval can be changed in the server configurations.
- Only one CV process can run at the same time.
Confidence Threshold graph
The purpose of this view is to provide a tool to analyze the estimated performance of the Classes in the solution with regard to the confidence threshold setting.
Class Performance table
The table displays one row for each class and a single row for the average values for all classes. For each row, the following columns are displayed:
- Class name name of the class
- Precision, Recall, F1 these are the binary classification metrics for the row's class, i.e. for all the training data examples whose ground truth class is the row's class, training data predicted as belonging to that class are considered positive and any other predictions as negatives.
- Examples number of training data examples of that class at the moment of execution of the Cross Validation.
- Conflicting classes shows the number of mistaken predictions of the model. Those predictions can be either false positives (FP) or false negatives (FN); the arrow at the end of the column unfolds a list of rows inside the cell, each one specifying one of the classes that were confused with the class of the row, the kind of error, and the percentage of classified training data that suffered from that kind of error for that particular class.