Augmenters

Introduction

The Teneo Platform provides a system for analyzing and accessing conversational log data produced by the conversational AI applications using the proprietary Teneo Query Language (TQL). It can be used as a data mining and data extraction tool or function as a backend component for report and dashboard generation.

The Teneo Platform retrieves the log data and stores it in such a way that it can easily be queried for analysis. However, some queries can be demanding especially when they need to combine several fields, or when they need to use Engine to compute and fulfil conditions.

Augmenters are new property fields stored along the log data that is generated by pre-calculations on existing log data. Augmenters extend default log data by enriching or summarizing it in a format that can later be accessed by standard queries. There are two types of augmenters: adorners and aggregators.

Augmenters can only be created via the Teneo Log Data Manager frontend, using either TQL or Groovy. This section describes how to create them and how to use them in TQL queries.

Adorners

As the name suggests, adorners allow to extend data by furnishing it with additional or more precise information in order to simplify a query syntax or speed up its performance. For instance, to single out all sessions with at least one user input falling into the safetynet of a solution, an adorner can be created to add this property directly to the session level.

Adorners cannot be built on top of other adorners, to avoid dependencies between them. But they can be used within aggregators.

Once created, adorner data is stored along the rest of the stored log data (for a list of the specific log properties, see here). Therefore, it is possible to combine log properties with adornments in a single same query.

How to create adorners

When creating adorners, the user should provide the adorner name and keys, where the name is just a user-friendly identifier, and keys are the names of the actual new properties stored along the rest of the log data. A single adorner can generate more than one adornment, each having their own key. In addition, the user can limit the creation of adornments to a data range and/or a solution revision range.

Adorners in Teneo Query Language

Adorners in Teneo Query Language (TQL) are created with the adorn command.

To create a new TQL adorner, follow the below steps:

• Go to the Augmenters tab in the Teneo Log Data Manager frontend
• Next, select the Log Data Source where the adorner should be created
• Then, click on +Add and select TQL Adorner in the dropdown menu.

• In the New TQL Adorner page, give the adorner a name and fill in the specifics for the new adorner

In the above image, the adorner name is User correction, and the adorn TQL command creates two adornments: t.a.s:user_correction_target_flow

and t.a.s:user_correction_input

.

The query first finds all transactions (t1

) which fall into a safetynet

trigger, which are identified because they are stored in the SmallTalk

folder. Then, from these, the query fiends the next transactions (t2

) that has a non-empty user input (t2.e1

) and checks that this user input did not go into another safetynet trigger (t2.e2

). The information stored in the adorner is: the name of the flow where t2

ended up, and the user input.

The adorn Teneo Query Language schematic syntax requires the following parameters:

1adorn key1 = selection1, key2 = selection2, .. : <TQL_condition>
2

Adornment keys must follow the below pattern:

1<LEVEL.a.TYPE:name>
2

where:

• LEVEL
  indicates if the property is stored at session s
  , transaction t
  , or event e
  level
• a
  stands for adorner
• TYPE
  indicates the data type (see table below): In the case of adorners of type array, the TYPE
  must following the pattern a:TYPE
  , where a:
  is the symbol for array and TYPE
  is the symbol describing the type of the element of the array
• name
  (following the colon) is the adornment property name which should be alphanumeric and cannot contain spaces nor punctuation marks.

Thus, adorners generate new properties with the following characteristics:

After saving the adorner, a new update action will appear in the Pending actions panel.

This action will be updated in the backstage for every new adorner or any update in existing ones until it is executed by pressing the Perform all or play button. See the Apply adorners section below for more information.

The following is another adorner example that uses an Engine matching operator. These types of adorners are useful because Engine matching queries are demanding, hence, pre-calculating these data can be very convenient. It creates an adorner at transaction level of type string with a key t.a.s:sentiment

. The value of the key sentiment is obtained by applying over the user input the sentiment classification provided by the Teneo Lexical Resource. For more information about the classify

operator, please see the Teneo Query Language manual.

1adorn t.a.a:s:sentiment = sentiment : 
2	classify ( 
3		solution = "sentiment", 
4		c = "%SENTIMENT_POSITIVE.INDICATOR", 
5		c = "%SENTIMENT_NEGATIVE.INDICATOR"
6		) t.e.userInput as (sentiment, "Positive", "Negative", "Neutral")"
7

Note that in the example, sentiment

is the name of a solution that has been uploaded in the backstage of the Log Data Source panel of Teneo Studio (in the Solutions area), and that it contains two Language Objects SENTIMENT_POSITIVE.INDICATOR

and SENTIMENT_NEGATIVE.INDICATOR

.

Groovy adorners

Adorners can also be created using a Groovy script without the need of using the Teneo Query Language. Groovy adorners are particularly useful when string processing or looping is required to obtain the intended adorner value.

• Go to the Augmenters tab in the Teneo Log Data Manager frontend
• Next, select the Log Data Source where the adorner should be created
• Then, click on +Add and select Groovy Adorner in the dropdown menu
• In the New Groovy Adorner page, give the adorner a name and fill in the specifics for the new adorner.

Note that the adornment keys cannot be automatically calculated as in the case of TQL adorners, and needs to be filled in as well.

The key needs to follow the standard adornment key format, as explained in the TQL adorners section above.

1LEVEL.a.PREFIX:NAME
2

In this case, adorner selections and constraints are optional, but it is useful to indicate them for user friendliness.

The adorner code itself is written in the Groovy script box with the following generic syntax:

1def myVar
2session.getTransactions().each {
3	t -> t.getEvents().each {
4		e -> // do something to event, e.g.
5			// assign a value to myVar
6	}
7	callback.adornTransaction(t.getId(), "t.a.s:myKey", myVar)
8}
9

The Groovy code in the image below searched for events with the pathtype

"continue-flow" and stores the index and the flow name. Then, when it finds another event with pathType

"drop-flow", it checks if it is a subsequent event from the earlier one, in such case it is adorned if the flow name is the same as the previous one. The adornment key is at transaction level, named t.a.s:AbortedFlow

, and it stores the flow name of the second flow.

The following example creates a session adorner with the number of events per each session:

1def count = 0
2session.getTransactions().each {
3	t -> t.getEvents().each {
4		count++
5	}
6}
7callback.adornSession(session.getId(), "s.a.n:numberOfEvents", count)
8

Note the use of the adornSession

method to adorn transactions and the getTransactions

and getEvents

methods to retrieve transactions and events respectively. These are Teneo Platform specific methods provided to work with session log data. The table in the section Augmenters Groovy methods summarizes the list of available methods.

Apply adorners

After an adorner has been created, it must be executed (Perform All or play button) on the Augmenter tab, under Pending actions. Then, only those sessions that match an adorner constraint are updated with the new key properties defined in the adorner.
In addition, when a solution revision and/or date ranges are provided in the adorner definition, only those session that match these properties will be updated.

Once an adorner is created and while it is enabled, any new session added to the Log Data Source is automatically adorned in the background without the need of any user intervention, regardless the new session is sent automatically from the conversational AI application or it is part of a synchronize action (sessions are reimported into a Log Data Source).

When an adorner is disabled, a new Pending action appears, and on action execution, the adornments are removed from the data.

Query adorners

Adorners data (adornment keys) is stored along the standard log data, and can be queried like any other property in the query panel of the Log Data Source in Teneo Studio.

The following are examples of queries that retrieve the information stored with the adorners described in this section.

(1) la t.a.s:user_correction_target_flow,t.a.s:user_correction_input

(2) lu t.a.a:s:sentiment : exists t.a.a:s:sentiment

(3) la t.e.userInput, t.a.s:AbortedFlow : t.a.s:AbortedFlow != "", t.e.type == "request"

(4) la s.id, s.a.n:numberOfEvents : s.a.n:numberOfEvents > 100

The following retrieves information from an adorner of type array of string (a:s

)

(5) la s.id, s.a.a:s:words : s.a.a:s:words == {“hi”, “hello”}

The following retrieves information from an adorner of type array of numbers (a:n

)

(6) la s.id, s.a.a:n:numbers : s.a.a:n:numbers ~~ {1, 2}

In addition, adorners definitions and the new property keys appear in the session viewer panel along the rest of the session properties:

Aggregators

Aggregators allow the construction of summaries over event, transaction, or session properties, including those generated by adorners. They simplify and speed up queries and post-query processing to obtain final counts. For example, an aggregator can be created at session level to store the number of inputs falling into the safetynet. These summary counts are automatically updated every time new data is added or imported into the Log Data Source.

Aggregators cannot be built on top of other aggregators in order to avoid dependencies among them. Once created, aggregated values are not stored along the rest of the regular log properties, but aside, as a detached division in the data structure. For this reason, they cannot be used directly as query constraints in regular queries. Instead, they can be used in sub-queries, which may serve the same purpose.

How to create aggregators

Aggregators counts are computed around dates

and keys

, which are defined by the user when the aggregator is created. A single aggregator can contain more than one key, e.g. one key contains the counts for each flow identifier, while another key has the counts for each folder name. That way it is possible to group several summary counts in a single aggregator, which in turn is identified by its name.

The aggregator name is a user-friendly id, while the aggregator keys

are used to store the data in the Log Data Source and hence must follow the below, specific pattern:

1<TYPE:NAME>
2

where:

• TYPE
  indicates the data type (see the above table for allowed values), and
• name
  (following the colon) is the key name provided when the aggregator is created.

Thus, aggregators data contain the following properties:

Note that as a result, date

, week

, version

and count

are reserved words that cannot be used as names in the aggregator keys.

TQL aggregators

Teneo Query Language (TQL) aggregators are created with the aggregate command.

To create a new TQL aggregator, follow the below steps:

• Click on the Augmenters tab in the Log Data Manager frontend
• Next, select the Log Data Source
• Click +Add and select TQL Aggregator in the dropdown menu

• In the New TQL Aggregator page, give the aggregator a name and write the TQL code according to the following schematic syntax:

1aggregate key1 = selection1, key2 = selection2 : <TQL_condition>
2

• Optionally, supply a solution revision range and/or a date range that will limit the sessions to be aggregated.

The example in the below image shows the creation of the aggregator "triggered Flows / Day" with the key s:triggeredFlowIdPerDay

(hence a session level count):

1aggregate s:triggeredFlowsIdPerDay = t.e.fid : t.e.pathType == "flow-trigger"
2

Once the aggregator is created, a new Pending actions menu appears at the bottom of the Augmenters page. Aggregators are calculated once the user requests this, by clicking either the Perform all or play button.

As in the case with adorners, when a new aggregator is created or updated, the Pending actions for updated augmenters is extended accordingly. Hence, it is possible to edit a collection of adorners and aggregators and apply them all at the end of the edition.

Groovy aggregators

Aggregators can also be created using Groovy.

To create a new Groovy aggregator, follow the below steps:

• Click on the Augmenters tab in the Log Data Manager frontend
• Next, select the Log Data Source
• Click +Add and select Groovy script aggregator in the dropdown menu
• In the New Groovy Aggregator page, write the name of the new aggregator
• Write the Groovy script in the Groovy section
• Add the Aggregator Keys
  Keys should be supplied manually because they cannot be automatically be extracted from the Groovy code (as opposed to TQL)
• Optionally, provide the Solution Revision and Date Range
• Aggregator Selections and Aggregator Constraints are optional too, but users are encouraged to provide these for user-friendliness.

The example in the following image creates an aggregator called "groovyagg" and generates the bucket counts s:flowIds

and s:id

. The Groovy code namely iterates over the list of events and aggregates those where pathType

is "flow-trigger".

The code in the image is also displayed in the following code piece:

1String sessionId = session.getValue("id")
2session.getTransactions().each {
3	it.getEvents().each {
4		String pathType = it.getValue("pathType")
5		if (pathType != null && pathType.equals("flow-trigger")) {
6			callback.aggregate(["s:flowIds":it.getValue("fid"),
7				"s:id":sessionId])
8		}
9	}
10}
11

Note the use of the getTransactions()

, getValue

and aggregate()

methods. These are Teneo methods to create the aggregations. the table in the Augmenters Groovy methods section summarizes the list of available methods.

Applying aggregators

After an aggregator has been created, it must be executed (Perform All or play button) on the Augmenters tab, under Pending actions. Then, only those sessions that match the aggregation constraint are included in the count bucket.
In addition, when a solution revision and/or date ranges are provided in the aggregator definition, only those session that match these properties will be considered.

Once an aggregator is created and while it is enabled, any new session added to the Log Data Source is automatically added to the bucket counts in all the existing aggregators in the background without the need of any user intervention, regardless the new session is sent automatically from the conversational AI application or it is part of a synchronize action (sessions are reimported into a Log Data Source).

When an aggregator is disabled, a new Pending action appears, and on action execution, the aggreation data is removed.

Query aggregators

The aggregated results are stored in separated namespaces. In order to query them, use the "a" command and provide the aggregator name. The syntax is the following:

1lu (a = "<aggregator name>") <aggregator key>
2

The examples below retrieve the data stored in the aggregators created in the previous section:

(1) lu (a = "groovyagg") s:flowIds, count

(2) lu (a = "groovyagg") s:id, count

(3) lu (a = "triggered Flows / Day") s:triggeredFlowsIdPerDay, count

In addition to the counts, aggregators store the session date

, week

, and version

, which can be also queried and used to further aggregate the results when combined with the distribute command.

See, for example, the difference among the following queries:

(1) d (a = "triggered Flows / Day") s:triggeredFlowsIdPerDay, count

(2) d (a = "triggered Flows / Day") week, s:triggeredFlowsIdPerDay, count

(3) d (a = "triggered Flows / Day") date, s:triggeredFlowsIdPerDay, count

(4) d (a = "triggered Flows / Day") version, s:triggeredFlowsIdPerDay, count

(5) d (a = "triggered Flows / Day") date, count

(6) d (a = "triggered Flows / Day") week, count

(7) d (a = "triggered Flows / Day") version, count

Exporting and importing augmenters

Augmenters definition can be moved between Log Data Sources. All augmenters for a specific Log Data Sources can be exported to a zipped archive by clicking the Export Augmenters link in the web UI. Correspondingly, augmenters in an exported augmenter archive can be uploaded by clicking on the Import Augmenters link.

The zipped archive contains augmenter definitions, where each augmenter is represented as a JSON file.

The files in the archive can be modified or deleted, and therefore they can be modified before they are imported.

When augmenters are imported, they are stored but not automatically applied. However, the system will verify the correctness of the provided augmenters, and invalid augmenters will not be imported.

Note that augmenters that make use of Teneo Engine matching will not be imported if the solution that they refer to is not uploaded under the specified name to the system.

Augmenters Groovy methods

When writing Groovy augmenters, the user may need to explore and retrieve the data from the log store. Teneo Log Data provides the variable named session

that allows accessing the session data, and a collection of Groovy methods that allow exploring and retrieving its data content.

The following table summarizes these methods and where they can be used.

Teneo Log Data also provides some callback methods to fill in the adorner values and the aggregators.

The following table summarizes these methods and their signatures: