Teneo Developers
Table of Contents
Was this page helpful?

Share to

From Request To Response

The Teneo Platform allows for development of conversational AI applications where end-users, once a solution has been published to the Engine, can have natural language dialogues through the AI application with the Teneo Engine; the main functionality of the Teneo Engine could be described as an answering process or rather a coherent sequence of answering processes which makes up a dialogue. The sequence of events start when a request is sent to the Teneo Engine typically containing a user input and ends when the response is delivered to the end-user in the conversational AI application. After a dialogue ends, Teneo Engine generates session logs and these are afterwards available for querying in the Log Data Source in Teneo Studio.

Conversational AI applications can use the Teneo API to sent and receive messages from the Teneo Engine, the Interfacing with API JSPs page explains more details related to the usage of the Teneo API JSPs via http(s) and also provide a section describing how POSTing to Engine as application/JSON is possible. The JSPs views can read the Engine response and post-process and/or format the data before it is returned to the AI application.

Overview

The following graph provides a view of the possible processes from when a user input (request) is send to the conversational AI application until an answer (response) is returned; the various phases are described further in the following sections.

flowchart TD
  subgraph session_processing [Session Processing]
    process_request{Request}
    process_input{Session Id}

    begin_dialog(Begin dialog script)
    new_session(New session script)
    timeout_session(Timed-out session script)
    end_session(End dialog script)

  process_input --> begin_dialog

  begin_dialog -->|has expired session Id| timeout_session
  begin_dialog -->|has no session Id| new_session

  process_request -->|is 'endsession'| end_session
  process_request --> process_input
  end

  subgraph session_logging [Session Logging]
    pre_logging(Pre-logging script)
    log_session[Log Session]
    pre_logging --> log_session
  end

  subgraph input_processing [Input Processing]
    pre_processing(Pre-processing Script)
    input_processors[Input Processors]
    pre_processing --> input_processors
  end

  subgraph input_matching [Input Matching]
    pre_matching(Pre-matching Script)
    pre_listeners[Pre-listeners]
    triggers[Triggers]
    transitions[Transitions]
    pre_matching --> pre_listeners
    pre_listeners --> transitions
    pre_listeners --> triggers
  end

  subgraph dialogue_processing[Dialogue Processing]
    subgraph post_listeners_group[Post Listeners]
      post_listeners(( ))
      post_listeners2(( ))
    end
    raise_flow((RAISE))
    resume_flow([Resume])
    continue_flow([Continue])
    on_toP(On top script)
    on_drop(On drop script)
    flow_nodes{Flow Nodes}
    drop_flow((DROP))
    pause_flow([Pause])

    raise_flow --> post_listeners
    continue_flow --> post_listeners2
    post_listeners --> on_toP
    resume_flow ---> on_toP
    post_listeners2 --> flow_nodes
    on_toP --> flow_nodes
    flow_nodes --->|Give Response / Flow Link| pause_flow
    flow_nodes -->|Last Node| on_drop
    %%flow_nodes -->|Flow Link Node| raise_flow
    on_drop --> drop_flow
    %%drop_flow -->|repeat per flow| flow_nodes    
  end

  request([Request])
  post_processing(Post-processing Script)
  response([Response])

  request --> session_processing

  process_input -->|has live session Id| input_processing
  new_session --> input_processing
  timeout_session --> input_processing  
  end_session ----> session_logging

  session_logging ------------> response 
  input_processing --> input_matching

  triggers --> raise_flow
  transitions --> continue_flow
  dialogue_processing --> post_processing
  post_processing --> response

  classDef global_script fill:#2F286E,stroke:#FFFFFF,color:#FF4C58;
  classDef other_action fill:#00000000,stroke:#FEC800,color:#FEC800,stroke-dasharray:5,5;
  classDef http_action fill:#FEC800,stroke:#000000,color:#000000;
  classDef raise_node fill:#00000000,stroke:#10B981,color:#10B981,stroke-width:2px,stroke-dasharray:0;
  classDef drop_node fill:#00000000,stroke:#EF4444,color:#EF4444,stroke-width:2px,stroke-dasharray:0;
  classDef resume_node fill:#00000000,stroke:#10B981,color:#10B981,stroke-width:2px,stroke-dasharray:5,5;
  classDef pause_node fill:#00000000,stroke:#EF4444,color:#EF4444,stroke-width:2px,stroke-dasharray:5,5;
  classDef http_action fill:#FEC800,stroke:#000000,color:#000000;

  classDef invisible fill:#FEC800,opacity:0.75,stroke:#00000000,color:#00000000,stroke-width:0px,stroke-dasharray:0;

  class begin_dialog,new_session,timeout_session,pre_matching,pre_processing,end_session,pre_logging,on_toP,on_drop,post_processing global_script;
  class triggers,transitions,process_request,process_input,log_session,input_processors,pre_listeners,post_listeners_group,drop_flow,flow_nodes,raise_flow,resume_flow,pause_flow other_action;
  class request,response http_action
  class raise_flow raise_node
  class drop_flow drop_node
  class resume_flow,continue_flow resume_node
  class pause_flow pause_node
  class post_listeners,post_listeners2 invisible

  classDef group fill:#00000000,stroke:#FEC800,color:#FEC800;
  class session_processing,session_logging,input_processing,input_matching,dialogue_processing group;

Request

The request is the data made available to the Teneo Engine and it is sent to an application where the Teneo Engine is running for the purpose of locating an answer.

The request data typically consists of a session identifier (if available, see the following section), the user input and maybe additional data such as data accessible via the environment (e.g., http cookies, environment variables, self-defined data). Also, parts of the request data are accessible via scripts, including cookies, environment variables and self-defined request parameters.

Session Processing

After receiving the request, first the session identifier, which usually is provided as an http cookie or an addition to the URL path, is evaluated by the application server; the following options are available:

  • No session Id is available; the Begin dialog script and New Session scripts are executed
  • An expired (timed-out) Id is available; the Begin dialog script and Timed-out session scripts are executed
  • A live session Id is available: the Input processing will begin

A last alternative is if the request is 'endsession', in which case the Session logging is executed.

Input Processing

Before the input is processed, the Pre-processing script is executed (at this script level it is possible to manipulate the user input) and then, the user input is prepared for the matching process by the Teneo Input Processors which - although they behave similar and provide similar input processing - are language dependent. Below is based on English and the Standard Input Processor chain which cater for roughly 80 languages in the Teneo Platform, other languages have similar input processing steps which are all described in details in the Language Capabilities section.

The preparation of the user input includes the below steps:

  • Division into sentences
  • Division into words
  • Simplification
  • Auto-spelling correction
  • Spelling tolerance

Depending on the solution language, other Input Processors may run at this point in the answering process and although they may affect the matching process (due to the generation of annotations) they do not modify the user input and are therefore not explicitly detailed here. To learn more, please refer to the following pages in the documentation: NLP Capabilities and Annotating Inputs.

Note that the Intent Classification performed by the Predict Input Processor is deferred to the input matching process and is called on demand by Engine to avoid unnecessary calls to external services.

Division Into Sentences

The division of the user input into sentences works in accordance with the list of sentence delimiters specified in the Input Processors chain. A sequence of uninterrupted delimiters count as a single delimiter where corresponding entries in the abbreviation list override sentence division. Before division, protected areas are created based on the abbreviation list, for which divisions is prevented (this is applied subsequently for division into words).

Consider the following example:

​ Isn't there a onctact section at www.teneo.ai? Is your compny in Cancun?

In this case, the areas www. and .com are protected as they correspond to entries in the abbreviation list and, for this reason, the input is evaluated as consisting of two sentences rather than four.

Division Into Words

The division of sentences into words is performed by the Standard Splitting Input Processor; for more specifics please see the NLP Capabilities section selecting the language specific Input Processors chain.

​ Isn't
​ there
​ a
​ onctact
​ section
​ at
​ www.teneo.ai
​ Is
​ your
​ compny
​ in
​ Cancún

Simplification

The Standard Simplifier takes care of the simplification for English, and with the above example, the following would happen during the simplification process:

​ Isn't > isnt
​ there
​ a
​ onctact
​ at
​ www.teneo.ai
​ Is > is
​ your
​ compny
​ in
​ Cancún > cancun

Automatic Spelling Correction

During the automatic spelling correction, the individual words forming the sentences are compared with entries in the auto correction file in the Standard Auto Correction Input Processor, again, in the Standard Input Processors chain applied for English. If an entry is found, that word is exchanged with the corrected form.

In the second sentence of the user input example, a letter is missing; a typographical error that the standard correction list anticipates:

​ compny > company

Spelling Tolerance

The spelling tolerance is applied to all the words in the user input which are not found in the solution dictionary (for English, in the Standard Similarity Match Correction Input Processor). The dictionary is made up of words from word syntaxes in the TLML syntaxes (triggers, transitions, listeners, language objects, entities). For these words, a property file driven mechanism will find the closest match in the dictionary, within the defined spelling tolerance value. So, instead of using an unknown word in the matching process, the closest dictionary word is used instead.

Input Matching

Once the user input is prepared and the Input Processors chain for the given language has finished, the outcome (i.e., the sentences and annotations) are first tested against Global Pre-listeners after which there are two possibilities:

  • testing the user input against triggers, or
  • testing the user input against transitions.

Whether the user input is first tested against triggers or transitions depends on whether a flow is already active or not. If there are no current or active flows, e.g., it is the first user input in a dialogue or the previous interaction ended with a drop-flow event, then the user input is first tested against the triggers of the solution.
But, if the user already send a previous input and the previous flow is still active, e.g., a complex flow was triggered (a flow with more nodes/transitions than a simple flow which usually contains one trigger and one output node) and the user input was given while traversing the flow (e.g., it hit the Bot Response label after which the user needed to provide a new input to continue the flow dialogue), then the new user input is tested against the outgoing transitions of the last visited node in the currently active flow.

Deferred Intent Classification

At this point, while testing the user input against triggers or transitions, the Predict Input Processor might be invoked by Engine to create classification annotations; this happens when Engine - during the matching process - finds references to class annotations in a Class Match, a Syntax Match or in solution scripting; this means that input annotations for intents are created on demand and classification is only triggered by Engine in the following cases:

  • during evaluation of a Class Match in a trigger or transition
  • during the evaluation of an annotation in a TLML Syntax (trigger, transition, listener, etc.) where the annotation name ends with .TOP_INTENT or .INTENT (for example, %$CITY.TOP_INTENT)
  • through read access to annotations from solution scripting or script conditions (embedded in a TLML Syntax) via the EngineAccess method _.getInputAnnotations() and its sub-methods, except for these cases:
    • Access to a single annotation via methods _.getInputAnnotations().getByName(...) can trigger classification only if the given annotation name is a known class name plus suffix .TOP_INTENT or .INTENT.
    • Invocation of method toString() on the object returned by _.getInputAnnotations() will not trigger classification.

If none of the above cases are given during the matching process, no input annotations related to classes are created by Predict.

Note that for testing purposes only Tryout allows to force immediate input classification meaning that intent annotations are always created for any tested input no matter if any of the above cases are given.

Dialogue Processing

For both of the above explained possibilities (testing against triggers or transitions), the dialogue processing continues by testing the Global Post Listeners, Flow Listeners, trigger After Matches, executing the On top scripts (for triggers only when a new flow is raised and including both the global and the flow scripts) traversing the flow (including the testing of transition Matches and After Matches).

If there are no match when testing the user input against transitions, the Teneo Engine will test against the triggers of the solution, now if a match is found the flow processing continues as stated above. In case no match is found (neither in transitions nor triggers), then usually solutions contain a Safetynet flow which will be the triggered flow and the output will be the one defined in the Safetynet.

When the user input is tested against transitions in a more complex flow structure and a match is found, the the selected output will be the one for the matching transition, and then, depending on the flow structure, this output can either be at the end of the flow where the drop-flow event will happen or an output node after which the bot will wait for a new user input before continuing (repeating the answering process for the new user input when it arrives).

If more than one interaction has happened already then there might be other flows in the flow stack, and when a flow is dropped off the flow stack it is possible that a Resume Prompt from a previous flow on the flow stack is attached to the selected output.

At this point, when a match is found, the Teneo Engine creates the Response, but before continuing and sending it to the end user, Engine first tests the Prompt triggers and executes the Post-processing scripts.

Flow Stack

When a user is inside a "complex" flow and then asks a new question which there isn't prepared for in the current flow (Flow 1), the Engine will try to match against other triggers in the solution. When the second flow (Flow 2) is triggered it will become the top flow in the flow stack and there will be two active flows on the stack. If the user asks another question when Flow 2 has not finished yet, a third flow will be triggered and it will become the top flow and now there will be three flows on the stack of active flows. If Flow 3 is finished, it will be dropped and the next from from the list of active flows (Flow 2) will become the top flow and it will be resumed and, if a resume prompt is available, this will be displayed to the end-user.

There can be any number of active flows on the flow stack, but the Teneo developer can, for example, limit the active flow stack to a certain number by adding an on-top script as follows:

groovy

1int maxActiveFlows = 2; // number of top flows to keep
2while (_.activeFlows.size() > maxActiveFlows){_.activeFlows.first().first().abort(); }
3

Note that sub-flows do not count in the list of active flows as they are related to the flow or sub-flow that triggered them and they can only be called from different flows.

Response

After the execution of the Post-processing scripts, the Response can be send back to the end-user.

The selected output(s) have an effect on the log data, session history and session context (outputs given) as well as the answer document (via replacement of view variables). The selected output(s) are composed into the final output and made available as answer data; parts of the data form the dynamic part of the answer document together with more general Teneo Engine data. The content of view variables that receive their values from output data can be modified at runtime via script expression replacements and/or via scripting API methods.

After determining the final answer for the request, the answer document is generated as follows: the Teneo Engine sends the required data to a JSP which builds the response document in the requested view. During this process, the dynamic parts in the view (the view variables) are replaced with the data provided by the Teneo Engine (i.e., general Teneo Engine data, the answer text, emotions, etc.). The design of the views should allow the creation of a complete document fulfilling the individual requirements after the replacement of the dynamic parts.

The application server takes the created view and the additional http headers that have been created and adds a response line to it. Then the completed response is sent back to the requester. This response includes session information in order to identify the next request as belonging to the same session, and then the application server delivers the answer to the frontend application which displays the document (or computes the changes in the document from the transferred data).

Once the response is delivered, the clock starts ticking for the session time-out. If the user makes another request (writes another user input) before time-out, the session is continued.

Session Logging

When an 'endsession' request is received, then the End-dialog script(s) and Pre-logging script(s) are executed before the Teneo Engine takes the dialogue data and generates the session log.