Skip to content

Scenario Insights ​

TL;DR:

  • Scenario Insights provides a step-by-step "funnel" view of how users progress through your bot's logic.
  • Use the Scenario Insights Drawer to monitor Node Health metrics like Success Rate, Errors, and Timeouts.
  • Identify performance bottlenecks by tracking Avg Time and p95 response times, especially for AI nodes.

Who this is for ​

  • Bot Builders: To understand which parts of their logic are failing or "hanging".
  • Marketing & UX Designers: To identify where users drop off and lose interest.
  • Analytics Teams: To track the conversion rate of specific bot flows.

What you will learn ​

  • How to read the Node Funnel and its ordering logic.
  • Understanding the Node Health metrics in the side drawer.
  • The difference between a technical Error and a Timeout.

The Node Funnel ​

The Node Funnel is a visual representation of your scenario's execution flow. It displays nodes in the order users are most likely to encounter them, starting from the Start Node.

Funnel Sorting Logic ​

Nodes are automatically sorted using a Breadth-First Search (BFS) algorithm based on the links in your scenario. This ensures that the flow follows the logical progression of a bot conversation, even in complex branching scenarios.

Funnel Metrics ​

  • Entered: The total number of times sessions entered this node.
  • Exited: The number of times sessions successfully moved past this node to the next step.
  • Reach Rate: The percentage of sessions that reached this node relative to the total number of sessions that started the scenario.
  • Success Rate: The ratio of successful exits to total entries, color-coded for quick visual auditing (Green for ≥95%, Amber for ≥70%, Red for <70%).

The Scenario Insights Drawer (Node Health) ​

When you click on any node in the funnel, the Scenario Insights Drawer opens on the right. This panel provides a "health check" for that specific node, giving you detailed technical and behavioral data.

1. Performance Overview (Top Row) ​

  • Entered / Exited: Absolute traffic counts passing through the node.
  • Success Rate: The percentage of successful completions.
  • Reach Rate: A clamped (max 100%) indicator of how much of your total audience reached this specific point in the flow.

2. Stability and Speed (Bottom Row) ​

  • Errors: The count of technical failures. These occur when a node throws an exception (e.g., an AI model returns invalid JSON or a Webhook returns a 500 status).
  • Timeouts: The count of node executions that were forcibly stopped by the system. This happens when a node takes longer than the allowed limit (e.g., 60 seconds) to respond.
  • Avg Time: The average response time of the node.
  • p95: The 95th percentile response time. This is the most critical metric for "speed perception"—if p95 is high, some users are experiencing significant delays, even if the average is low.

How to Understand the Insights ​

Identifying User Drop-off ​

Look for a sharp decrease in Reach Rate between two specific nodes. If 90% of users reach "Ask Question A" but only 20% reach "Ask Question B", it means your users are either abandoning the conversation or your logic is preventing them from progressing.

Troubleshooting AI nodes ​

AI Agent nodes are prone to higher p95 values and occasional Timeouts.

  • High Timeouts: The AI provider (like Gemini or OpenAI) is taking too long to generate a response. Consider simplifying the prompt or checking provider status.
  • High Errors: The AI might be failing to follow your formatting rules or reaching safety filters. Check the Node Errors log at the bottom of the drawer for specific error messages.

Detecting "Hanging" Logic ​

If a node has a high Entered count but a very low Exited count with 0 Errors, it usually means the node is waiting for user input that never arrives or the logic branch is dead-ended.

Troubleshooting FAQ ​

  • Q: Why is my Reach Rate 100% even though I know users are dropping off?
    A: Reach Rate is calculated based on the total sessions that entered the scenario during the selected time period. If every session starts at node A, node A will always have a 100% Reach Rate. Check the subsequent nodes to see where the rate starts to drop.

  • Q: What is the difference between an Error and a Timeout?
    A: An Error is an explicit failure returned by the system (e.g., "API Key invalid"). A Timeout is a "silent hang" where the system waited as long as it could before giving up and killing the task.

  • Q: Why are nodes out of order in the funnel?
    A: The funnel uses the active links in your scenario to establish order. If a node is disconnected or part of an isolated loop, it might appear at the end of the list.

TeleGo.io Docs