Interpret Your Agent Therapy Behavioral Diagnostic Report

After Agent Therapy finishes analyzing a session, it produces a diagnostic report that covers everything it found in your agent’s behavior log. This guide walks through every section of that report — what each field means, how to read severity ratings and confidence scores, and how to turn findings into concrete improvements. By the end, you’ll be able to open any report and immediately know what your agent did, what went wrong, and what to fix.

The Health Score

The health score is a single number between 0 and 100 that summarizes the overall behavioral quality of the session. Think of it as a grade: 100 means no issues were detected, and lower scores reflect the number, severity, and confidence of the findings in the session. In the API response, the health score appears at the top level of the session object:

{
  "id": "ses_7nBwQdTrXv",
  "agent_id": "agt_4kR9mXpLqZ",
  "status": "completed",
  "health_score": 74,
  ...
}

In the dashboard, the health score is displayed as a large colored number on the session detail page:

85–100 — Green. The agent is behaving as expected.
65–84 — Yellow. Minor issues detected; worth reviewing.
40–64 — Orange. Moderate issues that may affect users.
0–39 — Red. Serious problems requiring immediate attention.

The health score for a session also feeds into the rolling health score visible on the agent’s overview page, which is the weighted average of your agent’s last 10 sessions.

Findings

The findings array is the heart of the diagnostic report. Each entry represents a distinct behavioral pattern that the diagnostic engine detected in the log. A single session can produce zero findings (a clean bill of health) or many. Example findings array:

{
  "findings": [
    {
      "id": "fnd_1aZpKr",
      "pattern": "instruction_drift",
      "severity": "medium",
      "confidence": 0.82,
      "evidence": [
        "Assistant response at 2024-01-15T10:02:15Z diverged from the user's original task scope: user asked for a price lookup, but assistant began offering unsolicited product comparisons."
      ]
    },
    {
      "id": "fnd_2bYqMs",
      "pattern": "hallucination_burst",
      "severity": "high",
      "confidence": 0.91,
      "evidence": [
        "Assistant stated 'Our Enterprise plan includes unlimited seats' at 2024-01-15T10:04:22Z. No tool call was made to verify this. Retrieved context does not contain this claim."
      ]
    }
  ]
}

Finding fields explained:

Field	Type	Description
`id`	string	Unique identifier for this finding, used to link recommendations.
`pattern`	string	The name of the detected behavioral pattern (see common patterns below).
`severity`	string	One of `critical`, `high`, `medium`, or `low` — reflects potential user impact.
`confidence`	number	A value between `0.0` and `1.0` indicating how certain the engine is about the detection. Scores above `0.75` are considered reliable.
`evidence`	array of strings	Excerpts from the log that triggered the finding, with timestamps and context.

Severity levels:

Severity	Meaning
`critical`	The agent behaved in a way that is likely harmful, unsafe, or severely misleading. Requires immediate action.
`high`	A significant behavioral failure that will degrade user experience or produce incorrect outputs.
`medium`	A notable deviation that may cause confusion or reduce trust, but is unlikely to cause direct harm.
`low`	A minor stylistic or efficiency issue. Worth tracking but not urgent.

Common patterns:

Pattern	Description
`instruction_drift`	The agent deviated from its stated goal or the user’s original intent mid-conversation.
`hallucination_burst`	The agent stated facts not supported by its context, retrieved data, or tool outputs.
`tool_misuse`	The agent called a tool with incorrect parameters, ignored tool output, or called tools redundantly.
`goal_hijacking`	The agent’s behavior suggests it was manipulated by user input into pursuing an unintended goal.
`verbosity_spike`	The agent produced responses significantly longer than needed, reducing clarity.
`refusal_loop`	The agent repeatedly refused valid requests without clear justification.

Recommendations

Every finding in the report comes with between one and three concrete recommendations. Recommendations are stored in the top-level recommendations array and linked to findings via finding_id. Example recommendation:

{
  "recommendations": [
    {
      "finding_id": "fnd_2bYqMs",
      "action": "Apply prompt correction to restrict unverified claims",
      "rationale": "Adding an explicit instruction such as 'Only state facts that are confirmed by tool output or retrieved context' significantly reduces hallucination rates in retrieval-augmented agents.",
      "intervention_type": "prompt_correction"
    },
    {
      "finding_id": "fnd_2bYqMs",
      "action": "Add a post-generation verification step",
      "rationale": "A lightweight self-check prompt asking the agent to cite its source for each factual claim can catch hallucinations before they reach the user.",
      "intervention_type": "pipeline_change"
    }
  ]
}

Recommendation fields explained:

Field	Description
`finding_id`	The ID of the finding this recommendation addresses.
`action`	A short imperative description of what to do.
`rationale`	An explanation of why this action addresses the underlying problem.
`intervention_type`	The category of fix. One of `prompt_correction`, `pipeline_change`, `tool_config`, `model_swap`, or `monitoring`.

When multiple recommendations exist for one finding, they are ordered from lowest to highest effort. Start with the first recommendation — it is typically the highest-impact, lowest-friction change.

The Summary

The summary field contains a paragraph of plain-English prose that describes what happened during the session. It is generated alongside the structured findings and is designed to be readable by non-technical stakeholders — product managers, support leads, or executives who need to understand what went wrong without parsing JSON. Example summary:

{
  "summary": "The agent handled most of the session correctly, successfully retrieving pricing information and answering product questions. However, in two exchanges it stated facts about the Enterprise plan that were not supported by the retrieved context — a pattern consistent with hallucination. Additionally, the agent drifted from the user's original request in one turn, offering unsolicited product comparisons. These issues contributed to a health score of 74. Addressing the prompt-level constraints around unverified claims is recommended as the highest-priority fix."
}

In the dashboard, the summary appears at the top of the session detail page before the structured findings, giving you a quick narrative orientation before diving into the data.

Exporting Reports

From the dashboard:

Open the session detail page for the completed session.
Click the Export button in the top-right corner.
Select Download PDF to generate a formatted report suitable for sharing or filing.

The PDF includes the health score, summary, all findings with evidence, and all recommendations. From the API: Fetch the full session object as JSON at any time by calling:

curl --request GET \
  --url https://api.agenttherapy.ai/v1/sessions/ses_7nBwQdTrXv \
  --header "Authorization: Bearer YOUR_API_KEY"

The response includes the complete report in a single JSON object. You can pipe this output to a file, forward it to a data warehouse, or process it programmatically.

Diagnostic reports are retained for 90 days by default. After 90 days, report data is permanently deleted. To extend or shorten the retention window for your organization, go to Settings → Data Retention and adjust the Report Retention Period slider.

Get Started

Core Concepts

Guides

Integrations

Support

Interpret Your Agent Therapy Behavioral Diagnostic Report

The Health Score

Findings

Recommendations

The Summary

Exporting Reports

​The Health Score

​Findings

​Recommendations

​The Summary

​Exporting Reports

The Health Score

Findings

Recommendations

The Summary

Exporting Reports