Monitor LangChain Agents with Agent Therapy REST API

Agent Therapy integrates with LangChain through a custom callback handler that runs alongside your existing agent without requiring any changes to your core agent logic. You attach AgentTherapyCallbackHandler to your chain or agent executor, and it silently captures every LLM call, tool invocation, and chain event in real time — then packages and submits them as a therapy session automatically at the end of each run.

Prerequisites

Before you begin, make sure you have the following:

LangChain installed in your Python environment (pip install langchain)
An Agent Therapy account with an active API key — find yours in the Agent Therapy dashboard
A registered agent in Agent Therapy — create one in the dashboard or via POST /agents

Install the SDK

Install the Agent Therapy Python SDK, which bundles the callback handler and the API client:

pip install agent-therapy-sdk

Using the Callback Handler

Import AgentTherapyCallbackHandler from the SDK and pass it to your agent’s callbacks list. The handler authenticates using your Agent Therapy API key and tags all captured data to the specified agent_id.

from langchain.agents import initialize_agent, AgentType
from langchain.tools import Tool
from langchain_openai import ChatOpenAI
from agent_therapy import AgentTherapyCallbackHandler

# Initialize the callback handler
handler = AgentTherapyCallbackHandler(
    api_key="at_...",
    agent_id="agt_..."
)

llm = ChatOpenAI(model="gpt-4o", temperature=0)

tools = [
    Tool(
        name="Search",
        func=lambda q: "result for: " + q,
        description="Useful for answering questions about current events",
    )
]

# Attach the handler to your agent
agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    callbacks=[handler],
)

# Run your agent as normal — Agent Therapy captures everything automatically
result = agent.invoke({"input": "What is the capital of France?"})
print(result["output"])

The handler submits a session to Agent Therapy at the end of each agent.invoke() call. No additional code is needed.

Enable verbose=True on your agent (initialize_agent(..., verbose=True)) to surface richer intermediate reasoning steps in Agent Therapy’s diagnostic reports. Verbose mode causes LangChain to emit additional chain events that the handler captures, giving the analysis engine more signal — especially useful for debugging multi-step reasoning failures.

What Gets Captured

The AgentTherapyCallbackHandler hooks into LangChain’s callback system and records the following events during each agent run:

All LLM inputs and outputs — every prompt sent to the language model and every completion returned, including token counts where available
Tool calls and results — the name of each tool invoked, the input passed to it, and the output it returned
Chain starts and ends — entry and exit events for every chain in the execution graph, with elapsed time
Errors and exceptions — any on_chain_error, on_tool_error, or on_llm_error event, including the full exception message and traceback

All captured events are timestamped and assembled into an ordered behavior log before submission, preserving the exact execution sequence.

Accessing Session Reports

Sessions are created automatically after each agent run. You can view them in the Agent Therapy dashboard under the registered agent’s profile, or retrieve them programmatically using the REST API.

import requests

AT_API_KEY = "at_..."
AT_AGENT_ID = "agt_..."

# List the 10 most recent sessions for your agent
response = requests.get(
    "https://api.agenttherapy.ai/v1/sessions",
    headers={"Authorization": f"Bearer {AT_API_KEY}"},
    params={
        "agent_id": AT_AGENT_ID,
        "limit": 10,
        "order": "desc",
    },
)

sessions = response.json()["sessions"]
for session in sessions:
    print(
        f"Session {session['id']} — "
        f"status: {session['status']} — "
        f"health_score: {session.get('health_score', 'pending')}"
    )

Once a session’s status is "completed", fetch the full transcript and findings using GET /sessions/{id}:

session_id = sessions[0]["id"]

report = requests.get(
    f"https://api.agenttherapy.ai/v1/sessions/{session_id}",
    headers={"Authorization": f"Bearer {AT_API_KEY}"},
).json()

print(report["summary"])
for finding in report["findings"]:
    print(f"  [{finding['severity']}] {finding['title']}: {finding['description']}")

Reports are typically available within 10–30 seconds of session submission, depending on log size.

Get Started

Core Concepts

Guides

Integrations

Support

Monitor LangChain Agents with Agent Therapy REST API

Prerequisites

Install the SDK

Using the Callback Handler

What Gets Captured

Accessing Session Reports

​Prerequisites

​Install the SDK

​Using the Callback Handler

​What Gets Captured

​Accessing Session Reports

Prerequisites

Install the SDK

Using the Callback Handler

What Gets Captured

Accessing Session Reports