Connect Custom AI Agents to the Agent Therapy REST API

Agent Therapy is framework-agnostic. If your agent produces conversation logs — whether it’s a custom LangGraph workflow, a hand-rolled ReAct loop, a fine-tuned open-source model, or anything else — you can connect it to Agent Therapy using the REST API. All you need to do is collect the turns your agent produces, format them according to the universal log schema, and submit them as a session. Agent Therapy handles the analysis and returns a structured health report.

The Universal Log Format

Every session submitted to Agent Therapy is built from a behavior_log array. Each element in the array represents a single conversation turn. The schema supports four roles and a flexible metadata object for any additional context you want to preserve.

[
  {
    "role": "user",
    "content": "What are the side effects of ibuprofen?",
    "timestamp": "2024-11-01T10:00:00Z",
    "metadata": {
      "user_id": "usr_882",
      "session_source": "web_chat"
    }
  },
  {
    "role": "assistant",
    "content": "Common side effects of ibuprofen include nausea, heartburn, dizziness, and mild headache.",
    "timestamp": "2024-11-01T10:00:02Z",
    "metadata": {
      "model": "gpt-4o",
      "tokens_used": 48
    }
  },
  {
    "role": "tool",
    "content": "{\"source\": \"MedlinePlus\", \"excerpt\": \"Ibuprofen may cause stomach bleeding...\"}",
    "timestamp": "2024-11-01T10:00:01Z",
    "tool_name": "medical_database_lookup",
    "tool_call_id": "call_abc123",
    "metadata": {
      "latency_ms": 340
    }
  }
]

Field reference:

Field	Type	Required	Description
`role`	`"user"` \| `"assistant"` \| `"tool"` \| `"system"`	✅	Who produced this turn
`content`	string	✅	The text content of the turn
`timestamp`	ISO 8601 string	—	When the turn occurred; strongly recommended
`tool_name`	string	✅ if `role` is `"tool"`	The name of the tool that produced this output
`tool_call_id`	string	—	Links a tool result back to a specific assistant tool call
`metadata`	object	—	Any additional context — model name, token counts, latency, user IDs, etc.

Collecting Logs in Your Application

The easiest way to collect logs is to wrap your agent’s conversation loop with a lightweight logger class. The AgentTherapyLogger below accumulates turns as your agent runs and flushes them to Agent Therapy when the session ends.

import requests
from datetime import datetime, timezone
from typing import Any


class AgentTherapyLogger:
    """
    Collect conversation turns during an agent run and submit them
    to Agent Therapy as a session when the run completes.
    """

    def __init__(self, api_key: str, agent_id: str):
        self.api_key = api_key
        self.agent_id = agent_id
        self.behavior_log: list[dict] = []
        self._base_url = "https://api.agenttherapy.ai/v1"

    def log_turn(
        self,
        role: str,
        content: str,
        tool_name: str | None = None,
        tool_call_id: str | None = None,
        metadata: dict[str, Any] | None = None,
    ) -> None:
        """
        Record a single conversation turn.

        Call this after every LLM response, user message, tool result,
        or system prompt you want Agent Therapy to analyze.
        """
        turn: dict[str, Any] = {
            "role": role,
            "content": content,
            "timestamp": datetime.now(tz=timezone.utc).isoformat(),
        }
        if tool_name is not None:
            turn["tool_name"] = tool_name
        if tool_call_id is not None:
            turn["tool_call_id"] = tool_call_id
        if metadata:
            turn["metadata"] = metadata

        self.behavior_log.append(turn)

    def submit(self, context: str = "") -> str:
        """
        Submit the accumulated behavior log to Agent Therapy.

        Returns the session ID. The session is processed asynchronously —
        use the session ID to poll for results or configure a webhook.
        """
        if not self.behavior_log:
            raise ValueError("No turns logged — nothing to submit.")

        resp = requests.post(
            f"{self._base_url}/sessions",
            headers={
                "Authorization": f"Bearer {self.api_key}",
                "Content-Type": "application/json",
            },
            json={
                "agent_id": self.agent_id,
                "behavior_log": self.behavior_log,
                "context": context,
            },
        )
        resp.raise_for_status()
        session_id = resp.json()["id"]
        self.behavior_log = []  # Reset for the next run
        return session_id


# Example usage
logger = AgentTherapyLogger(api_key="at_...", agent_id="agt_...")

logger.log_turn(role="user", content="Summarize the Q3 earnings report.")
logger.log_turn(
    role="tool",
    content='{"revenue": "$4.2B", "yoy_growth": "12%"}',
    tool_name="document_retriever",
    tool_call_id="call_001",
    metadata={"doc_id": "earnings_q3_2024.pdf"},
)
logger.log_turn(
    role="assistant",
    content="Q3 revenue came in at $4.2 billion, representing 12% year-over-year growth.",
    metadata={"model": "gpt-4o", "tokens": 31},
)

session_id = logger.submit(context="Earnings report summarization — production run")
print(f"Session submitted: {session_id}")

Calling the API Directly

If you prefer to use the REST API without the helper class — or you’re working in a language other than Python — here’s the minimal flow using curl: register your agent, submit a session, and poll for the completed report.

# 1. Register your agent (one-time setup)
curl -s -X POST https://api.agenttherapy.ai/v1/agents \
  -H "Authorization: Bearer at_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Custom Agent",
    "framework": "custom",
    "model": "llama-3.1-70b",
    "description": "Internal document QA agent"
  }'
# Response: { "id": "agt_01j...", "name": "My Custom Agent", ... }


# 2. Submit a session with a behavior log
curl -s -X POST https://api.agenttherapy.ai/v1/sessions \
  -H "Authorization: Bearer at_..." \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agt_01j...",
    "context": "Document QA session — ticket #4821",
    "behavior_log": [
      {
        "role": "user",
        "content": "What does section 4.2 say about data retention?",
        "timestamp": "2024-11-01T14:00:00Z"
      },
      {
        "role": "assistant",
        "content": "Section 4.2 states that customer data must be retained for a minimum of 7 years.",
        "timestamp": "2024-11-01T14:00:03Z",
        "metadata": { "model": "llama-3.1-70b" }
      }
    ]
  }'
# Response: { "id": "ses_09k...", "status": "queued", ... }


# 3. Poll for results (repeat until status is "completed")
curl -s https://api.agenttherapy.ai/v1/sessions/ses_09k... \
  -H "Authorization: Bearer at_..."
# Response: { "id": "ses_09k...", "status": "completed", "health_score": 91, ... }

Best Practices

Always include timestamps

Even if your agent doesn’t natively track turn timing, add datetime.now().isoformat() to each turn when you log it. Timestamps let Agent Therapy detect latency spikes, time-based behavioral drift, and unusually long reasoning chains that might indicate a reasoning loop.

Log tool calls as separate turns

Don’t embed tool results inside the assistant turn’s content. Log each tool invocation as its own turn with role: "tool" and a tool_name. This gives Agent Therapy the resolution it needs to identify which specific tools are contributing to failures — and which are working correctly.

Submit sessions regularly, not just on failures

Submitting sessions only when something goes wrong creates survivorship bias in your agent’s health history. Regular submissions from healthy runs establish a behavioral baseline, making it much easier for Agent Therapy to flag genuine anomalies when they appear.

Add descriptive context strings

The context field is free text that appears in your dashboard and in session reports. Use it to capture the task type, environment, user segment, or run ID — for example, "Customer onboarding flow — prod — user_segment: enterprise". Good context strings make it far easier to correlate behavioral findings with specific workloads.

Sessions are processed asynchronously. After you submit a session, the API immediately returns a session object with "status": "queued" — the analysis runs in the background. To know when the report is ready, either poll GET /sessions/{id} until status is "completed", or register a webhook endpoint via POST /webhooks to receive a session.completed event pushed to your endpoint automatically.

Get Started

Core Concepts

Guides

Integrations

Support

Connect Custom AI Agents to the Agent Therapy REST API

The Universal Log Format

Collecting Logs in Your Application

Calling the API Directly

Best Practices

Always include timestamps

Log tool calls as separate turns

Submit sessions regularly, not just on failures

Add descriptive context strings

​The Universal Log Format

​Collecting Logs in Your Application

​Calling the API Directly

​Best Practices

Always include timestamps

Log tool calls as separate turns

Submit sessions regularly, not just on failures

Add descriptive context strings

The Universal Log Format

Collecting Logs in Your Application

Calling the API Directly

Best Practices