Connect OpenAI Assistants to Agent Therapy REST API

Agent Therapy’s OpenAI integration lets you pipe conversation history from any OpenAI Assistant directly into the Agent Therapy analysis engine. Once connected, every thread becomes a diagnostic session: Agent Therapy scans message sequences for behavioral anti-patterns — instruction drift, repetitive failure loops, over-refusal, and more — and returns a structured health report with actionable recommendations.

How It Works

Agent Therapy ingests OpenAI thread messages (the full conversation history stored in the Assistants API) and analyzes them for behavioral patterns. When you export a thread and submit it, Agent Therapy reconstructs the turn-by-turn dialogue, identifies the roles involved, and applies its behavioral model to score the assistant’s performance. Results surface in your Agent Therapy dashboard under the registered agent’s profile, and you can retrieve them programmatically via the REST API. No changes to your OpenAI Assistant configuration are required — the integration works entirely on the output side by reading threads after they occur.

Prerequisites

Before you begin, make sure you have the following:

An OpenAI API key with access to the Assistants API (beta.threads endpoints)
An Agent Therapy account with an active API key — find yours in the Agent Therapy dashboard
The openai Python package installed (pip install openai)

Step-by-Step Setup

Before submitting any sessions, register your OpenAI Assistant as an agent so Agent Therapy knows how to categorize and score it. Send a POST request to /agents with the framework field set to "openai".

import requests

AT_API_KEY = "at_..."  # Your Agent Therapy API key

response = requests.post(
    "https://api.agenttherapy.ai/v1/agents",
    headers={
        "Authorization": f"Bearer {AT_API_KEY}",
        "Content-Type": "application/json",
    },
    json={
        "name": "Customer Support Assistant",
        "framework": "openai",
        "model": "gpt-4o",
        "description": "Handles tier-1 customer support queries",
    },
)

agent = response.json()
agent_id = agent["id"]  # Save this — e.g. "agt_01j..."
print(f"Registered agent: {agent_id}")

Store the returned agent_id. You’ll attach it to every session you submit for this assistant.

Export thread messages using the OpenAI SDK

After an assistant run completes, retrieve the full message list from the thread. The OpenAI SDK returns messages in reverse-chronological order, so sort them before processing.

from openai import OpenAI

openai_client = OpenAI(api_key="sk-...")

def fetch_thread_messages(thread_id: str) -> list:
    """Retrieve all messages from an OpenAI thread, sorted oldest-first."""
    messages = openai_client.beta.threads.messages.list(
        thread_id=thread_id,
        order="asc",   # oldest message first
        limit=100,     # increase if your threads are longer
    )
    return messages.data

Each item in messages.data is a ThreadMessage object containing the role (user or assistant), content blocks, and metadata timestamps.

Transform messages to Agent Therapy format

Map the OpenAI ThreadMessage objects to Agent Therapy’s universal behavior_log format. Each turn needs a role, content string, and an optional timestamp.

from datetime import datetime, timezone

def transform_messages(thread_messages: list) -> list:
    """Convert OpenAI ThreadMessage objects to Agent Therapy log format."""
    logs = []
    for msg in thread_messages:
        # Extract plain text from content blocks (handles text-type blocks)
        content_parts = []
        for block in msg.content:
            if block.type == "text":
                content_parts.append(block.text.value)

        logs.append({
            "role": msg.role,  # "user" or "assistant"
            "content": "\n".join(content_parts),
            "timestamp": datetime.fromtimestamp(
                msg.created_at, tz=timezone.utc
            ).isoformat(),
            "metadata": {
                "openai_message_id": msg.id,
                "thread_id": msg.thread_id,
            },
        })
    return logs

Submit the session to Agent Therapy

With your formatted log ready, submit it as a session via POST /sessions. Agent Therapy queues the session for analysis and returns a session_id you can use to poll for results.

def submit_session(agent_id: str, behavior_log: list, thread_id: str) -> str:
    """Submit a behavior log to Agent Therapy and return the session ID."""
    response = requests.post(
        "https://api.agenttherapy.ai/v1/sessions",
        headers={
            "Authorization": f"Bearer {AT_API_KEY}",
            "Content-Type": "application/json",
        },
        json={
            "agent_id": agent_id,
            "behavior_log": behavior_log,
            "context": f"OpenAI thread {thread_id}",
            "source": "openai_assistants",
        },
    )
    session = response.json()
    print(f"Session submitted: {session['id']} — status: {session['status']}")
    return session["id"]

Automated Monitoring

Rather than submitting threads manually, wrap the full pipeline in a single function that you call after every assistant run. This ensures Agent Therapy receives a session for every interaction without extra effort.

import requests
from openai import OpenAI
from datetime import datetime, timezone

openai_client = OpenAI(api_key="sk-...")
AT_API_KEY = "at_..."
AT_AGENT_ID = "agt_..."  # Your registered agent ID


def monitor_thread(thread_id: str) -> str:
    """
    Fetch an OpenAI thread, transform it, and submit it to Agent Therapy.
    Call this after every assistant run completes.

    Returns the Agent Therapy session ID.
    """
    # 1. Fetch messages from OpenAI
    messages = openai_client.beta.threads.messages.list(
        thread_id=thread_id,
        order="asc",
        limit=100,
    )

    # 2. Transform to Agent Therapy format
    behavior_log = []
    for msg in messages.data:
        content_parts = [
            block.text.value
            for block in msg.content
            if block.type == "text"
        ]
        behavior_log.append({
            "role": msg.role,
            "content": "\n".join(content_parts),
            "timestamp": datetime.fromtimestamp(
                msg.created_at, tz=timezone.utc
            ).isoformat(),
            "metadata": {"openai_message_id": msg.id},
        })

    # 3. Submit to Agent Therapy
    resp = requests.post(
        "https://api.agenttherapy.ai/v1/sessions",
        headers={
            "Authorization": f"Bearer {AT_API_KEY}",
            "Content-Type": "application/json",
        },
        json={
            "agent_id": AT_AGENT_ID,
            "behavior_log": behavior_log,
            "context": f"OpenAI thread {thread_id}",
            "source": "openai_assistants",
        },
    )
    resp.raise_for_status()
    return resp.json()["id"]


# Example usage — call after run.status == "completed"
if __name__ == "__main__":
    session_id = monitor_thread(thread_id="thread_abc123")
    print(f"Monitoring session created: {session_id}")

You can integrate monitor_thread() into any post-run hook, webhook handler, or polling loop that watches for completed runs via openai_client.beta.threads.runs.retrieve().

Thread size limits: Agent Therapy sessions support up to 500 turns per submission. If your threads exceed this length, split them into chronological chunks of 500 messages and submit each chunk as a separate session with the same agent_id. Use the context field (e.g., "thread_abc123 — part 2 of 3") to keep chunks identifiable in your dashboard.

Get Started

Core Concepts

Guides

Integrations

Support

Connect OpenAI Assistants to Agent Therapy REST API

How It Works

Prerequisites

Step-by-Step Setup

Automated Monitoring

​How It Works

​Prerequisites

​Step-by-Step Setup

​Automated Monitoring

How It Works

Prerequisites

Step-by-Step Setup

Automated Monitoring