Sessions & Episodes

​

Sessions as Episodes

• Starts with a specific task
• Continues through multiple tool calls
• Ends when finished: true is received from a ToolOutput.

​

RL Episode Terminology

​

Episode Lifecycle

​

Complete Flow

​

States in Detail

​

1. Session ID Generation

POST /create_session

{"sid": "abc-123-def-456"}

​

2. Episode Initialization

POST /create
X-Session-ID: abc-123-def-456
Content-Type: application/json

{
  "env_name": "math",
  "task_spec": {"question": "What is 2+2?"},
  "secrets": {"api_key": "sk-..."}
}

1. Server resolves env_name (or defaults to first environment) and task_spec (or loads from split/index)
2. Instantiates the environment class with task_spec and secrets
3. Calls environment.setup() (async)
4. Marks session as “ready” when setup completes

​

3. Initial Observation

GET /math/prompt
X-Session-ID: abc-123-def-456

[
  {"text": "What is 2+2?", "detail": null, "type": "text"}
]

​

4. Action-Observation Loop

POST /math/call
X-Session-ID: abc-123-def-456

{"name": "submit", "input": {"answer": "4"}}

{
  "ok": true,
  "output": {
    "blocks": [{"text": "Correct!", "detail": null, "type": "text"}],
    "metadata": null,
    "reward": 1.0,
    "finished": true
  }
}

1. Agent takes action (calls tool)
2. Environment executes action
3. Environment returns next state (blocks), reward, and termination flag
4. If finished: false, repeat from step 1
5. If finished: true, episode is complete

• Action: Tool call
• Observation: Blocks
• Reward: Reward signal
• Terminal: Finished flag

​

5. Episode Termination

POST /delete
X-Session-ID: abc-123-def-456

1. Calls environment.teardown()
2. Removes session from active sessions
3. Frees memory and resources

​

Episode Termination

​

The finished Signal

interface ToolOutput {
  blocks: Blocks
  metadata?: JSONObject
  reward?: number
  finished?: boolean  // ← Episode termination signal (default: false)
}

• Episode is complete
• Agent should stop calling tools
• Agent should call /delete to cleanup
• Task succeeded or failed (check reward or blocks for details)

• Episode continues
• Agent should take another action
• State may have changed (reflected in blocks)

​

Termination Patterns

​

Pattern 1: Immediate Termination

# Single action episode
result = session.call_tool("submit", {"answer": "42"})
assert result.finished == True

​

Pattern 2: Multi-Step Termination

# Multi-step episode
result1 = session.call_tool("bash", {"command": "cat file.txt"})
assert result1.finished == False  # Continue

result2 = session.call_tool("submit", {"answer": "Paris"})
assert result2.finished == True  # Complete

​

Pattern 3: Failure Termination

result = session.call_tool("submit", {"answer": 999})
assert result.finished == True
assert result.reward == 0.0  # Failed

​

State Management

​

What’s Preserved in a Session?

• Instance variables in environment class
• Files created during episode (if environment has filesystem or persistent sandbox)
• Any side effects from tool executions

# State persists across tool calls
session.call_tool("bash", {"command": "export VAR=hello"})
result = session.call_tool("bash", {"command": "echo $VAR"})
# → "hello" (state preserved)

​

What’s NOT Preserved?

• Each session is independent at the protocol level
• Session 1 and Session 2 have separate environment instances
• No shared instance state between sessions (though implementations may share class-level or cached state)

• 15 minutes of inactivity → session deleted
• State is lost
• Must create new session

• Episode data is final
• Further tool calls should not be made
• Call /delete for cleanup

​

Session Timeout

​

Inactivity Definition

• /ping resets timer
• /{env_name}/call resets timer
• /{env_name}/prompt resets timer
• Any request with X-Session-ID resets the timer (except /delete, which removes the session)

​

Keeping Sessions Alive

import threading
import time

def keep_alive(session_id):
    while True:
        requests.post(
            "http://server/ping",
            headers={"X-Session-ID": session_id}
        )
        time.sleep(300)  # Every 5 minutes

# Start background thread
threading.Thread(target=keep_alive, args=(session_id,), daemon=True).start()

​

Timeout Cleanup

1. Server calls environment.teardown()
2. Session removed from active sessions
3. Subsequent requests with that session ID → 404 Not Found

​

Session Best Practices

​

1. Always Delete Sessions

# Good - cleanup
session_id = create_session()
try:
    # ... episode logic
    pass
finally:
    delete_session(session_id)

# Bad - resource leak
session_id = create_session()
# ... episode logic
# Forgot to delete!

​

2. Check finished Flag

# Good - respect termination
result = session.call_tool("submit", {"answer": "42"})
if result.finished:
    delete_session(session_id)
else:
    # Continue episode
    pass

# Bad - ignore termination
result = session.call_tool("submit", {"answer": "42"})
# Continue calling tools even if finished=True

​

3. Handle Errors Gracefully

# Good - cleanup on error
try:
    result = session.call_tool("bash", {"command": "rm -rf /"})
except Exception as e:
    delete_session(session_id)
    raise

​

4. Use Context Managers

# Best - automatic cleanup
with session_manager.session(task=task) as session:
    result = session.call_tool("submit", {"answer": "42"})
    # Automatically deleted when exiting context

​

Debugging Sessions

​

Common Issues

• Cause: Session timed out or was deleted
• Fix: Check that episode completes within 15 minutes or use /ping

• Cause: Trying to create episode with already-used session ID
• Fix: Generate new session ID with /create_session

• Cause: Calling tool after /delete was called
• Fix: Don’t reuse session IDs after deletion

​

Next Steps