# ORS vs MCP
Source: https://openrewardstandard.io/comparison/ors-vs-mcp
Comparing Open Reward Standard and Model Context Protocol
The Model Context Protocol (MCP) and Open Reward Standard (ORS) are both protocols for connecting language models to external systems, but they serve different purposes.
## Overview
**Model Context Protocol (MCP)**:
* Purpose: Connect LLMs to tools, data sources, and workflows
* Focus: General-purpose tool access
* Use case: Extending LLM capabilities with external APIs, databases, file systems
**Open Reward Standard (ORS)**:
* Purpose: Connect agents to reinforcement learning environments
* Focus: RL training and agent evaluation
* Use case: Training agents with reward signals, structured evaluation benchmarks
## Key Differences
| Feature | MCP | ORS |
| ----------------------- | ----------------------------- | --------------------------------- |
| **Primary Purpose** | Tool access, data integration | RL training environments |
| **Episode Termination** | No concept | `finished` signal |
| **Rewards** | No concept | Numeric feedback for RL |
| **Tasks** | No concept | Organised problems to solve |
| **Splits** | No concept | Tasks organised into splits |
| **Session Management** | Basic | Episode-centric (RL trajectories) |
| **Tool Calling** | Yes | Yes Yes |
| **Protocol** | JSON-RPC over stdio/SSE | HTTP/REST + SSE |
## Detailed Comparison
### Tool Calling
Both protocols support tool calling with similar interfaces:
**MCP Tool Spec**:
```json theme={null}
{
"name": "read_file",
"description": "Read contents of a file",
"inputSchema": {
"type": "object",
"properties": {
"path": {"type": "string"}
}
}
}
```
**ORS Tool Spec**:
```json theme={null}
{
"name": "read_file",
"description": "Read contents of a file",
"input_schema": {
"type": "object",
"properties": {
"path": {"type": "string"}
}
}
}
```
ORS intentionally aligns with MCP's tool specification format.
### Tool Responses
**MCP Response**:
```json theme={null}
{
"content": [
{"type": "text", "text": "File contents here"}
]
}
```
**ORS Response**:
```json theme={null}
{
"blocks": [
{"type": "text", "text": "File contents here"}
],
"reward": 0.0,
"finished": false
}
```
ORS adds:
* `reward` - For RL training feedback
* `finished` - For episode termination
### Episode Structure
**MCP**: No concept of episodes. Stateless or loosely stateful tool calls.
**ORS**: Episodes are first-class:
* Session = RL episode
* Episode continues until `finished: true`
* One complete trajectory through environment
* Clear start (task) and end (finished signal)
### Task Organization
**MCP**: No built-in task organization.
**ORS**: Tasks and splits:
* Tasks: Individual problems to solve
* Splits: tasks grouped into splits, e.g. train/val/test
## Next Steps
Build your first ORS server
Deep dive into ORS protocol
Learn about Model Context Protocol
Implement an ORS server
***
**Key Takeaway**: MCP and ORS solve different problems. MCP connects LLMs to tools. ORS connects agents to RL training environments. Both are valuable, and they can work together in sophisticated systems.
# Prompts
Source: https://openrewardstandard.io/concepts/prompts
Initial instructions for episodes
Prompts are the initial instructions given to agents at the start of an episode. They provide the context and goal for the task, forming the agent's initial observation.
## What is a Prompt?
A **prompt** is a sequence of blocks (text and/or images) that describes the task to the agent:
```typescript theme={null}
type Prompt = Blocks // Array of TextBlock | ImageBlock
```
**Purpose**:
* Tell the agent what problem to solve
* Provide initial context
* Set expectations for the episode
## Accessing Prompts
Prompts are retrieved via the API after creating a session:
```http theme={null}
GET /gsm8k/prompt
X-Session-ID: abc-123
```
Response:
```json theme={null}
[
{
"text": "What is 2+2?",
"detail": null,
"type": "text"
}
]
```
**Typical flow**:
1. Create session with task
2. Get prompt for that task
3. Agent reads prompt to understand what to do
4. Agent begins calling tools to solve task
## Prompt Structure
### Text Prompts
Simple text instructions:
```json theme={null}
[
{
"text": "Solve this math problem: If x + 5 = 12, what is x?",
"detail": null,
"type": "text"
}
]
```
### Multi-Line Prompts
Complex instructions:
```json theme={null}
[
{
"text": "You are given a Python programming challenge.\n\nWrite a function that reverses a string.\n\nRequirements:\n-Function name: reverse_string\n-Input: string\n-Output: reversed string\n\nYou have access to a Python REPL. Submit your function when ready.",
"detail": null,
"type": "text"
}
]
```
### Multi-Modal Prompts
Text + images:
```json theme={null}
[
{
"text": "What object is shown in this image?",
"detail": null,
"type": "text"
},
{
"data": "iVBORw0KGgoAAAANSUhEUgA...",
"mimeType": "image/jpeg",
"detail": null,
"type": "image"
}
]
```
## Generating Prompts
Prompts are generated by an environment's `get_prompt()` method. For example, in the Python SDK:
```python theme={null}
from ors import Environment, tool, Blocks, TextBlock, ToolOutput
class MathEnvironment(Environment):
def get_prompt(self) -> Blocks:
"""Generate prompt from task"""
question = self.task_spec["question"]
return [
TextBlock(
text=f"Solve this math problem: {question}",
type="text"
)
]
```
**Key points**:
* Prompts are task-specific
* Use `self.task_spec` to access task data
* Return list of blocks (even for single text)
## Prompt Design Patterns
### Pattern 1: Simple Question
Direct question from task:
```python theme={null}
def get_prompt(self) -> Blocks:
return [TextBlock(text=self.task_spec["question"])]
```
Example:
> "What is the capital of France?"
***
### Pattern 2: Contextual Instructions
Add context and instructions:
```python theme={null}
def get_prompt(self) -> Blocks:
problem = self.task_spec["problem"]
prompt_text = f"""You are a coding assistant. Solve the following problem:
{problem}
You have access to a Python REPL. Use the 'python' tool to write and test code.
When you have a solution, use the 'submit' tool.
"""
return [TextBlock(text=prompt_text)]
```
Note the tool instructions may be redundant if the available are already passed into the agent's context.
***
### Pattern 3: Role-Playing
Set agent persona:
```python theme={null}
def get_prompt(self) -> Blocks:
scenario = self.task_spec["scenario"]
prompt_text = f"""You are a customer service agent.
A customer has the following issue:
{scenario}
Your goal is to resolve their issue professionally and efficiently.
Use the 'respond' tool to communicate with the customer.
"""
return [TextBlock(text=prompt_text)]
```
***
### Pattern 4: Multi-Modal
Include images:
```python theme={null}
import base64
def get_prompt(self) -> Blocks:
question = self.task_spec["question"]
image_path = self.task_spec["image_path"]
# Load and encode image
with open(image_path, "rb") as f:
image_data = base64.b64encode(f.read()).decode()
return [
TextBlock(text=question),
ImageBlock(
data=image_data,
mimeType="image/png",
type="image"
)
]
```
***
### Pattern 5: Few-Shot Examples
Provide examples in prompt:
```python theme={null}
def get_prompt(self) -> Blocks:
problem = self.task_spec["problem"]
prompt_text = f"""Solve the following math problem.
Examples:
-Problem: "2 + 2" → Answer: 4
-Problem: "10 - 3" → Answer: 7
-Problem: "5 * 6" → Answer: 30
Now solve:
Problem: "{problem}"
Use the 'submit' tool with your answer.
"""
return [TextBlock(text=prompt_text)]
```
## Dynamic Prompts
Prompts can be customized based on task properties:
```python theme={null}
def get_prompt(self) -> Blocks:
difficulty = self.task_spec.get("difficulty", "medium")
if difficulty == "easy":
instructions = "This is a simple problem. Take your time."
elif difficulty == "hard":
instructions = "This is a challenging problem. Think carefully and use all available tools."
else:
instructions = "Solve this problem step by step."
problem = self.task_spec["problem"]
return [TextBlock(text=f"{instructions}\n\nProblem: {problem}")]
```
## Next Steps
Design tools agents use after reading prompts
Organize tasks that prompts are generated from
Implement get\_prompt() in your environment
See prompt data structure (Blocks)
***
**Key Takeaway**: Prompts are the agent's starting point for each episode. Design them to be clear, specific, and informative. Good prompts guide agents toward successful task completion by setting context.
# Rewards
Source: https://openrewardstandard.io/concepts/rewards
How rewards work in ORS
A reward signal defines the goal in a reinforcement learning problem. Typically this is a single number (scalar feedback) that tells an agent how good or bad an outcome is in an immediate sense.
In ORS, rewards are obtained by executing tools and are returned as part of a `ToolOutput`. They can be used for training in reinforcement learning, or for agentic evaluation - for example, binary reward
can be used for computing accuracy.
## What is a Reward?
A reward is a **numeric signal** (typically a float) that indicates how good an outcome was:
```typescript theme={null}
interface ToolOutput {
blocks: Blocks
reward?: number // ← The reward signal
finished: boolean
metadata?: JSONObject
}
```
**Characteristics**:
* Optional field (can be null)
* Often normalised in the range \[-1, 1] or \[0, 1]
* Returned with every tool call
* Accumulated over an episode
### Reward Examples
**Positive reward** (success):
```json theme={null}
{
"blocks": [{"text": "Correct!", "detail": null, "type": "text"}],
"metadata": null,
"reward": 1.0,
"finished": true
}
```
**Zero reward** (neutral):
```json theme={null}
{
"blocks": [{"text": "File listed successfully", "detail": null, "type": "text"}],
"metadata": null,
"reward": 0.0,
"finished": false
}
```
**Negative reward** (penalty):
```json theme={null}
{
"blocks": [{"text": "Error: Command failed", "detail": null, "type": "text"}],
"metadata": null,
"reward": -0.1,
"finished": false
}
```
## Reward Design Patterns
### Pattern 1: Sparse Rewards
**Only reward at episode end**:
```python theme={null}
def submit(self, params) -> ToolOutput:
correct = (params.answer == self.task["answer"])
return ToolOutput(
blocks=[TextBlock(text="Correct!" if correct else "Incorrect")],
reward=1.0 if correct else 0.0,
finished=True
)
```
**All intermediate steps**:
```python theme={null}
def bash(self, params) -> ToolOutput:
output = execute_command(params.command)
return ToolOutput(
blocks=[TextBlock(text=output)],
reward=0.0, # No reward for intermediate steps
finished=False
)
```
**Pros**:
* Simple to implement
* Clear success/failure signal
* Easy to understand
**Cons**:
* Hard for agent to learn (delayed feedback)
* Credit assignment problem
* Slow learning
**Best for**: Simple tasks, short-horizon tasks
***
### Pattern 2: Dense Rewards
**Reward every meaningful action**:
```python theme={null}
def bash(self, params) -> ToolOutput:
output = execute_command(params.command)
# Reward progress
reward = 0.0
if "answer.txt" in output:
reward = 0.2 # Found relevant file
if len(output) > 0:
reward += 0.1 # Successful command
return ToolOutput(
blocks=[TextBlock(text=output)],
reward=reward,
finished=False
)
def submit(self, params) -> ToolOutput:
correct = (params.answer == self.task["answer"])
return ToolOutput(
blocks=[TextBlock(text="Correct!" if correct else "Incorrect")],
reward=1.0 if correct else 0.0, # Final reward
finished=True
)
```
**Pros**:
* Faster learning
* Guides agent toward solution
* Better credit assignment
**Cons**:
* Harder to design
* Can bias agent toward suboptimal paths
* Risk of reward hacking
**Best for**: Complex tasks, long-horizon episodes
***
### Pattern 3: Shaped Rewards
**Reward based on distance to goal**:
```python theme={null}
def calculate_progress_reward(self, state) -> float:
"""Reward based on how close to solution"""
# Example: Math problem solving
# Check if agent has seen the problem
if not state["viewed_problem"]:
return 0.0
# Check if agent has attempted calculation
if state["attempted_calculation"]:
reward = 0.3
# Check if calculation is close to answer
if state["last_guess"]:
error = abs(state["last_guess"] - self.task["answer"])
if error < 5:
reward = 0.7 # Getting close!
if error < 1:
reward = 0.9 # Very close!
return reward
return 0.1 # Viewed problem
```
**Pros**:
* Strong learning signal
* Guides exploration
* Accelerates training
**Cons**:
* Requires domain knowledge
* Can create reward hacking opportunities
* Complex to implement
**Best for**: Well-understood domains, complex navigation
***
### Pattern 4: Penalty-Based Rewards
**Penalize bad actions**:
```python theme={null}
def bash(self, params) -> ToolOutput:
# Penalize dangerous commands
dangerous = ["rm -rf", ":(){ :|:& };:", "dd if=/dev/zero"]
if any(cmd in params.command for cmd in dangerous):
return ToolOutput(
blocks=[TextBlock(text="Error: Dangerous command blocked")],
reward=-1.0, # Large penalty
finished=True # End episode
)
# Penalize inefficient actions
if self.step_count > 20:
reward = -0.1 # Penalty for taking too long
else:
reward = 0.0
output = execute_command(params.command)
return ToolOutput(
blocks=[TextBlock(text=output)],
reward=reward,
finished=False
)
```
**Use cases**:
* Safety constraints
* Efficiency requirements
* Avoiding bad behaviors
***
### Pattern 5: Binary Rewards
**Simple success/failure**:
```python theme={null}
def submit(self, params) -> ToolOutput:
correct = (params.answer == self.task["answer"])
return ToolOutput(
blocks=[TextBlock(text="Correct!" if correct else "Incorrect")],
reward=1.0 if correct else 0.0, # Binary: 1 or 0
finished=True
)
```
**Best for**: Classification, Q\&A, simple decisions
## Reward Scales
### Common Reward Ranges
**\[0, 1] scale**:
* 0 = failure
* 1 = perfect success
* 0.5 = partial success
**\[-1, 1] scale**:
* -1 = worst outcome
* 0 = neutral
* +1 = best outcome
**Custom scales**:
* Can use any range, but normalise for RL algorithms
* Some RL algorithms normalise automatically; e.g. group advantage in GRPO
## Cumulative Rewards
In RL, we often care about **total (undiscounted) return**:
```python theme={null}
total_return = 0.0
# Episode loop
while not finished:
...
result = session.call_tool(...)
total_return += result.reward or 0.0
finished = result.finished
print(f"Total return: {total_return}")
```
**Interpretation**:
* Higher total return = better performance
* Compare across episodes for learning progress
* Use for evaluation metrics
## Rewards for Evaluation
Rewards in ORS can be used for evaluation as well as training:
```python theme={null}
# Run evaluation
total_rewards = []
for task in test_tasks:
session = create_session(task)
episode_reward = 0.0
while not finished:
result = agent.step(session)
episode_reward += result.reward or 0.0
finished = result.finished
total_rewards.append(episode_reward)
# Evaluation metrics
avg_reward = sum(total_rewards) / len(total_rewards)
success_rate = sum(1 for r in total_rewards if r > 0.9) / len(total_rewards)
print(f"Average reward: {avg_reward:.3f}")
print(f"Success rate: {success_rate:.1%}")
```
## Next Steps
Design tools that return rewards
Organize tasks for RL training
Understand episode lifecycle and reward accumulation
Build an ORS server with reward logic
***
**Key Takeaway**: Rewards are the learning signal for RL. Design them carefully to align with your true objective, provide timely feedback, and avoid unintended behaviors. Good reward design is critical for successful RL training.
# Tasks & Splits
Source: https://openrewardstandard.io/concepts/tasks-splits
Organising problems for RL training and evaluation
Tasks and splits are how ORS organises problems for training and evaluation. Tasks are the individual problems agents solve, while splits categorize these tasks into organised units - for example train/test splits, or splits for different types of problem (e.g. those which requires CPUs versus GPUs).
## Tasks
### What is a Task?
A **task** is a specific problem for an agent to solve. Each task is represented as a JSON object with task-specific data.
### Task Examples
**Math environment**:
```json theme={null}
{
"question": "If x + 5 = 12, what is x?",
"answer": "7",
"difficulty": "easy"
}
```
**Coding environment**:
```json theme={null}
{
"problem_id": "reverse_string",
"description": "Write a function to reverse a string",
"test_cases": [
{"input": "hello", "output": "olleh"},
{"input": "world", "output": "dlrow"}
],
"time_limit_seconds": 5
}
```
**Web navigation**:
```json theme={null}
{
"task_id": "find_price",
"goal": "Find the price of iPhone 15",
"start_url": "https://example.com",
"success_criteria": "Price found and extracted correctly"
}
```
### Task Lifecycle
```
1. Environment defines tasks
2. Tasks organized into splits (e.g. train/test)
3. Agent requests tasks from a split
4. For each task:
a. Create episode with task
b. Get prompt (derived from task)
c. Solve task via tool calls and receive rewards
d. Receive finished signal
e. Cleanup episode
```
### Accessing Tasks
The simplest way to retrieve tasks is to list all tasks in a split:
```http theme={null}
POST /math/tasks
Content-Type: application/json
{
"split": "train"
}
```
Response:
```json theme={null}
{
"tasks": [
{"question": "What is 2+2?", "answer": "4"},
{"question": "If x + 5 = 12, what is x?", "answer": "7"},
...
],
"env_name": "math"
}
```
For large datasets, loading all tasks at once is wasteful. Use these endpoints for efficient access:
**Count tasks** — `POST /{env_name}/num_tasks`:
```http theme={null}
POST /math/num_tasks
{"split": "train"}
```
Returns the number of tasks in a split.
**Get a single task** — `POST /{env_name}/get_task`:
```http theme={null}
POST /math/get_task
{"split": "train", "index": 42}
```
Returns the task at a specific index.
**Get a range of tasks** — `POST /{env_name}/get_task_range`:
```http theme={null}
POST /math/get_task_range
{"split": "train", "start": 0, "stop": 100}
```
Returns tasks in the range `[start, stop)`. Both `start` and `stop` are optional.
The server validates split names on all task endpoints and returns `400` for invalid splits.
### Task as Episode Input
Tasks are passed when creating episodes. You can provide the task inline via `task_spec`, or reference it by `split` and `index`:
```http theme={null}
POST /create
X-Session-ID: abc-123
{
"env_name": "math",
"task_spec": {
"question": "What is 2+2?",
"answer": "4"
},
"secrets": {}
}
```
Or load directly from a split (the server resolves the task):
```http theme={null}
POST /create
X-Session-ID: abc-123
{
"split": "train",
"index": 0,
"secrets": {}
}
```
All fields are optional: `env_name` defaults to the first registered environment, `secrets` defaults to `{}`. Exactly one of `task_spec` or `split`+`index` must be provided (see [CreateSession](/specification/data-types#createsession)).
The environment uses the task to:
* Generate the initial prompt
* Determine correct answers
* Calculate rewards
* Track episode progress
## Splits
### What is a split?
A **split** is a named category of tasks. Splits organise tasks for different purposes in ML workflows.
**An example split structure**:
* **train** - Tasks for training
* **validation** - Tasks for hyperparameter tuning
* **test** - Tasks for evaluation
### Split Structure
```typescript theme={null}
interface Split {
name: string // Split identifier
type: "train" | "validation" | "test" // Category
}
```
Examples:
```json theme={null}
[
{"name": "train", "type": "train"},
{"name": "validation", "type": "validation"},
{"name": "test", "type": "test"}
]
```
### Accessing Splits
List available splits:
```http theme={null}
GET /math/splits
```
Response:
```json theme={null}
[
{"name": "train", "type": "train"},
{"name": "test", "type": "test"}
]
```
Then request tasks from a specific split:
```http theme={null}
POST /math/tasks
{"split": "train"}
```
## Custom Splits
Environments can define custom splits beyond train/validation/test:
```json theme={null}
[
{"name": "easy", "type": "train"},
{"name": "medium", "type": "train"},
{"name": "hard", "type": "test"},
{"name": "expert", "type": "test"}
]
```
**Use cases**:
* Difficulty-based splits (easy/medium/hard)
* Domain-specific splits (algebra/geometry/calculus)
* Time-based splits (before\_2020/after\_2020)
* Resource-based splits (CPU/GPU sandboxes)
**Type defaults**: Environments can return splits as either `Split` objects or bare strings. The server normalises bare strings: `"train"`, `"validation"`, and `"test"` map to their corresponding type, while any other name defaults to `"type": "validation"`.
**Convention**: When using `Split` objects explicitly, map to standard types:
* Training-related → `"type": "train"`
* Evaluation-related → `"type": "test"`
* Tuning-related → `"type": "validation"`
## Next Steps
Design tools for solving tasks
Create reward signals for tasks
Build an ORS server with tasks
See how tasks are accessed via API
***
**Key Takeaway**: Tasks are the problems agents solve. Splits organize tasks for proper ML workflows. Design task structures that are clear, validated, and organized into train/test splits to enable both learning and fair evaluation.
# Tools
Source: https://openrewardstandard.io/concepts/tools
Understanding tools in ORS
In ORS, **tools are the actions that agents can take**. Every interaction with an environment happens through tool calls. This is the fundamental principle of ORS.
## Core Principle: Actions are Tools
> **The only way agents interact with environments is by calling tools.**
This design choice:
* Leverages existing function calling support from LLM providers
* Provides a clear, structured interface
* Makes agent actions explicit and traceable
* Enables type-safe interactions with JSON Schema
## What is a Tool?
A tool is a function that:
1. Has a name and description
2. Optionally defines input parameters (via JSON Schema)
3. Returns a `ToolOutput` with content, reward, and finished flag
**Example tools**:
* `submit` -Submit an answer to a problem
* `bash` -Execute a bash command
* `read_file` -Read a file's contents
* `web_search` -Search the web
* `python` -Execute Python code
## Tool Specification
Tools are advertised via two endpoints:
* `GET /{env_name}/tools` — shared tools available to all tasks (no session required)
* `GET /{env_name}/task_tools` — shared tools plus task-specific tools (requires `X-Session-ID`, since task-specific tools depend on the active task)
Example response from `GET /{env_name}/tools`:
```json theme={null}
{
"tools": [
{
"name": "submit",
"description": "Submit your answer to the current math problem",
"input_schema": {
"type": "object",
"properties": {
"answer": {
"type": "number",
"description": "Your numeric answer"
}
},
"required": ["answer"]
}
}
]
}
```
### Tool Spec Fields
**`name`** (string, required):
* Tool identifier used in tool calls
* Should be descriptive (e.g., `bash`, not `b`)
* Convention: lowercase with underscores
**`description`** (string, required):
* Human-readable explanation of what the tool does
* Used by LLMs to decide when to call the tool
* Should be clear and specific
**`input_schema`** (object, nullable):
* JSON Schema defining tool parameters
* `null` if the tool takes no parameters (always present in output, never omitted)
* Enables validation and type checking
### JSON Schema for Parameters
The `input_schema` follows [JSON Schema](https://json-schema.org/) specification:
```json theme={null}
{
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "The bash command to execute",
"examples": ["ls -la", "cat file.txt"]
},
"timeout": {
"type": "number",
"description": "Command timeout in seconds",
"default": 30
}
},
"required": ["command"]
}
```
**Supported types**:
* `string`, `number`, `boolean`
* `object` (nested parameters)
* `array` (lists of values)
* `null`
**Schema features**:
* `required` -Mandatory fields
* `default` -Default values
* `enum` -Allowed values
* `description` -Field documentation
* `examples` -Example values
## Calling a Tool
Tools are called via `POST /{env_name}/call` with a JSON body:
```json theme={null}
{
"name": "submit",
"input": {"answer": "42"},
"task_id": "call-001"
}
```
* `name`: Tool to call (required)
* `input`: Parameters matching the tool's `input_schema` (required)
* `task_id`: Optional identifier for [SSE reconnection](/specification/sse) — clients can reconnect and retrieve results within a 60-second window
## Tool Output
Every tool call returns a `ToolOutput`:
```typescript theme={null}
interface ToolOutput {
blocks: Blocks // Content (text/images)
reward?: number // RL feedback signal
finished?: boolean // Episode termination (default: false)
metadata?: JSONObject // Optional extra data
}
```
### Wire Format
Tool call responses are delivered via [Server-Sent Events](/specification/sse) and wrapped in a `RunToolSuccess` or `RunToolError` envelope:
**Successful completion**:
```json theme={null}
{
"ok": true,
"output": {
"blocks": [
{"text": "Correct! The answer is 4.", "detail": null, "type": "text"}
],
"metadata": null,
"reward": 1.0,
"finished": true
}
}
```
**Intermediate step**:
```json theme={null}
{
"ok": true,
"output": {
"blocks": [
{"text": "total 48\ndrwxr-xr-x 8 user staff 256 Jan 1 12:00 .", "detail": null, "type": "text"}
],
"metadata": null,
"reward": 0.0,
"finished": false
}
}
```
**Error**:
```json theme={null}
{
"ok": false,
"error": "Tool 'read_file' failed: File not found"
}
```
### Key Fields
**`blocks`**: The content returned by the tool
* Always an array (even for single text output)
* Can be text, images, or both
* This is what the agent observes
**`reward`**: Feedback for RL training
* Optional (can be null)
* Environment-defined; see [Rewards](/concepts/rewards) for design patterns
**`finished`**: Episode termination signal
* Defaults to `false`; always present in serialized output
* When `true`, episode is complete
* Agent should stop calling tools and cleanup session
**`metadata`**: Optional structured data
* By convention, not included in the agent's context window - but this is not enforced by the protocol
* Used for logging, debugging, analysis
* Can include execution time, resource usage, etc.
## Tool Design Patterns
### Pattern 1: Parameterless Tools
Tools that don't need input:
```json theme={null}
{
"name": "get_hint",
"description": "Get a hint for the current problem",
"input_schema": null
}
```
Called as:
```json theme={null}
{"name": "get_hint", "input": {}}
```
### Pattern 2: Simple Parameter Tools
Tools with basic parameters:
```json theme={null}
{
"name": "submit",
"description": "Submit an answer",
"input_schema": {
"type": "object",
"properties": {
"answer": {"type": "number"}
},
"required": ["answer"]
}
}
```
### Pattern 3: Complex Parameter Tools
Tools with rich parameters:
```json theme={null}
{
"name": "web_search",
"description": "Search the web",
"input_schema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query"
},
"max_results": {
"type": "number",
"description": "Maximum results",
"default": 10,
"minimum": 1,
"maximum": 100
},
"filters": {
"type": "object",
"properties": {
"date_range": {"type": "string"},
"domain": {"type": "string"}
}
}
},
"required": ["query"]
}
}
```
### Pattern 4: Enum Parameters
Tools with constrained choices:
```json theme={null}
{
"name": "change_difficulty",
"description": "Change problem difficulty",
"input_schema": {
"type": "object",
"properties": {
"difficulty": {
"type": "string",
"enum": ["easy", "medium", "hard"],
"description": "Target difficulty level"
}
},
"required": ["difficulty"]
}
}
```
## Task-Specific Tools
Some environments need different tools depending on the task. For example, a multiple-choice task might offer a `select_option` tool, while an open-ended task offers `submit_answer` instead.
ORS supports this through two separate tool-listing endpoints:
| Endpoint | Requires session | Returns |
| ---------------------------- | -------------------- | ---------------------------------------- |
| `GET /{env_name}/tools` | No | Shared tools (constant across all tasks) |
| `GET /{env_name}/task_tools` | Yes (`X-Session-ID`) | Shared tools + task-specific tools |
Since task-specific tools depend on the active task, they can only be resolved within a session. Clients should call `task_tools` instead of `tools` after creating a session to get the complete tool set. If an environment has no task-specific tools, both endpoints return the same result.
## Multi-Modal Tools
Tools can return images and text:
```json theme={null}
{
"blocks": [
{"text": "Here's a visualization of the solution:", "detail": null, "type": "text"},
{
"data": "iVBORw0KGgoAAAANSUhEUgA...",
"mimeType": "image/png",
"detail": null,
"type": "image"
}
],
"metadata": null,
"reward": 0.0,
"finished": false
}
```
**Example use cases**:
* Screenshots from web navigation
* Visual feedback for agents
## Tool Calling Flow
![Tool Calling Flow - Agent to environment tool execution cycle]()
## Best Practices
### 1. Clear Tool Names
```python theme={null}
# Good - descriptive
"submit_answer"
"read_file"
"execute_python"
# Bad - unclear
"do_thing"
"action1"
"tool"
```
### 2. Comprehensive Descriptions
```python theme={null}
# Good - specific and helpful
"Submit your final answer to the current math problem. The problem will be graded and you'll receive a reward."
# No Bad - vague
"Submit answer"
```
### 3. Validate Tool Inputs
```python theme={null}
def submit(self, params: SubmitParams) -> ToolOutput:
if not isinstance(params.answer, (int, float)):
return ToolOutput(
blocks=[TextBlock(text="Error: Answer must be a number")],
reward=-0.1,
finished=True
)
# ... check answer logic
```
### 4. Provide Informative Outputs
```python theme={null}
# Good - explains what happened
ToolOutput(
blocks=[TextBlock(text="Incorrect. Your answer was 5, but the correct answer is 7.")],
reward=0.0,
finished=True
)
# No Bad - no context
ToolOutput(
blocks=[TextBlock(text="Wrong")],
reward=0.0,
finished=True
)
```
### 5. Use `finished` Correctly
```python theme={null}
# Good - clear termination
if answer_is_correct:
return ToolOutput(..., finished=True)
else:
return ToolOutput(..., finished=True) # Task failed
# No Bad - ambiguous
return ToolOutput(..., finished=False) # Never terminates?
```
## Tool Security
### Input Validation
Always validate tool inputs:
* Check types match schema
* Validate ranges and constraints
* Reject malformed inputs
### Resource Limits
Prevent resource exhaustion:
* Set timeouts on tool execution
* Limit output size
## Next Steps
Organise problems for training and evaluation
Design reward signals for RL
Build an ORS server with custom tools
See how tools are listed and called
***
**Key Takeaway**: Tools are the agent's interface to the environment. Design them carefully with clear names, comprehensive descriptions, proper validation, and informative outputs. The quality of your tools directly impacts agent performance.
# Implementing an ORS Client
Source: https://openrewardstandard.io/guides/implementing-client
Build clients that interact with ORS servers
Learn how to build clients that interact with ORS servers for training and evaluating agents.
## Two Approaches
### Option 1: Use ORS Python SDK
**Simplest approach** - handles all protocol details:
```python theme={null}
from ors.client import ORS
client = ORS(base_url="http://localhost:8080")
env = client.environment("myenv")
# Run episode
with env.session(task=task) as session:
prompt = session.get_prompt()
result = session.call_tool("submit", {"answer": "42"})
```
### Option 2: Custom HTTP Client
**For other languages** - implement HTTP protocol:
## Python HTTP Client Example
```python theme={null}
import httpx
import json
class ORSClient:
def __init__(self, base_url: str):
self.base_url = base_url
self.client = httpx.Client(timeout=60.0)
def list_tools(self, env_name: str):
"""Get available tools"""
response = self.client.get(f"{self.base_url}/{env_name}/tools")
response.raise_for_status()
return response.json()["tools"]
def list_tasks(self, env_name: str, split: str):
"""Get tasks for split"""
response = self.client.post(
f"{self.base_url}/{env_name}/tasks",
json={"split": split}
)
response.raise_for_status()
return response.json()["tasks"]
def create_session(self):
"""Generate session ID"""
response = self.client.post(f"{self.base_url}/create_session")
response.raise_for_status()
return response.json()["sid"]
def create_episode(self, session_id: str, env_name: str, task_spec: dict):
"""Create episode instance"""
response = self.client.post(
f"{self.base_url}/create",
headers={"X-Session-ID": session_id},
json={
"env_name": env_name,
"task_spec": task_spec,
"secrets": {}
}
)
response.raise_for_status()
return response.json()
def get_prompt(self, session_id: str, env_name: str):
"""Get initial prompt"""
response = self.client.get(
f"{self.base_url}/{env_name}/prompt",
headers={"X-Session-ID": session_id}
)
response.raise_for_status()
return response.json()
def call_tool(self, session_id: str, env_name: str, tool_name: str, tool_input: dict):
"""Call tool (handles SSE)"""
with self.client.stream(
"POST",
f"{self.base_url}/{env_name}/call",
headers={
"X-Session-ID": session_id,
"Accept": "text/event-stream"
},
json={"name": tool_name, "input": tool_input}
) as response:
response.raise_for_status()
buffer = ""
event_type = ""
for line in response.iter_lines():
if line.startswith("event: "):
event_type = line[7:]
elif line.startswith("data: "):
data = line[6:]
if event_type == "end":
complete_data = buffer + data
result = json.loads(complete_data)
if result["ok"]:
return result["output"]
else:
raise Exception(result["error"])
elif event_type == "chunk":
buffer += data
elif event_type == "error":
raise Exception(data)
def ping(self, session_id: str):
"""Keep session alive (prevents 15-minute timeout)"""
response = self.client.post(
f"{self.base_url}/ping",
headers={"X-Session-ID": session_id}
)
response.raise_for_status()
return response.json()
def delete_episode(self, session_id: str):
"""Clean up episode"""
response = self.client.post(
f"{self.base_url}/delete",
headers={"X-Session-ID": session_id}
)
response.raise_for_status()
return response.json()
# Usage
client = ORSClient("http://localhost:8080")
# List available tools
tools = client.list_tools("math")
print(f"Available tools: {[t['name'] for t in tools]}")
# Get tasks
tasks = client.list_tasks("math", "train")
# Run episode
task = tasks[0]
session_id = client.create_session()
try:
client.create_episode(session_id, "math", task)
prompt = client.get_prompt(session_id, "math")
print(f"Prompt: {prompt[0]['text']}")
result = client.call_tool(session_id, "math", "submit", {"answer": "4"})
print(f"Reward: {result['reward']}")
print(f"Finished: {result['finished']}")
finally:
client.delete_episode(session_id)
```
**Session Timeouts**: ORS sessions expire after 15 minutes of inactivity. For long-running episodes, call `client.ping(session_id)` periodically to keep the session alive. The reference SDK pings automatically every 10 seconds.
## JavaScript/TypeScript Client
```typescript theme={null}
class ORSClient {
constructor(private baseUrl: string) {}
async listTools(envName: string) {
const response = await fetch(`${this.baseUrl}/${envName}/tools`);
const data = await response.json();
return data.tools;
}
async createSession(): Promise {
const response = await fetch(`${this.baseUrl}/create_session`, {
method: 'POST'
});
const data = await response.json();
return data.sid;
}
async callTool(
sessionId: string,
envName: string,
toolName: string,
toolInput: any
) {
const response = await fetch(`${this.baseUrl}/${envName}/call`, {
method: 'POST',
headers: {
'X-Session-ID': sessionId,
'Accept': 'text/event-stream',
'Content-Type': 'application/json'
},
body: JSON.stringify({ name: toolName, input: toolInput })
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
let eventType = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
const lines = chunk.split(/\r?\n/);
for (const line of lines) {
if (line.startsWith('event: ')) {
eventType = line.slice(7).trim();
} else if (line.startsWith('data: ')) {
const data = line.slice(6).trim();
if (eventType === 'end') {
const result = JSON.parse(buffer + data);
if (result.ok) {
return result.output;
} else {
throw new Error(result.error);
}
} else if (eventType === 'chunk') {
buffer += data;
} else if (eventType === 'error') {
throw new Error(data);
}
}
}
}
}
}
// Usage
const client = new ORSClient('http://localhost:8080');
const sessionId = await client.createSession();
// ... create episode, call tools
```
## Next Steps
Complete endpoint documentation
See complete client example
***
**Key Takeaway**: Building an ORS client is straightforward HTTP programming. Handle SSE for tool calls, manage session IDs, and implement proper error handling. Use the Python SDK for quick development or implement custom clients in any language.
# Implementing an ORS Server
Source: https://openrewardstandard.io/guides/implementing-server
Build ORS servers in any language
This guide shows how to implement an ORS server, either through the Python SDK or from scratch in any language.
## Two Approaches
### Option 1: Use [ORS Python SDK](https://github.com/openrewardstandard/python-sdk)
* **Pros**: Fast, handles protocol details, well-tested
* **Cons**: Python only
### Option 2: Implement HTTP Protocol
* **Pros**: Any language, full control, no dependencies
* **Cons**: More work, must handle all protocol details
## Option 1: Python SDK Implementation
```python theme={null}
from ors import Environment, Server, tool, ToolOutput, TextBlock, Split
from pydantic import BaseModel
class MyToolParams(BaseModel):
param1: str
param2: int
class MyEnvironment(Environment):
"""Your environment description"""
@classmethod
def list_splits(cls):
"""Return available splits"""
return [Split(name="train", type="train"), Split(name="test", type="test")]
@classmethod
def list_tasks(cls, split: str):
"""Return tasks for split"""
if split == "train":
return [{"description": "Task 1"}, {"description": "Task 2"}]
return [{"description": "Test task"}]
def get_prompt(self):
"""Generate prompt from task"""
return [TextBlock(text=f"Task: {self.task_spec['description']}")]
@tool
def my_tool(self, params: MyToolParams) -> ToolOutput:
"""Tool implementation"""
return ToolOutput(
blocks=[TextBlock(text="Result")],
reward=1.0,
finished=True
)
# Run server
if __name__ == "__main__":
server = Server([MyEnvironment])
server.run(port=8080)
```
### Key Methods
**`list_splits(cls)` - Required**:
* Class method (no instance needed)
* Returns list of split names or Split objects
* Example: `return ["train", "validation", "test"]`
**`list_tasks(cls, split)` - Required**:
* Class method
* Returns list of task objects (JSON)
* Can be async (return `Awaitable`)
* Task structure is environment-specific
**`get_prompt(self)` - Required**:
* Instance method (has access to `self.task_spec`)
* Returns Blocks (list of TextBlock/ImageBlock)
* Called once per episode
* Can be async
**`setup(self)` - Optional**:
* Called when episode starts
* Initialize environment state
* Can be async
**`teardown(self)` - Optional**:
* Called when episode ends
* Cleanup resources
* Can be async
**`num_tasks(cls, split)` - Optional override**:
* Class method, async
* Returns count of tasks for a split
* Default: calls `list_tasks()` and returns `len()`
* Override for efficiency with large task sets
**`get_task(cls, split, index)` - Optional override**:
* Class method, async
* Returns a single task by index
* Default: calls `list_tasks()` and indexes into result
* Override for lazy loading
**`get_task_range(cls, split, start, stop)` - Optional override**:
* Class method, async
* Returns tasks for `range(start, stop)`, supports negative indices and `None`
* Default: calls `get_task()` for each index
### Tool Decorator
```python theme={null}
@tool
def submit(self, params: SubmitParams) -> ToolOutput:
"""Tool docstring becomes description"""
return ToolOutput(...)
```
**Requirements**:
* Decorated with `@tool`
* Takes `self` and optionally one Pydantic model parameter
* Returns `ToolOutput`
* Can be async
Tools can also take zero parameters (no Pydantic model needed). In that case `input_schema` is `null` in the tool listing:
```python theme={null}
@tool
def reset(self) -> ToolOutput:
"""Reset the environment"""
return ToolOutput(
blocks=[TextBlock(text="Environment reset")],
reward=0.0,
finished=False
)
```
### Task-Specific Tools
By default, tools are **shared** and visible to all tasks via `GET /{env_name}/tools`. You can mark tools as task-specific so they only appear within an active session:
```python theme={null}
@tool(shared=False)
def task_only_action(self, params: ActionParams) -> ToolOutput:
"""Only visible via /{env_name}/task_tools in an active session"""
return ToolOutput(...)
```
You can also override `list_task_tools()` to return tools dynamically based on the current task:
```python theme={null}
def list_task_tools(self) -> ListToolsOutput:
"""Override to provide task-specific tools"""
# Return different tools based on self.task_spec
return ListToolsOutput(tools=[...])
```
Clients use `GET /{env_name}/task_tools` (with an active session) to get the combined set of shared + task-specific tools.
### Complete Example
See [Quick Start](/quickstart) for a working GSM8K environment.
## Option 2: Custom HTTP Implementation
### Required Endpoints
Implement these HTTP endpoints:
**Discovery** (no session required):
* `GET /health`
* `GET /list_environments`
* `GET /{env_name}/tools`
* `GET /{env_name}/splits`
* `POST /{env_name}/tasks`
* `POST /{env_name}/num_tasks`
* `POST /{env_name}/task`
* `POST /{env_name}/task_range`
* `POST /create_session`
**Session Management** (requires X-Session-ID):
* `POST /create`
* `POST /delete`
* `POST /delete_session`
* `POST /ping`
**Episode Interaction** (requires X-Session-ID):
* `GET /{env_name}/prompt`
* `GET /{env_name}/task_tools`
* `POST /{env_name}/call` (returns result via SSE)
See [HTTP API Reference](/specification/http-api) for complete specs.
### Example: Node.js/TypeScript
```typescript theme={null}
import express from 'express';
import crypto from 'crypto';
const app = express();
app.use(express.json());
const CHUNK_SIZE = 4096;
// In-memory session store
const sessions = new Map();
// Health check
app.get('/health', (req, res) => {
res.json({ status: 'ok' });
});
// List environments
app.get('/list_environments', (req, res) => {
res.json(['myenvironment']);
});
// List tools
app.get('/:envName/tools', (req, res) => {
res.json({
tools: [
{
name: 'submit',
description: 'Submit answer',
input_schema: {
type: 'object',
properties: {
answer: { type: 'string' }
},
required: ['answer']
}
}
]
});
});
// Create session (no X-Session-ID required)
app.post('/create_session', (req, res) => {
const sid = crypto.randomUUID();
res.json({ sid });
});
// Create episode
app.post('/create', (req, res) => {
const sid = req.headers['x-session-id'] as string;
const { env_name, task_spec, split, index, secrets } = req.body;
// Resolve task_spec from split/index if not provided directly
let resolvedTaskSpec = task_spec;
if (!task_spec && split !== undefined && index !== undefined) {
resolvedTaskSpec = getTaskByIndex(split, index);
}
const env = new EnvironmentInstance(resolvedTaskSpec);
sessions.set(sid, env);
res.json({ sid });
});
// Call tool (SSE with chunking)
app.post('/:envName/call', async (req, res) => {
const sid = req.headers['x-session-id'] as string;
const { name, input } = req.body;
const env = sessions.get(sid);
if (!env) {
return res.status(404).json({ error: 'Session not found' });
}
// Set SSE headers
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
// Send task ID
const taskId = crypto.randomUUID();
res.write(`event: task_id\ndata: ${taskId}\n\n`);
// Execute tool
try {
const result = await env.callTool(name, input);
const resultJson = JSON.stringify({ ok: true, output: result });
// Chunk large responses (>4KB)
if (resultJson.length > CHUNK_SIZE) {
for (let i = 0; i < resultJson.length; i += CHUNK_SIZE) {
const chunk = resultJson.slice(i, i + CHUNK_SIZE);
const event = i + CHUNK_SIZE >= resultJson.length ? 'end' : 'chunk';
res.write(`event: ${event}\ndata: ${chunk}\n\n`);
}
} else {
res.write(`event: end\ndata: ${resultJson}\n\n`);
}
} catch (error) {
res.write(`event: error\ndata: ${error.message}\n\n`);
}
res.end();
});
app.listen(8080, () => {
console.log('ORS server running on port 8080');
});
```
The SSE chunking protocol is important for interoperability. Results larger than 4KB must be split into `chunk` events followed by a final `end` event. Clients concatenate all `chunk` data with the `end` data to reconstruct the full JSON. See the [SSE specification](/specification/sse) for details.
### Session Management
**Key points**:
* Store session ID → environment instance mapping
* `/create_session` generates a session ID (no headers required)
* `/create` binds a session to an environment + task (requires X-Session-ID)
* `/create` accepts either `task_spec` directly, or `split` + `index` to resolve the task server-side
* Implement 15-minute inactivity timeout (reset on any request with that session ID)
* Clean up on `/delete` (call environment teardown)
* Handle concurrent sessions
### Error Handling
Return proper HTTP status codes:
* `400` - Bad request (invalid input, missing X-Session-ID)
* `404` - Not found (session, tool, environment)
* `410` - Gone (session was deleted but still referenced)
* `500` - Server error
For tool errors, use SSE `error` event.
## Testing Your Server
**Quick test**:
```bash theme={null}
# Start server
python server.py
# Test in another terminal
curl http://localhost:8080/health
curl http://localhost:8080/list_environments
```
## Deployment
### Local Development
```bash theme={null}
python server.py
```
### Production Deployment
**Docker**:
```dockerfile theme={null}
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "server.py"]
```
**Run**:
```bash theme={null}
docker build -t my-ors-server .
docker run -p 8080:8080 my-ors-server
```
### Cloud Deployment
Deploy to any cloud platform:
* **AWS**: ECS, Lambda, EC2
* **GCP**: Cloud Run, Compute Engine
* **Azure**: Container Instances, App Service
* **Fly.io**: `fly launch`
* **Railway**: Connect GitHub repo
## Next Steps
Complete endpoint documentation
See a complete working example
***
**Key Takeaway**: Implementing an ORS server is straightforward. Use the Python SDK for quick development, or implement the HTTP protocol in any language for full control. Focus on proper reward design and episode termination.
# What is ORS?
Source: https://openrewardstandard.io/index
The **Open Reward Standard (ORS)** is an open-source HTTP-based protocol for connecting AI models to reinforcement learning (RL) environments.
It specifies how an AI model can interact with an environment to manipulate its state and obtain results and rewards.
![ORS Architecture — Agent interacting with an ORS Environment Server]()
### Key Features
ORS is designed for **reinforcement learning** and **agentic evaluation**. Its key features include:
* **Episodes**: Sessions are RL episodes that continue until a `finished` signal
* **Rewards**: Numeric feedback that can be used for reinforcement learning
* **Tool calling**: Actions are tools - agents interact with an environment via function calling
* **Tasks & Splits**: Tasks are organised into splits for training and evaluation
* **Language-agnostic**: the underlying HTTP protocol can be implemented in any language
### Example Server
Here is a server written with the [example Python SDK](https://github.com/openrewardstandard/python-sdk).
```python theme={null}
from pydantic import BaseModel
from ors import Environment, Server, tool, ToolOutput, TextBlock, Split
class SubmitInput(BaseModel):
answer: str
class MathEnv(Environment):
def __init__(self, task_spec=None, secrets=None):
super().__init__(task_spec=task_spec or {}, secrets=secrets or {})
@classmethod
def list_splits(cls):
return [Split(name="test", type="test")]
@classmethod
def list_tasks(cls, split: str):
return [{"question": "What is 2+2?", "answer": "4"}]
def get_prompt(self):
return [TextBlock(text=self.task_spec["question"])]
@tool
def submit(self, params: SubmitInput) -> ToolOutput:
correct = params.answer.strip() == self.task_spec["answer"]
return ToolOutput(
blocks=[TextBlock(text="Correct!" if correct else "Incorrect")],
reward=1.0 if correct else 0.0,
finished=True,
)
if __name__ == "__main__":
Server([MathEnv]).run(port=8080)
```
### Example Client
Here is a client written with the [example Python SDK](https://github.com/openrewardstandard/python-sdk).
```python theme={null}
from ors.client import ORS
client = ORS(base_url="http://localhost:8080")
env = client.environment("mathenv")
tasks = env.list_tasks(split="test")
tools = env.list_tools()
with env.session(task=tasks[0]) as session:
prompt = session.get_prompt()
print(f"Question: {prompt[0].text}")
result = session.call_tool("submit", {"answer": "4"})
print(f"Reward: {result.reward}, Finished: {result.finished}")
```
### Core Concepts
An ORS server provides access to:
1. **Tools** - Core methods for interacting with environments (e.g., `bash`, `submit_solution`)
2. **Tasks** - Specific problems to be accomplished (e.g., math problems, coding challenges)
3. **Splits** - Categorised lists of tasks (e.g. train, validation, test)
4. **Prompts** - Instructions given to the agent for each task
5. **Rewards** - Numeric feedback signals for RL training
6. **Episodes** - Stateful sessions that continue until `finished: true`
### Actions are Tools
A key principle in ORS: **the only way agents interact with environments is by calling tools.**
This design:
* Leverages existing function calling support from LLM providers
* Provides a clear interface boundary
* Makes agent actions explicit and traceable
This design also enforces a Cartesian boundary between the agent and the environment. ORS environments make no assumptions about the agent that is interacting with them. For example, there is no notion of chat messages or tokenised outputs. This avoids entanglement with the core agent loop.
## Why ORS?
### Primary use case: RL training
ORS allows you to write environments for reinforcement learning:
* **Reward signals**: Actions yield numeric rewards that can be used in RL
* **Episode structure**: Sessions are episodes with `finished` signals from tools
* **State manipulation**: Agents interact with stateful environments over multiple steps by calling tools
### Secondary use case: Evaluation
ORS also excels at agentic evaluation:
* Structured benchmarks with train/test splits
* Reproducible evaluation across different agents
* Standard interface for diverse task types
## How does ORS compare to MCP?
The [Model Context Protocol](https://modelcontextprotocol.io/docs/getting-started/intro) (MCP) is excellent for connecting LLMs to tools and data sources. But it serves a more specific purpose of providing tool access, rather than the full set of primitives for reinforcement learning:
| Feature | MCP | ORS |
| ----------------------- | ---------------------- | ---------------------------- |
| **Purpose** | Tool access, workflows | **RL training environments** |
| **Episode termination** | No | Yes - `finished` signal |
| **Rewards** | No | Yes - For RL training |
| **Tasks & Splits** | No | Yes - Train/validation/test |
| **Tool calling** | Yes | Yes |
| **Protocol** | JSON-RPC | HTTP/REST + SSE |
**Key difference**: ORS includes reward and finished signals that enable reinforcement learning, plus task organization for training and evaluation.
ORS and MCP serve complementary purposes. Use MCP for general tool access, ORS for RL training and structured evaluation.
## Next Steps
Read the [introduction](/introduction) to learn core concepts
Follow the [quick start](/quickstart) to run a local ORS server
Use the [implementation guide](/guides/implementing-server) to create an ORS server
## Example Implementations
Looking for existing ORS environment implementations to reference or use?
Browse reference ORS environment implementations
***
**Note**: The [ORS Python SDK](https://github.com/openrewardstandard/python-sdk) is one implementation of ORS. The standard itself is language-agnostic and can be implemented in Python, TypeScript, Go, Rust, or any language with HTTP support.
# Introduction to ORS
Source: https://openrewardstandard.io/introduction
Understanding the Open Reward Standard
The **Open Reward Standard (ORS)** is an HTTP-based protocol that standardises how agents interact with reinforcement learning environments.
It defines:
* How agents discover available **tools** (actions they can take)
* How agents access **tasks** (problems to solve)
* How agents receive **rewards** (feedback signals for RL training)
* How **episodes** progress until completion (via `finished` signals)
In ORS, an environment is a server that agents connect to via HTTP. The server implements the ORS protocol, providing endpoints for tool discovery, task retrieval, and tool execution.
## Key Principle: Actions are Tools
A fundamental assumption in ORS:
> **The only way agents interact with environments is by calling tools.**
This design decision has important benefits:
* **Leverages existing capabilities**: Major LLMs support function calling
* **Clear interface boundary**: Agent actions are explicit and well-defined
* **Traceable interactions**: Every action is a structured function call
* **Type safety**: Tools have schemas defining their inputs and outputs
For example, in a math environment, the agent might have access to a `submit` tool which it can use to submit an answer to a prompt:
```python theme={null}
{
"name": "submit",
"description": "Submit an answer to the current math problem",
"input_schema": {
"type": "object",
"title": "AnswerParams",
"properties": {
"answer": {"type": "number"}
},
"required": ["answer"]
}
}
```
## Primary use case: Reinforcement Learning
ORS is designed to accelerate **agentic reinforcement learning**, by making it easier to define and interact with RL environments.
### How RL works with ORS
1. An agent is connected to a task from an ORS environment via its exposed tools, and the initial prompt detailing the task to be accomplished.
2. The agent executes tools, receives tool output and rewards, and continues until a finished signal.
3. At the end of the episode, we have a trajectory as well as rewards we can use for credit assignment.
4. We use an RL algorithm of choice, e.g. [GRPO](https://arxiv.org/pdf/2402.03300) based policy gradient.
### Example: Math Problem Solving
Consider training an agent on math problems. Here's the protocol flow:
```
1. List available tasks
POST /math/tasks {"split": "train"}
→ {"tasks": [{"question": "If x + 5 = 12, what is x?", "answer": "7"}, ...]}
2. Create session
POST /create_session
→ {"sid": "session-123"}
3. Create episode with a task
POST /create
Headers: X-Session-ID: session-123
Body: {"env_name": "math", "task_spec": {"question": "If x + 5 = 12, what is x?", "answer": "7"}}
4. Get initial prompt
GET /math/prompt
Headers: X-Session-ID: session-123
→ [{"text": "If x + 5 = 12, what is x?", "detail": null, "type": "text"}]
5. Call submit tool
POST /math/call
Headers: X-Session-ID: session-123
Body: {"name": "submit", "input": {"answer": "7"}}
→ (SSE) {"ok": true, "output": {
"blocks": [{"text": "Correct!", "detail": null, "type": "text"}],
"metadata": null,
"reward": 1.0,
"finished": true
}}
```
To use this data for reinforcement learning, we would need to represent the trajectory in a way that the model can consume - e.g. tokenising it - and using the reward (`1.0`) as part of the gradient update. For example, in GRPO it would be used to calculate the group advantage.
## Secondary use case: Agentic Evaluation
While designed for RL training, ORS can also be used for agentic evaluation:
* **Standardised benchmarks**: Common interface across different environments
* **Train/test splits**: Tasks are organised into clear training/evaluation splits
* **Reproducible results**: The environment is standardised for different agents
* **Diverse task types**: ORS supports tasks from basic question/answer environments to more complicated agentic workflows involving sandbox execution and computer-use.
## Core Components
An ORS server provides access to four core components:
### 1. Tools
**Tools** are the actions available to agents. Each tool has:
* A name (e.g., `bash`, `submit`, `read_file`)
* A description explaining what it does
* An input schema (JSON Schema) defining parameters
* A return type (ToolOutput with blocks, reward, finished)
### 2. Tasks
**Tasks** are the problems agents need to solve. Each task is a JSON object containing problem-specific data:
```json theme={null}
{
"question": "What is 2+2?",
"ground_truth": 4,
"difficulty": "easy"
}
```
The structure is environment-specific. Math environments have questions and answers. Coding environments have problem descriptions and test cases.
### 3. Splits
**Splits** can be used to organise tasks into categories.
For example, standard splits can be used:
* `train` - Tasks for training agents
* `validation` - Tasks for hyperparameter tuning
* `test` - Tasks for final evaluation
But also custom splits, for example versions of the environment, or splits with different hardware requirements (e.g. CPU or GPU).
### 4. Prompts
**Prompts** are the initial instructions given to agents for each task. They're returned as blocks (text or images):
```python theme={null}
# Agent gets prompt at start of episode
prompt = session.get_prompt()
# → [TextBlock(text="What is 2+2?")]
```
Prompts can be multi-modal (text + images) and are generated dynamically based on the task.
## Episodes are sessions
In ORS, **a session is an RL episode**.
### Episode Lifecycle
```
1. Create session → Start episode with a specific task
2. Get prompt → Receive initial state
3. Call tools → Take actions, get results and rewards
4. Repeat step 3 → Until finished=True
5. End session → Episode complete
```
**The episode continues until a tool returns `finished: true`**.
This is different from typical API sessions - there's semantic meaning to when an episode ends. It represents task completion (success or failure).
### Episode Example
**Episode 1: Single-step (correct answer)**
```
POST /create_session → session_id_1
POST /create (task: problem_1)
POST /env/call ("submit", {"answer": 42})
→ finished=true, reward=1.0
```
**Episode 2: Multi-step interaction**
```
POST /create_session → session_id_2
POST /create (task: problem_2)
Step 1: Explore
POST /env/call ("bash", {"command": "cat question.txt"})
→ finished=false, reward=0.0
Step 2: Solve
POST /env/call ("submit", {"answer": "Tokyo"})
→ finished=true, reward=1.0
```
## Rewards
**Rewards** are numeric feedback signals that enable RL training.
### Reward Design
* **Sparse rewards**: Only at task completion (0 or 1)
* **Dense rewards**: After each action (incremental progress)
* **Shaped rewards**: Guide agent toward solution
Example sparse rewards:
```
POST /env/call ("submit", {"answer": 42})
→ reward=1.0, finished=true # Correct
POST /env/call ("submit", {"answer": 43})
→ reward=0.0, finished=true # Incorrect
```
Example dense rewards (trading environment):
```
POST /env/call ("place_trade", {"ticker": "AAPL", "action": "buy", "quantity": 10})
→ reward=0.0, finished=false # Position opened
POST /env/call ("place_trade", {"ticker": "GOOGL", "action": "buy", "quantity": 5})
→ reward=0.0, finished=false # Another position opened
POST /env/call ("end_day", {})
→ reward=0.023, finished=true # Day settled, portfolio returned +2.3%
```
## Protocol Overview
ORS uses **HTTP + Server-Sent Events** for communication:
### HTTP for Control
Standard REST endpoints for:
* Listing tools, splits, tasks
* Creating/deleting sessions
* Health checks
### SSE for Tool Execution
Tool calls return results via Server-Sent Events:
* Chunks large responses into smaller pieces for reliable delivery
* Keeps connections alive during long-running tool calls
* Allows clients to reconnect and resume results via task IDs
### Language-Agnostic
Because ORS is HTTP-based, ORS can be implemented in any language:
* **Python**: [ORS SDK](https://github.com/openrewardstandard/python-sdk) (reference implementation)
* **TypeScript**: Custom server with Express/Fastify
* **Go**: Custom server with stdlib http
* **Rust**: Custom server with Actix/Axum
## ORS vs MCP
Both ORS and MCP involve agents calling tools, but they serve different purposes:
**MCP (Model Context Protocol)**:
* **Purpose**: Connect LLMs to tools, data sources, workflows
* **Use case**: General-purpose tool access
* **Protocol**: JSON-RPC over stdio/SSE
* **Key feature**: Seamless tool integration
**ORS (Open Reward Standard)**:
* **Purpose**: Connect agents to RL training environments
* **Use case**: Training and evaluating agents
* **Protocol**: HTTP + SSE
* **Key features**: Rewards, episodes, task organization
### What's Different?
ORS adds RL-specific features:
| Feature | MCP | ORS | Why ORS Needs It |
| ------------ | --- | --- | --------------------- |
| **Rewards** | No | Yes | RL training signal |
| **Finished** | No | Yes | Episode termination |
| **Tasks** | No | Yes | Problem organization |
| **Splits** | No | Yes | Train/test separation |
### Can They Work Together?
Yes! They serve complementary purposes:
* **MCP**: Agent uses tools to access external data/APIs
* **ORS**: Agent operates in structured RL environment with rewards
You might use both: an agent in an ORS environment that uses MCP tools to access external resources.
## Next Steps
Build your first ORS server with GSM8K example
Dive into the HTTP API details
Understand tools, tasks, rewards, and prompts
Learn how to implement an ORS server
***
**Key Takeaway**: ORS brings RL to language models by providing a standardised protocol with rewards, episode structure, and task organization.
# Quick Start
Source: https://openrewardstandard.io/quickstart
Build your first ORS server in 15 minutes
Get started with ORS by building a simple math environment server and testing it locally.
## What You'll Build
A working ORS server for math problems (GSM8K-style) with:
* One tool (`submit`) for submitting answers
* Train and test task splits
* Reward signals for RL training
* Local HTTP server you can test with curl or Python
**Time**: \~15 minutes
## Prerequisites
* Python 3.11+ installed
* Basic Python knowledge
* Terminal/command line access
## Step 1: Install Dependencies
The [ORS Python SDK](https://github.com/openrewardstandard/python-sdk) is one implementation of the ORS specification. We'll use it for this quickstart.
```bash theme={null}
pip install ors-sdk pandas
```
## Step 2: Download GSM8K Data
Download the GSM8K dataset from the [HuggingFace repository](https://huggingface.co/datasets/openai/gsm8k/tree/main/main):
1. Download [train-00000-of-00001.parquet](https://huggingface.co/datasets/openai/gsm8k/resolve/main/main/train-00000-of-00001.parquet)
2. Download [test-00000-of-00001.parquet](https://huggingface.co/datasets/openai/gsm8k/resolve/main/main/test-00000-of-00001.parquet)
Place both files in your working directory.
GSM8K is a dataset of grade school math word problems. Each task has a question and an integer answer.
## Step 3: Create Your Environment
Create a file `gsm8k_env.py`:
```python theme={null}
from ors import Environment, Server, tool, ToolOutput, TextBlock
from pydantic import BaseModel
import pandas as pd
# Load GSM8K tasks from parquet files
train_tasks = pd.read_parquet("train-00000-of-00001.parquet").to_dict(orient="records")
test_tasks = pd.read_parquet("test-00000-of-00001.parquet").to_dict(orient="records")
# Add IDs to tasks
for i, task in enumerate(train_tasks):
task['id'] = str(i)
for i, task in enumerate(test_tasks):
task['id'] = str(i)
# Tool parameter schema (must be defined before GSM8KEnvironment)
class SubmitParams(BaseModel):
answer: str
class GSM8KEnvironment(Environment):
"""GSM8K math problem environment"""
@classmethod
def list_splits(cls):
return ["train", "test"]
@classmethod
def list_tasks(cls, split: str):
if split == "train":
return train_tasks
elif split == "test":
return test_tasks
raise ValueError(f"Unknown split: {split}")
def get_prompt(self):
question = self.task_spec["question"]
return [TextBlock(text=question)]
@tool
def submit(self, params: SubmitParams) -> ToolOutput:
"""Submit your answer to the math problem"""
# Extract the final answer from GSM8K format (after ####)
gold_answer = self.task_spec["answer"].split("####")[-1].strip()
user_answer = str(params.answer).strip()
if user_answer == gold_answer:
return ToolOutput(
blocks=[TextBlock(text="Correct!")],
reward=1.0,
finished=True
)
else:
return ToolOutput(
blocks=[TextBlock(text=f"Incorrect. The answer was {gold_answer}.")],
reward=0.0,
finished=True
)
# Create and run server
if __name__ == "__main__":
server = Server([GSM8KEnvironment])
server.run(port=8080)
```
**What this code does**:
* Loads GSM8K tasks from the parquet files
* Defines an ORS environment with math tasks
* Implements `list_splits()`, `list_tasks()`, and `get_prompt()`
* Creates a `submit` tool that checks answers and returns rewards
* Starts an HTTP server on port 8080
## Step 4: Run the Server
```bash theme={null}
python gsm8k_env.py
```
You should see:
```
INFO: Started server process [12345]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
```
Your ORS server is now running!
## Step 5: Test with HTTP
Let's test the server with curl. Open a new terminal:
### List environments
```bash theme={null}
curl http://localhost:8080/list_environments
```
Response:
```json theme={null}
["gsm8kenvironment"]
```
### List tools
```bash theme={null}
curl http://localhost:8080/gsm8kenvironment/tools
```
Response:
```json theme={null}
{
"tools": [
{
"name": "submit",
"description": "Submit your answer to the math problem",
"input_schema": {...}
}
]
}
```
### List splits
```bash theme={null}
curl http://localhost:8080/gsm8kenvironment/splits
```
Response:
```json theme={null}
[
{"name": "train", "type": "train"},
{"name": "test", "type": "test"}
]
```
### List tasks
```bash theme={null}
curl -X POST http://localhost:8080/gsm8kenvironment/tasks \
-H "Content-Type: application/json" \
-d '{"split": "train"}'
```
Response (first 2 tasks shown):
```json theme={null}
{
"tasks": [
{
"id": "0",
"question": "Natalia sold clips to 48 of her friends in April...",
"answer": "Natalia sold 48/2 = <<48/2=24>>24 clips in May.\n...#### 72"
},
{
"id": "1",
"question": "Weng earns $12 an hour for babysitting...",
"answer": "...#### 20"
}
],
"env_name": "gsm8kenvironment"
}
```
The full dataset contains 7,473 training tasks and 1,319 test tasks.
## Step 6: Run an Episode
Now let's run a complete episode (session):
### Create session ID
```bash theme={null}
curl -N -X POST http://localhost:8080/create_session
```
Response (SSE stream):
```
event: task_id
data: abc-123-def-456
event: end
data:
```
The session ID is in the `task_id` event. Save it for the next steps.
### Create episode
Use a task from the dataset:
```bash theme={null}
curl -X POST http://localhost:8080/create \
-H "X-Session-ID: abc-123-def-456" \
-H "Content-Type: application/json" \
-d '{
"env_name": "gsm8kenvironment",
"task_spec": {
"id": "0",
"question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?",
"answer": "Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72"
},
"secrets": {}
}'
```
Response:
```json theme={null}
{"sid":"abc-123-def-456"}
```
### Get prompt
```bash theme={null}
curl http://localhost:8080/gsm8kenvironment/prompt \
-H "X-Session-ID: abc-123-def-456"
```
Response:
```json theme={null}
[
{
"text": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?",
"detail": null,
"type": "text"
}
]
```
### Call submit tool
```bash theme={null}
curl -N -X POST http://localhost:8080/gsm8kenvironment/call \
-H "X-Session-ID: abc-123-def-456" \
-H "Accept: text/event-stream" \
-H "Content-Type: application/json" \
-d '{"name": "submit", "input": {"answer": "72"}}'
```
Response (SSE stream):
```
event: task_id
data: 877bb56c594e4a0f921ad55c439a3762
event: end
data: {"ok":true,"output":{"blocks":[{"text":"Correct!","detail":null,"type":"text"}],"metadata":null,"reward":1.0,"finished":true}}
```
**Success!** The agent got reward 1.0 and `finished: true`.
### Cleanup
```bash theme={null}
curl -X POST http://localhost:8080/delete \
-H "X-Session-ID: abc-123-def-456"
```
## Step 7: Test with Python Client
Create `test_client.py`:
```python theme={null}
from ors.client import ORS
# Connect to local server
client = ORS(base_url="http://localhost:8080")
env = client.environment("gsm8kenvironment")
# Get tasks
tasks = env.list_tasks(split="train")
print(f"Found {len(tasks)} training tasks")
# Run an episode
task = tasks[0] # First task from GSM8K
with env.session(task=task) as session:
# Get prompt
prompt = session.get_prompt()
print(f"Question: {prompt[0].text[:80]}...") # Show first 80 chars
# Submit answer (the correct answer is 72)
result = session.call_tool("submit", {"answer": "72"})
print(f"Result: {result.blocks[0].text}")
print(f"Reward: {result.reward}")
print(f"Finished: {result.finished}")
```
Run it:
```bash theme={null}
python test_client.py
```
Output:
```
Found 7473 training tasks
Question: Natalia sold clips to 48 of her friends in April, and then she sold half...
Result: Correct!
Reward: 1.0
Finished: True
```
## Understanding the Code
### Key Components
**1. Environment Class**
```python theme={null}
class GSM8KEnvironment(Environment):
```
Inherits from `Environment` base class, which handles HTTP protocol details.
**2. Splits and Tasks**
```python theme={null}
@classmethod
def list_splits(cls):
return ["train", "test"]
@classmethod
def list_tasks(cls, split: str):
return [...] # Task list
```
Organize problems into train/test sets.
**3. Prompt Generation**
```python theme={null}
def get_prompt(self):
return [TextBlock(text=f"Solve: {self.task_spec['question']}")]
```
Convert task into initial agent prompt.
**4. Tools**
```python theme={null}
@tool
def submit(self, params: SubmitParams) -> ToolOutput:
# Check answer, return reward and finished signal
```
Actions agents can take. Return `ToolOutput` with reward and finished flag.
**5. Tool Output**
```python theme={null}
ToolOutput(
blocks=[TextBlock(text="Correct!")],
reward=1.0, # RL feedback signal
finished=True # Episode termination
)
```
Structured response with content, reward, and termination signal.
## What You've Learned
* How to implement an ORS server using the Python SDK
* Core ORS concepts: splits, tasks, tools, prompts, rewards
* How sessions (episodes) work
* The HTTP API for ORS
* How to test an ORS server locally
## Next Steps
Add bash, calculator, or other tools to your environment
Learn reward design patterns for RL
Deep dive into building ORS servers
Understand the complete ORS protocol
## Common Issues
### "ModuleNotFoundError: No module named 'ors'"
Install the SDK:
```bash theme={null}
pip install ors-sdk
```
### "404 Environment not found"
Check the environment name matches the class name (lowercase):
* Class: `GSM8KEnvironment`
* Name: `gsm8kenvironment`
### "Connection refused"
Make sure the server is running:
```bash theme={null}
python gsm8k_env.py
```
### "Session not found"
Create a new session ID:
```bash theme={null}
curl -N -X POST http://localhost:8080/create_session
```
# Data Types
Source: https://openrewardstandard.io/specification/data-types
Core data structures in the ORS protocol
This page documents data structures used in the Open Reward Standard. Type definitions are shown in TypeScript notation but apply to any language implementation.
## Core Types
### JSONValue
Recursive type representing any valid JSON value:
```typescript theme={null}
type JSONValue =
| string
| number
| boolean
| null
| { [key: string]: JSONValue }
| JSONValue[]
type JSONObject = { [key: string]: JSONValue }
```
Used throughout the protocol for flexible, task-specific data.
## Block Types
Blocks are the fundamental unit of content in ORS. They can be text or images.
### TextBlock
Text content with optional metadata:
```typescript theme={null}
interface TextBlock {
type: "text"
text: string
detail?: JSONObject
}
```
**Fields**:
* `type`: Always `"text"` (literal)
* `text`: The text content (string)
* `detail`: Optional metadata about this text block (object)
**Example**:
```json theme={null}
{
"type": "text",
"text": "What is 2+2?",
"detail": {"language": "en"}
}
```
***
### ImageBlock
Image content encoded as base64:
```typescript theme={null}
interface ImageBlock {
type: "image"
data: string
mimeType: string
detail?: JSONObject
}
```
**Fields**:
* `type`: Always `"image"` (literal)
* `data`: Base64-encoded image data (string)
* `mimeType`: MIME type like `"image/png"`, `"image/jpeg"` (string)
* `detail`: Optional metadata about this image (object)
**Example**:
```json theme={null}
{
"type": "image",
"data": "iVBORw0KGgoAAAANSUhEUgA...",
"mimeType": "image/png",
"detail": {"width": 800, "height": 600}
}
```
***
### Blocks
Array of blocks (text and/or images):
```typescript theme={null}
type Blocks = Array
```
**Usage**: Prompts and tool outputs are represented as blocks.
**Example** (multi-modal prompt):
```json theme={null}
[
{
"type": "text",
"text": "What object is shown in this image?"
},
{
"type": "image",
"data": "iVBORw0KGgo...",
"mimeType": "image/jpeg"
}
]
```
## Tool Types
### ToolSpec
Specification for an available tool:
```typescript theme={null}
interface ToolSpec {
name: string
description: string
input_schema: JSONObject | null
}
```
**Fields**:
* `name`: Tool identifier, used when calling the tool (string)
* `description`: Human-readable description of what the tool does (string)
* `input_schema`: JSON Schema defining tool parameters, or `null` if the tool takes no parameters (always present in output, nullable)
**Example**:
```json theme={null}
{
"name": "bash",
"description": "Execute a bash command in the environment",
"input_schema": {
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "The bash command to execute"
}
},
"required": ["command"]
}
}
```
**Note**: If `input_schema` is `null`, the tool takes no parameters.
***
### ToolOutput
Result of executing a tool:
```typescript theme={null}
interface ToolOutput {
blocks: Blocks
reward?: number
finished: boolean
metadata?: JSONObject
}
```
**Fields**:
* `blocks`: Output content (array of TextBlock/ImageBlock, required)
* `reward`: RL reward signal (number, optional)
* `finished`: Whether episode is complete (boolean, default: `false`). Optional in input — always present in output.
* `metadata`: Optional additional data (object)
**Example** (successful completion):
```json theme={null}
{
"blocks": [
{"text": "Correct! The answer is 4.", "detail": null, "type": "text"}
],
"metadata": null,
"reward": 1.0,
"finished": true
}
```
**Example** (intermediate step):
```json theme={null}
{
"blocks": [
{"text": "File contents: Hello World", "detail": null, "type": "text"}
],
"metadata": {"file_size": 11},
"reward": 0.0,
"finished": false
}
```
**Key Fields**:
**`finished`**:
* **Critical for episode termination**
* When `true`, the episode is complete
* Agent should stop calling tools and delete the session
* Represents task completion (success or failure)
**`reward`**:
* RL feedback signal
* Environment-defined; common patterns include sparse rewards (only at episode end) and dense rewards (after each action)
***
### ToolCall
Request to execute a tool:
```typescript theme={null}
interface ToolCall {
name: string
input: JSONObject
task_id?: string
}
```
**Fields**:
* `name`: Tool to call (string, required)
* `input`: Tool parameters matching its input\_schema (object, required)
* `task_id`: Optional identifier for SSE reconnection (string). When provided, clients can reconnect to an in-progress or recently completed tool call and retrieve its result within a 60-second linger window.
**Example**:
```json theme={null}
{
"name": "submit",
"input": {"answer": "42"},
"task_id": "trace-123"
}
```
***
### ListToolsOutput
Response from `GET /{env_name}/tools`:
```typescript theme={null}
interface ListToolsOutput {
tools: ToolSpec[]
}
```
**Example**:
```json theme={null}
{
"tools": [
{
"name": "submit",
"description": "Submit an answer",
"input_schema": {...}
},
{
"name": "bash",
"description": "Execute bash command",
"input_schema": {...}
}
]
}
```
## Tool Result Types
Tool execution returns one of two result types:
### RunToolSuccess
Successful tool execution:
```typescript theme={null}
interface RunToolSuccess {
ok: true
output: ToolOutput
}
```
**Example**:
```json theme={null}
{
"ok": true,
"output": {
"blocks": [{"text": "Success", "detail": null, "type": "text"}],
"metadata": null,
"reward": 1.0,
"finished": true
}
}
```
***
### RunToolError
Failed tool execution:
```typescript theme={null}
interface RunToolError {
ok: false
error: string
}
```
**Example**:
```json theme={null}
{
"ok": false,
"error": "Tool 'submit' failed: Answer must be a number"
}
```
***
### RunToolOutput
Union type for tool results:
```typescript theme={null}
type RunToolOutput = RunToolSuccess | RunToolError
```
The `ok` field discriminates between success and error:
* `ok: true` → result has `output` field
* `ok: false` → result has `error` field
**Usage**: Returned in SSE `end` event from `POST /{env_name}/call`.
## Task Types
### Task
Tasks are environment-specific JSON objects:
```typescript theme={null}
type Task = JSONObject
```
**No fixed schema** - each environment defines its own task structure.
**Examples**:
Math environment:
```json theme={null}
{
"question": "What is 2+2?",
"answer": "4",
"difficulty": "easy"
}
```
Coding environment:
```json theme={null}
{
"description": "Write a function to reverse a string",
"test_cases": [
{"input": "hello", "output": "olleh"},
{"input": "world", "output": "dlrow"}
],
"time_limit": 5
}
```
Web navigation:
```json theme={null}
{
"goal": "Find the price of product XYZ",
"start_url": "https://example.com",
"max_steps": 20
}
```
***
### Split
Categorization of task lists:
```typescript theme={null}
interface Split {
name: string
type: "train" | "validation" | "test"
}
```
**Fields**:
* `name`: Split identifier (string)
* `type`: Split category (string, one of: "train", "validation", "test")
**Example**:
```json theme={null}
{
"name": "train",
"type": "train"
}
```
**Common splits**:
* `{"name": "train", "type": "train"}` - Training tasks
* `{"name": "validation", "type": "validation"}` - Validation tasks
* `{"name": "test", "type": "test"}` - Test tasks
**Custom splits**: Environments can define custom splits (e.g., "hard", "easy") which default to type "validation".
***
### ListTasks
Request body for `POST /{env_name}/tasks`:
```typescript theme={null}
interface ListTasks {
split: string
}
```
**Example**:
```json theme={null}
{
"split": "train"
}
```
***
### NumTasks
Request body for `POST /{env_name}/num_tasks`:
```typescript theme={null}
interface NumTasks {
split: string
}
```
**Fields**:
* `split`: Split name to count tasks in (string, required)
***
### GetTask
Request body for `POST /{env_name}/task`:
```typescript theme={null}
interface GetTask {
split: string
index: number
}
```
**Fields**:
* `split`: Split name (string, required)
* `index`: Task index within the split (number, required)
***
### GetTaskRange
Request body for `POST /{env_name}/task_range`:
```typescript theme={null}
interface GetTaskRange {
split: string
start?: number
stop?: number
}
```
**Fields**:
* `split`: Split name (string, required)
* `start`: Start index, inclusive (number, optional)
* `stop`: Stop index, exclusive (number, optional)
## Session Types
### CreateSession
Request to create an episode:
```typescript theme={null}
interface CreateSession {
env_name?: string
task_spec?: JSONObject
split?: string
index?: number
secrets?: { [key: string]: string }
}
```
**Fields**:
* `env_name`: Environment to instantiate (string, optional — defaults to first registered environment)
* `task_spec`: Task data for this episode (object, optional — provide this or `split`+`index`)
* `split`: Split name to load task from (string, optional — required with `index` if `task_spec` not provided)
* `index`: Task index within the split (number, optional — required with `split` if `task_spec` not provided)
* `secrets`: API keys, credentials, etc. (object, optional, defaults to `{}`)
Exactly one of `task_spec` or the `split`+`index` pair must be provided. Providing both or neither is a validation error.
**Example** (with inline task spec):
```json theme={null}
{
"env_name": "math",
"task_spec": {
"question": "If x + 5 = 12, what is x?",
"answer": "7"
},
"secrets": {
"openai_api_key": "sk-..."
}
}
```
**Example** (with split and index):
```json theme={null}
{
"env_name": "math",
"split": "train",
"index": 42,
"secrets": {}
}
```
**Security note**: Secrets are passed to the environment but should not be logged or persisted by the server.
## Type Examples by Use Case
### Discovery Flow
```typescript theme={null}
// 1. List environments
GET /list_environments
→ string[]
// 2. List tools
GET /{env_name}/tools
→ ListToolsOutput
// 3. List splits
GET /{env_name}/splits
→ Split[]
// 4. List tasks
POST /{env_name}/tasks
Body: ListTasks
→ { tasks: Task[], env_name: string }
```
### Episode Flow
```typescript theme={null}
// 1. Create session
POST /create_session
→ { sid: string }
// 2. Create episode
POST /create
Body: CreateSession
Headers: X-Session-ID
→ { sid: string }
// 3. Get prompt
GET /{env_name}/prompt
Headers: X-Session-ID
→ Blocks
// 4. Call tool
POST /{env_name}/call
Body: ToolCall
Headers: X-Session-ID
→ SSE response with RunToolOutput
// 5. Delete episode
POST /delete
Headers: X-Session-ID
→ { sid: string }
```
## Type Validation
Implementations should validate:
### Input Validation
* Tool calls match the tool's `input_schema`
* Required fields are present
* Types match specifications
### Output Guarantees
* `ToolOutput.blocks` is non-empty array
* `ToolOutput.finished` is boolean
* `ToolOutput.reward` is number or null
* Block `type` is exactly "text" or "image"
### Episode Invariants
* Once `finished: true` is returned, no more tools should be called
* Session IDs are unique across the server
* Tool names match those in `list_tools()` output
## Language-Specific Notes
### Python (Reference Implementation)
```python theme={null}
from ors import (
TextBlock,
ImageBlock,
Blocks,
ToolOutput,
ToolSpec,
Split,
)
# Create tool output
output = ToolOutput(
blocks=[TextBlock(text="Correct!", type="text")],
reward=1.0,
finished=True
)
```
### TypeScript
```typescript theme={null}
interface ToolOutput {
blocks: Blocks;
reward?: number;
finished: boolean;
metadata?: JSONObject;
}
// Create tool output
const output: ToolOutput = {
blocks: [{type: "text", text: "Correct!"}],
reward: 1.0,
finished: true
};
```
### JSON Schema
For validation in any language, use JSON Schema:
```json theme={null}
{
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ToolOutput": {
"type": "object",
"properties": {
"blocks": {"type": "array"},
"reward": {"type": ["number", "null"]},
"finished": {"type": "boolean"},
"metadata": {"type": ["object", "null"]}
},
"required": ["blocks", "finished"]
}
}
}
```
## Next Steps
See how these types are used in API endpoints
Understand episode lifecycle and state
Learn about tool design and usage
Implement these types in your server
***
**Key Takeaway**: ORS uses simple, composable types. Blocks provide flexible content representation. ToolOutput bundles content, rewards, and episode termination. Task structure is environment-specific for maximum flexibility.
# HTTP API Reference
Source: https://openrewardstandard.io/specification/http-api
Complete ORS endpoint documentation
This page documents HTTP endpoints in the Open Reward Standard.
## Base URL
All endpoints are relative to the ORS server base URL:
```
http://localhost:8080 # Local development
https://your-server.com # Production
```
## Common Headers
### X-Session-ID
Most endpoints require the `X-Session-ID` header to identify the episode:
```http theme={null}
X-Session-ID: abc-123-def-456
```
**Required for**: `/create`, `/delete`, `/delete_session`, `/ping`, `/{env_name}/call`, `/{env_name}/prompt`, `/{env_name}/task_tools`
**Not required for**: `/create_session`, discovery endpoints, `/health`
## Discovery Endpoints
These endpoints provide information about available environments, tools, and tasks. They generally do not require session IDs, with one exception: listing tools may require an active session when the environment defines task-specific tools (see below).
### GET /health
Health check endpoint.
**Request**:
```http theme={null}
GET /health
```
**Response** (200 OK):
```json theme={null}
{
"status": "ok"
}
```
**Use case**: Verify server is running and responsive.
***
### GET /list\_environments
List all available environments on this server.
**Request**:
```http theme={null}
GET /list_environments
```
**Response** (200 OK):
```json theme={null}
[
"math",
"code_env",
"web_nav"
]
```
Returns an array of environment names (strings). Each name can be used in `/{env_name}/*` endpoints.
**Use case**: Discover what environments are available before creating a session.
***
### GET /\{env\_name}/tools
List all tools available in the specified environment.
**Request**:
```http theme={null}
GET /math/tools
```
**Response** (200 OK):
```json theme={null}
{
"tools": [
{
"name": "submit",
"description": "Submit an answer to the math problem",
"input_schema": {
"type": "object",
"properties": {
"answer": {
"type": "string",
"description": "Your answer to the problem"
}
},
"required": ["answer"]
}
}
]
}
```
**Response Schema**:
* `tools`: Array of ToolSpec objects
* `name`: Tool identifier (string)
* `description`: Human-readable description (string)
* `input_schema`: JSON Schema defining tool parameters (object, optional)
**Error Responses**:
* **404 Not Found**: Environment name not recognized
**Use case**: Discover what actions the agent can take before starting an episode.
Some environments define **task-specific tools** (using `@tool(shared=False)`) that are only available within an active session. Use the `GET /{env_name}/task_tools` endpoint (which requires an active session) to get the combined set of shared and task-specific tools. This sessionless `/{env_name}/tools` endpoint returns only shared tools.
***
### GET /\{env\_name}/splits
List all task splits available in the environment.
**Request**:
```http theme={null}
GET /math/splits
```
**Response** (200 OK):
```json theme={null}
[
{
"name": "train",
"type": "train"
},
{
"name": "test",
"type": "test"
}
]
```
**Response Schema**:
* Array of Split objects
* `name`: Split identifier (string)
* `type`: Split category - "train", "validation", or "test" (string)
**Error Responses**:
* **404 Not Found**: Environment name not recognized
**Use case**: Determine which task sets are available for training vs evaluation.
***
### POST /\{env\_name}/tasks
List tasks for a specific split.
**Request**:
```http theme={null}
POST /math/tasks
Content-Type: application/json
{
"split": "train"
}
```
**Request Schema**:
* `split`: Split name to query (string, required)
**Response** (200 OK):
```json theme={null}
{
"tasks": [
{
"question": "What is 2+2?",
"answer": "4"
},
{
"question": "If x + 5 = 12, what is x?",
"answer": "7"
}
],
"env_name": "math"
}
```
**Response Schema**:
* `tasks`: Array of task objects (structure is environment-specific)
* `env_name`: Environment name (string)
**Error Responses**:
* **400 Bad Request**: Invalid split name
* **404 Not Found**: Environment name not recognized
**Use case**: Get the list of tasks to iterate through during training or evaluation.
**Note**: Task structure is environment-specific. Math environments have questions and answers. Coding environments have problem descriptions and test cases.
***
### POST /\{env\_name}/num\_tasks
Get the number of tasks for a specific split.
**Request**:
```http theme={null}
POST /math/num_tasks
Content-Type: application/json
{
"split": "train"
}
```
**Request Schema**:
* `split`: Split name to query (string, required)
**Response** (200 OK):
```json theme={null}
{
"num_tasks": 500
}
```
**Error Responses**:
* **400 Bad Request**: Invalid split name
* **404 Not Found**: Environment name not recognized
**Use case**: Check the size of a task set without downloading all tasks. Useful for pagination with `/{env_name}/task_range`.
***
### POST /\{env\_name}/task
Get a single task by split and index.
**Request**:
```http theme={null}
POST /math/task
Content-Type: application/json
{
"split": "train",
"index": 0
}
```
**Request Schema**:
* `split`: Split name (string, required)
* `index`: Zero-based task index (integer, required)
**Response** (200 OK):
```json theme={null}
{
"task": {
"question": "What is 2+2?",
"answer": "4"
}
}
```
**Error Responses**:
* **400 Bad Request**: Invalid split name or invalid index (out of range)
* **404 Not Found**: Environment name not recognized
**Use case**: Fetch individual tasks for lazy loading or selective evaluation, without downloading the full task list.
***
### POST /\{env\_name}/task\_range
Get tasks for a range of indices.
**Request**:
```http theme={null}
POST /math/task_range
Content-Type: application/json
{
"split": "train",
"start": 0,
"stop": 10
}
```
**Request Schema**:
* `split`: Split name (string, required)
* `start`: Start index, inclusive (integer, optional, default: 0). Supports negative indices.
* `stop`: Stop index, exclusive (integer, optional, default: num\_tasks). Supports negative indices.
**Response** (200 OK):
```json theme={null}
{
"tasks": [
{"question": "What is 2+2?", "answer": "4"},
{"question": "If x + 5 = 12, what is x?", "answer": "7"}
]
}
```
**Error Responses**:
* **400 Bad Request**: Invalid split name or invalid index range
* **404 Not Found**: Environment name not recognized
**Use case**: Paginate through large task sets. Follows Python `range()`/slice conventions: start is inclusive, stop is exclusive. Negative indices are resolved relative to the total number of tasks.
## Session Management
These endpoints manage episode lifecycle.
### POST /create\_session
Generate a new session ID.
**Request**:
```http theme={null}
POST /create_session
```
**Response** (200 OK):
```json theme={null}
{
"sid": "abc-123-def-456"
}
```
**Response Schema**:
* `sid`: Session identifier (string, UUID format)
**Use case**: First step in episode lifecycle. Get a session ID to use in subsequent requests.
**Note**: This endpoint only generates an ID. The episode instance is created by `/create`.
***
### POST /create
Create an episode instance for a specific task.
**Request**:
```http theme={null}
POST /create
X-Session-ID: abc-123-def-456
Content-Type: application/json
{
"env_name": "math",
"task_spec": {
"question": "What is 2+2?",
"answer": "4"
},
"secrets": {
"api_key": "sk-..."
}
}
```
**Request Headers**:
* `X-Session-ID`: Session identifier from `/create_session` (required)
**Request Schema**:
* `env_name`: Environment to instantiate (string, optional). Defaults to the first registered environment if omitted. If the server hosts a single environment, requests to unrecognized paths are redirected to it automatically.
* `task_spec`: Task-specific data passed to the environment constructor (object). Provide **either** `task_spec` **or** both `split` and `index`, not both.
* `split`: Split name to resolve the task from (string). Must be provided together with `index`.
* `index`: Task index within the split (integer). Must be provided together with `split`.
* `secrets`: Secrets to pass to environment (object, optional, default: )
You must provide exactly one of: `task_spec` (pass the full task object directly) or `split` + `index` (let the server resolve the task via its `get_task()` method). Providing both or neither will return a 400 error.
**Response** (200 OK):
```json theme={null}
{
"sid": "abc-123-def-456"
}
```
**Error Responses**:
* **400 Bad Request**: Session already exists, or missing X-Session-ID header
* **404 Not Found**: Environment name not recognized
**Use case**: Start an episode by initializing the environment with a specific task. This calls the environment's `setup()` method.
**Important**: The environment initialization happens asynchronously. Subsequent requests wait for setup to complete.
***
### POST /ping
Keep the session alive to prevent timeout.
**Request**:
```http theme={null}
POST /ping
X-Session-ID: abc-123-def-456
```
**Request Headers**:
* `X-Session-ID`: Session identifier (required)
**Response** (200 OK):
```json theme={null}
{
"status": "ok"
}
```
**Error Responses**:
* **400 Bad Request**: Missing X-Session-ID header
* **404 Not Found**: Session not found
**Use case**: Sessions timeout after 15 minutes of inactivity. Call `/ping` periodically to keep the session alive. The reference SDK pings every 10 seconds; at minimum, ping well before the 15-minute timeout.
***
### POST /delete
Delete an episode and clean up resources.
**Request**:
```http theme={null}
POST /delete
X-Session-ID: abc-123-def-456
```
**Request Headers**:
* `X-Session-ID`: Session identifier (required)
**Response** (200 OK):
```json theme={null}
{
"sid": "abc-123-def-456"
}
```
**Error Responses**:
* **400 Bad Request**: Missing X-Session-ID header
* **404 Not Found**: Session not found
**Use case**: Clean up after episode completes. This calls the environment's `teardown()` method.
**Important**: Always call `/delete` when done with an episode to free server resources.
***
### POST /delete\_session
Optional cleanup endpoint for session ID.
**Request**:
```http theme={null}
POST /delete_session
X-Session-ID: abc-123-def-456
```
**Request Headers**:
* `X-Session-ID`: Session identifier (required)
**Response** (200 OK):
```json theme={null}
{
"sid": "abc-123-def-456"
}
```
**Use case**: Optional cleanup after `/delete`. In practice, `/delete` is sufficient.
## Episode Interaction
These endpoints interact with an active episode.
### GET /\{env\_name}/task\_tools
List all tools (shared + task-specific) available in the current session.
**Request**:
```http theme={null}
GET /math/task_tools
X-Session-ID: abc-123-def-456
```
**Request Headers**:
* `X-Session-ID`: Session identifier (required)
**Response** (200 OK):
```json theme={null}
{
"tools": [
{
"name": "submit",
"description": "Submit an answer",
"input_schema": { "type": "object", "properties": { "answer": { "type": "string" } }, "required": ["answer"] }
},
{
"name": "get_hint",
"description": "Get a hint for this specific task",
"input_schema": null
}
]
}
```
**Response Schema**:
* `tools`: Array of ToolSpec objects (shared tools + task-specific tools combined)
**Error Responses**:
* **400 Bad Request**: Missing X-Session-ID header
* **404 Not Found**: Session not found
* **410 Gone**: Session was deleted
**Use case**: Get the full set of tools available for the current task, including task-specific tools that are not returned by the sessionless `/{env_name}/tools` endpoint. Tools marked with `@tool(shared=False)` or returned by `list_task_tools()` only appear here.
***
### GET /\{env\_name}/prompt
Get the initial prompt for the current task.
**Request**:
```http theme={null}
GET /math/prompt
X-Session-ID: abc-123-def-456
```
**Request Headers**:
* `X-Session-ID`: Session identifier (required)
**Response** (200 OK):
```json theme={null}
[
{
"text": "What is 2+2?",
"detail": null,
"type": "text"
}
]
```
**Response Schema**:
* Array of Block objects
* TextBlock: `{"text": "...", "detail": null, "type": "text"}`
* ImageBlock: `{"data": "base64...", "mimeType": "image/png", "detail": null, "type": "image"}`
**Error Responses**:
* **400 Bad Request**: Missing X-Session-ID header
* **404 Not Found**: Session not found (no active environment for this session ID)
* **410 Gone**: Session was deleted
**Use case**: Get the initial state/instructions for the episode. Call this after `/create` and before calling tools.
**Note**: Prompts can be multi-modal (text + images). The environment is resolved by the session, not by the `{env_name}` path segment. The path parameter is not validated against the session's actual environment.
***
### POST /\{env\_name}/call
Call a tool in the environment.
**Request**:
```http theme={null}
POST /math/call
X-Session-ID: abc-123-def-456
Accept: text/event-stream
Content-Type: application/json
{
"name": "submit",
"input": {
"answer": "4"
},
"task_id": "optional-trace-id"
}
```
**Request Headers**:
* `X-Session-ID`: Session identifier (required)
* `Accept`: Should be `text/event-stream` (recommended). The server returns SSE regardless, but setting this header is good practice for client clarity.
**Request Schema**:
* `name`: Tool name to call (string, required)
* `input`: Tool-specific parameters (object, required)
* `task_id`: Identifier for reconnection and tracing (string, optional). If provided on a reconnect, the server returns the result of the original task instead of starting a new one. Completed results linger for 60 seconds, enabling recovery from dropped connections.
**Response** (200 OK - Server-Sent Events):
Small response (fits in one event):
```
event: task_id
data: 877bb56c594e4a0f921ad55c439a3762
event: end
data: {"ok": true, "output": {"blocks": [{"text": "Correct!", "detail": null, "type": "text"}], "metadata": null, "reward": 1.0, "finished": true}}
```
Large response (>4 KB, chunked):
```
event: task_id
data: 877bb56c594e4a0f921ad55c439a3762
event: chunk
data: {"ok": true, "output": {"blocks": [{"text": "Very long out
event: end
data: put...", "detail": null, "type": "text"}], "metadata": null, "reward": 0.5, "finished": false}}
```
**SSE Event Types**:
| Event | Data | Description |
| --------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `task_id` | Task identifier string | Always sent first. Use this ID to reconnect if the connection drops. |
| `chunk` | Partial JSON string | Intermediate chunk for responses exceeding 4 KB. Concatenate all `chunk` data with the final `end` data to reconstruct the full JSON. |
| `end` | JSON string (or final chunk) | Final event. Contains the complete response JSON, or the last chunk if the response was split. |
| `error` | Error message string | Sent if the `task_id` is unknown (on reconnect) or the task raised an exception. No `end` event follows. |
**Response Schema** (reconstructed from SSE data):
On success:
```json theme={null}
{
"ok": true,
"output": {
"blocks": [
{"text": "Tool output", "detail": null, "type": "text"}
],
"metadata": null,
"reward": 1.0,
"finished": true
}
}
```
On error:
```json theme={null}
{
"ok": false,
"error": "Error message"
}
```
**Error Responses**:
* **400 Bad Request**: Missing X-Session-ID, invalid tool name, or invalid input
* **404 Not Found**: Session not found, environment mismatch, or tool not found
* **410 Gone**: Session was deleted
**Use case**: Execute an action in the environment. This is the core interaction loop.
**Important**:
* Responses are delivered via Server-Sent Events (see Tool Execution with SSE section above)
* The `finished` field indicates if the episode is complete
* The `reward` field provides RL feedback signal
## Complete Episode Flow
Here's a complete example showing all endpoints in sequence:
```bash theme={null}
# 1. Health check
curl http://localhost:8080/health
# → {"status": "ok"}
# 2. List environments
curl http://localhost:8080/list_environments
# → ["math"]
# 3. List tools
curl http://localhost:8080/math/tools
# → {"tools": [{"name": "submit", ...}]}
# 4. List splits
curl http://localhost:8080/math/splits
# → [{"name": "train", "type": "train"}, ...]
# 5. List tasks
curl -X POST http://localhost:8080/math/tasks \
-H "Content-Type: application/json" \
-d '{"split": "train"}'
# → {"tasks": [...], "env_name": "math"}
# 6. Create session ID
curl -X POST http://localhost:8080/create_session
# → {"sid": "abc-123"}
# 7. Create episode (option A: pass task_spec directly)
curl -X POST http://localhost:8080/create \
-H "X-Session-ID: abc-123" \
-H "Content-Type: application/json" \
-d '{"env_name": "math", "task_spec": {"question": "What is 2+2?"}, "secrets": {}}'
# → {"sid": "abc-123"}
# 7b. Or create episode (option B: resolve by split + index)
# curl -X POST http://localhost:8080/create \
# -H "X-Session-ID: abc-123" \
# -H "Content-Type: application/json" \
# -d '{"env_name": "math", "split": "train", "index": 0, "secrets": {}}'
# → {"sid": "abc-123"}
# 8. Get prompt
curl http://localhost:8080/math/prompt \
-H "X-Session-ID: abc-123"
# → [{"text": "What is 2+2?", "detail": null, "type": "text"}]
# 9. Call tool (SSE response)
curl -X POST http://localhost:8080/math/call \
-H "X-Session-ID: abc-123" \
-H "Accept: text/event-stream" \
-H "Content-Type: application/json" \
-d '{"name": "submit", "input": {"answer": "4"}}'
# → SSE stream with result
# 10. Delete episode
curl -X POST http://localhost:8080/delete \
-H "X-Session-ID: abc-123"
# → {"sid": "abc-123"}
```
## Error Handling
### Standard HTTP Status Codes
* **200 OK**: Request succeeded
* **400 Bad Request**: Invalid input (missing header, invalid JSON, etc.)
* **404 Not Found**: Resource not found (environment, session, tool)
* **410 Gone**: Session was deleted
* **500 Internal Server Error**: Server error during processing
### Error Response Format
For HTTP errors (non-SSE):
```json theme={null}
{
"detail": "Error message explaining what went wrong"
}
```
For tool execution errors (SSE):
```json theme={null}
{
"ok": false,
"error": "Tool execution failed: reason"
}
```
## Rate Limiting and Timeouts
### Session Timeout
Sessions automatically expire after **15 minutes** of inactivity:
* "Activity" means any request with that session's X-Session-ID
* Use `/ping` to keep sessions alive
* Timeout is reset after each request
### Request Timeout
Individual requests may have server-specific timeouts. All tool calls use SSE, which keeps connections alive with periodic pings during long-running execution.
## Next Steps
Detailed schemas for all request/response objects
Build an ORS client using these endpoints
Build an ORS server that implements these endpoints
***
**Reference Implementation**: The [ORS Python SDK](https://github.com/openrewardstandard/python-sdk) implements this complete API.
# Protocol Overview
Source: https://openrewardstandard.io/specification/overview
Understanding the ORS HTTP protocol
The Open Reward Standard is an **HTTP-based protocol** for connecting language model agents to reinforcement learning environments. It uses standard REST endpoints for control operations and Server-Sent Events (SSE) for delivering tool outputs.
## Design Principles
### 1. Language-Agnostic
ORS uses HTTP, making it implementable in any programming language:
* Python, TypeScript, Go, Rust, Java, etc.
* Any web framework or HTTP library
* Standard REST patterns
### 2. Episode-Centric
The protocol is organized around RL episodes (sessions):
* One session = one episode
* Episode continues until `finished: true`
* Stateful interaction across multiple tool calls
### 3. Tool-Based Interaction
All agent actions are tool calls:
* Discovered via `GET /{env_name}/tools`
* Executed via `POST /{env_name}/call`
* Return structured outputs with rewards
## Protocol Architecture
![ORS Protocol Architecture — Agent communicating with ORS Server and Environment Logic]()
### Key Components
**Agent Side**:
* Makes HTTP requests
* Parses SSE responses
* Maintains session ID
**ORS Server**:
* Implements HTTP endpoints
* Manages episode state
* Executes tools and returns rewards
## Episode Lifecycle
An episode (session) follows this lifecycle:
```
1. Create Session
↓
2. Create Episode Instance
↓
3. Get Prompt (initial state)
↓
4. Call Tools (actions)
├─ Receive reward
├─ Check finished flag
└─ If not finished, repeat step 4
↓
5. Delete Episode (cleanup)
```
### Example Flow
```http theme={null}
# 1. Create session ID
POST /create_session
→ {"sid": "abc-123"}
# 2. Create episode instance with task
POST /create
Headers: X-Session-ID: abc-123
Body: {
"env_name": "math",
"task_spec": {"question": "What is 2+2?", "answer": "4"},
"secrets": {}
}
→ {"sid": "abc-123"}
# 3. Get initial prompt
GET /math/prompt
Headers: X-Session-ID: abc-123
→ [{"text": "What is 2+2?", "detail": null, "type": "text"}]
# 4. Call tool
POST /math/call
Headers: X-Session-ID: abc-123
Accept: text/event-stream
Body: {"name": "submit", "input": {"answer": "4"}}
→ SSE stream (see below for format)
# 5. Delete episode
POST /delete
Headers: X-Session-ID: abc-123
→ {"sid": "abc-123"}
```
## Endpoint Categories
ORS endpoints fall into four categories:
### 1. Discovery Endpoints
Get information about the environment:
```http theme={null}
GET /list_environments # List available environments
GET /{env_name}/tools # List available tools
GET /{env_name}/splits # List available splits
POST /{env_name}/tasks # List tasks for a split
```
These are **stateless** - no session required.
### 2. Session Management
Create and manage episodes:
```http theme={null}
POST /create_session # Generate session ID
POST /create # Create episode instance
POST /delete # Delete episode
POST /delete_session # (Cleanup - optional)
POST /ping # Keep session alive
```
These require the **X-Session-ID header** (except create\_session).
### 3. Episode Interaction
Interact with the active episode:
```http theme={null}
GET /{env_name}/prompt # Get initial prompt
POST /{env_name}/call # Call a tool
```
These require **X-Session-ID** and an active episode.
### 4. Health
```http theme={null}
GET /health # Server health check
```
## Session Management
### X-Session-ID Header
Episodes are identified by a session ID passed in the `X-Session-ID` header:
```http theme={null}
POST /create
X-Session-ID: abc-123
```
**Flow**:
1. Call `POST /create_session` to get a session ID
2. Use that ID in all subsequent requests
3. Server maintains episode state for that ID
4. Call `POST /delete` to clean up
### Session Timeout
Sessions automatically expire after **15 minutes of inactivity**.
To prevent timeout:
```http theme={null}
POST /ping
X-Session-ID: abc-123
```
Call `/ping` periodically to keep the session alive. The reference SDK pings every 10 seconds; at minimum, ping well before the 15-minute timeout.
## Tool Execution with SSE
Tool calls return results via **Server-Sent Events**:
```http theme={null}
POST /{env_name}/call
Headers:
X-Session-ID: abc-123
Accept: text/event-stream
Body: {
"name": "bash",
"input": {"command": "ls -la"}
}
```
Response (SSE stream):
```
event: task_id
data: 877bb56c594e4a0f921ad55c439a3762
event: end
data: {"ok": true, "output": {"blocks": [{"text": "Output text", "detail": null, "type": "text"}], "metadata": null, "reward": 0.0, "finished": false}}
```
For large responses (>4KB), the result is split across `chunk` events before the final `end` event. See [Server-Sent Events](/specification/sse) for the full event type reference.
### Why SSE?
Server-Sent Events are used because:
* **Long-running tool calls**: Keeps connections alive while tools execute (bash commands, LLM calls, etc.)
* **Chunking large responses**: Results over 4KB are split into chunks for reliable delivery
* **Reconnection**: Clients can reconnect with a task ID and retrieve completed results
* **Standard protocol**: Built into browsers and HTTP libraries
## Error Handling
### HTTP Status Codes
Standard HTTP status codes:
* **200 OK**: Successful request
* **400 Bad Request**: Invalid input
* **404 Not Found**: Session/environment/tool not found
* **500 Internal Server Error**: Server error
### SSE Errors
Errors can also arrive as SSE events during tool execution streaming:
```
event: task_id
data: task-xyz-790
event: error
data: Session not found
```
These represent server-level failures (session not found, tool not recognized, internal error). They are distinct from tool logic errors, which return `{"ok": false, "error": "..."}` in an `end` event. See [Server-Sent Events](/specification/sse#error) for details.
### Tool Errors
Tool execution errors are returned in the ToolOutput:
```json theme={null}
{
"ok": false,
"error": "Tool 'submit' failed: Invalid answer format"
}
```
Successful tool calls:
```json theme={null}
{
"ok": true,
"output": {
"blocks": [{"text": "Correct!", "detail": null, "type": "text"}],
"metadata": null,
"reward": 1.0,
"finished": true
}
}
```
## Stateful Sessions
Sessions maintain state across tool calls:
```python theme={null}
# Episode state persists between calls
session.call_tool("bash", {"command": "echo 'hello' > file.txt"})
session.call_tool("bash", {"command": "cat file.txt"})
# → "hello"
```
**What's maintained**:
* Environment-specific state (variables, files, etc.)
* Task context
* Episode progress
**What ends an episode**:
* Client calls `POST /delete` (explicit cleanup — calls environment's `teardown()`)
* Session times out after 15 minutes of inactivity (automatic reaper)
The `finished: true` signal indicates the episode is **logically complete** from an RL perspective (e.g., the agent submitted a correct answer). It does **not** trigger automatic server-side teardown. The client should call `POST /delete` to free resources after receiving `finished: true`.
**What's NOT maintained**:
* State across different sessions
## Security Considerations
### Secrets
Tasks can receive secrets via the `secrets` field:
```http theme={null}
POST /create
Body: {
"env_name": "web_env",
"task_spec": {...},
"secrets": {
"api_key": "sk-..."
}
}
```
Secrets are passed to the environment when a session is created. The environment can use those secrets for internal
logic of the environment. For example, a model provider API key an be used to initialise an LLM client within the environment for an LLM grader.
### Isolation
ORS sessions are isolated at the protocol level. Each session ID maps to its own environment instance with independent state. But deeper isolation depends on the server implementation:
* **Protocol-level**: Each session has its own environment instance and state
* **Implementation-level**: Filesystem, network, and process isolation require additional infrastructure (e.g., containers, sandboxes)
## Implementation Approaches
### Option 1: Use [ORS Python SDK](https://github.com/openrewardstandard/python-sdk)
The Python SDK implements the full ORS protocol:
```python theme={null}
from ors import Environment, Server, tool
class MyEnvironment(Environment):
@classmethod
def list_splits(cls):
return ["train", "test"]
# ... implement other methods
server = Server([MyEnvironment])
server.run(port=8080)
```
The SDK handles:
* HTTP endpoint routing
* Session management
* SSE response delivery
* Error handling
### Option 2: Implement from Scratch
Implement the protocol in any language:
1. Create HTTP server
2. Implement required endpoints
3. Manage session state
4. Return tool outputs via SSE
See [Implementation Guide](/guides/implementing-server) for details.
## Next Steps
Complete endpoint documentation
Request and response schemas
Deep dive on episodes and sessions
***
**Key Takeaway**: ORS is a straightforward HTTP protocol with RESTful endpoints for discovery and management, plus SSE for delivering tool execution results. It's designed to be simple to implement in any language.
# Sessions & Episodes
Source: https://openrewardstandard.io/specification/sessions
Understanding the episodic lifecycle in ORS
An episode is a complete run of experience from start to termination. In ORS, a session with a server is equivalent to an episode if the agent terminates the session after receiving a finished signal.
## Sessions as Episodes
In ORS an episode
* Starts with a specific task
* Continues through multiple tool calls
* Ends when `finished: true` is received from a `ToolOutput`.
### RL Episode Terminology
| RL Term | ORS Term | Description |
| ------------------ | ------------------------------ | ------------------------------------------- |
| **Episode** | Session | One complete trajectory |
| **State** | Environment instance | Full internal state on the server |
| **Observation** | Blocks (prompt + tool outputs) | Partial view of state returned to the agent |
| **Action** | Tool call | Agent action |
| **Reward** | `ToolOutput.reward` | Feedback signal |
| **Terminal state** | `finished: true` | Episode complete |
## Episode Lifecycle
### Complete Flow
![Episode Lifecycle - 5 steps from session creation to cleanup]()
### States in Detail
#### 1. Session ID Generation
```http theme={null}
POST /create_session
```
**Purpose**: Generate a unique identifier for this episode.
**Response**:
```json theme={null}
{"sid": "abc-123-def-456"}
```
**Note**: This just creates an ID. No environment is instantiated yet.
***
#### 2. Episode Initialization
```http theme={null}
POST /create
X-Session-ID: abc-123-def-456
Content-Type: application/json
{
"env_name": "math",
"task_spec": {"question": "What is 2+2?"},
"secrets": {"api_key": "sk-..."}
}
```
All body fields are optional. `env_name` defaults to the first registered environment. Either `task_spec` or both `split`+`index` must be provided (see [CreateSession](/specification/data-types#createsession)).
**What happens**:
1. Server resolves `env_name` (or defaults to first environment) and `task_spec` (or loads from `split`/`index`)
2. Instantiates the environment class with `task_spec` and `secrets`
3. Calls `environment.setup()` (async)
4. Marks session as "ready" when setup completes
**Blocking**: Subsequent requests wait for setup to complete before proceeding.
***
#### 3. Initial Observation
```http theme={null}
GET /math/prompt
X-Session-ID: abc-123-def-456
```
**Purpose**: Get the initial observation (o₀) for the episode.
**Response**:
```json theme={null}
[
{"text": "What is 2+2?", "detail": null, "type": "text"}
]
```
**RL Interpretation**: This is the initial observation that the agent uses to select its first action.
***
#### 4. Action-Observation Loop
```http theme={null}
POST /math/call
X-Session-ID: abc-123-def-456
{"name": "submit", "input": {"answer": "4"}}
```
**Response (SSE)**:
```json theme={null}
{
"ok": true,
"output": {
"blocks": [{"text": "Correct!", "detail": null, "type": "text"}],
"metadata": null,
"reward": 1.0,
"finished": true
}
}
```
**What happens**:
1. Agent takes action (calls tool)
2. Environment executes action
3. Environment returns next state (blocks), reward, and termination flag
4. If `finished: false`, repeat from step 1
5. If `finished: true`, episode is complete
**RL Interpretation**: This is the core RL loop:
* Action: Tool call
* Observation: Blocks
* Reward: Reward signal
* Terminal: Finished flag
***
#### 5. Episode Termination
```http theme={null}
POST /delete
X-Session-ID: abc-123-def-456
```
**Purpose**: Clean up episode resources.
**What happens**:
1. Calls `environment.teardown()`
2. Removes session from active sessions
3. Frees memory and resources
**Important**: Always call `/delete` when done, even if episode finished naturally.
## Episode Termination
### The `finished` Signal
The `finished` field in `ToolOutput` is **critical**:
```typescript theme={null}
interface ToolOutput {
blocks: Blocks
metadata?: JSONObject
reward?: number
finished?: boolean // ← Episode termination signal (default: false)
}
```
**When `finished: true`**:
* Episode is complete
* Agent should stop calling tools
* Agent should call `/delete` to cleanup
* Task succeeded or failed (check reward or blocks for details)
**When `finished: false`**:
* Episode continues
* Agent should take another action
* State may have changed (reflected in blocks)
### Termination Patterns
#### Pattern 1: Immediate Termination
Task completes in one step:
```python theme={null}
# Single action episode
result = session.call_tool("submit", {"answer": "42"})
assert result.finished == True
```
#### Pattern 2: Multi-Step Termination
Task requires multiple actions:
```python theme={null}
# Multi-step episode
result1 = session.call_tool("bash", {"command": "cat file.txt"})
assert result1.finished == False # Continue
result2 = session.call_tool("submit", {"answer": "Paris"})
assert result2.finished == True # Complete
```
#### Pattern 3: Failure Termination
Task fails (but episode still terminates):
```python theme={null}
result = session.call_tool("submit", {"answer": 999})
assert result.finished == True
assert result.reward == 0.0 # Failed
```
## State Management
### What's Preserved in a Session?
**Environment state**:
* Instance variables in environment class
* Files created during episode (if environment has filesystem or persistent sandbox)
* Any side effects from tool executions
**Example**:
```python theme={null}
# State persists across tool calls
session.call_tool("bash", {"command": "export VAR=hello"})
result = session.call_tool("bash", {"command": "echo $VAR"})
# → "hello" (state preserved)
```
### What's NOT Preserved?
**Across episodes**:
* Each session is independent at the protocol level
* Session 1 and Session 2 have separate environment instances
* No shared instance state between sessions (though implementations may share class-level or cached state)
**After timeout**:
* 15 minutes of inactivity → session deleted
* State is lost
* Must create new session
**After `finished: true`**:
* Episode data is final
* Further tool calls should not be made
* Call `/delete` for cleanup
## Session Timeout
Sessions automatically expire after **15 minutes** of inactivity.
### Inactivity Definition
"Inactivity" means no requests with that session's `X-Session-ID`:
* `/ping` resets timer
* `/{env_name}/call` resets timer
* `/{env_name}/prompt` resets timer
* Any request with `X-Session-ID` resets the timer (except `/delete`, which removes the session)
### Keeping Sessions Alive
For long-running episodes, periodically call `/ping`:
```python theme={null}
import threading
import time
def keep_alive(session_id):
while True:
requests.post(
"http://server/ping",
headers={"X-Session-ID": session_id}
)
time.sleep(300) # Every 5 minutes
# Start background thread
threading.Thread(target=keep_alive, args=(session_id,), daemon=True).start()
```
### Timeout Cleanup
When a session times out:
1. Server calls `environment.teardown()`
2. Session removed from active sessions
3. Subsequent requests with that session ID → 404 Not Found
## Session Best Practices
### 1. Always Delete Sessions
```python theme={null}
# Good - cleanup
session_id = create_session()
try:
# ... episode logic
pass
finally:
delete_session(session_id)
```
```python theme={null}
# Bad - resource leak
session_id = create_session()
# ... episode logic
# Forgot to delete!
```
### 2. Check `finished` Flag
```python theme={null}
# Good - respect termination
result = session.call_tool("submit", {"answer": "42"})
if result.finished:
delete_session(session_id)
else:
# Continue episode
pass
```
```python theme={null}
# Bad - ignore termination
result = session.call_tool("submit", {"answer": "42"})
# Continue calling tools even if finished=True
```
### 3. Handle Errors Gracefully
```python theme={null}
# Good - cleanup on error
try:
result = session.call_tool("bash", {"command": "rm -rf /"})
except Exception as e:
delete_session(session_id)
raise
```
### 4. Use Context Managers
```python theme={null}
# Best - automatic cleanup
with session_manager.session(task=task) as session:
result = session.call_tool("submit", {"answer": "42"})
# Automatically deleted when exiting context
```
## Debugging Sessions
### Common Issues
**Issue**: "404 Session not found"
* **Cause**: Session timed out or was deleted
* **Fix**: Check that episode completes within 15 minutes or use `/ping`
**Issue**: "Session already exists"
* **Cause**: Trying to create episode with already-used session ID
* **Fix**: Generate new session ID with `/create_session`
**Issue**: "Session deleted" (410)
* **Cause**: Calling tool after `/delete` was called
* **Fix**: Don't reuse session IDs after deletion
## Next Steps
Understand reward signals in episodes
Build a client that manages sessions
***
**Key Takeaway**: Sessions are RL episodes. They start with a task, continue until `finished: true`, and should always be cleaned up with `/delete`.