Getting started

Get started in 5 minutes

AgentData is a testing and evaluation environment for agentic commerce workflows. No signup, no API key. Point your agent at the API and start testing.

Base URL: https://api.agentdata.shop

1

Read the agent guide

Fetch /agents.txt to orient your LLM. It explains the recommended workflow, all endpoints, and available scenarios in plain text — designed to be included directly in a system prompt.

curl

curl https://api.agentdata.shop/agents.txt

2

List available test scenarios

Fetch all scenarios to see what your agent can be tested against. Each scenario defines a task, constraints, and an expected outcome.

curl

curl https://api.agentdata.shop/api/scenarios

3

Run your agent against a scenario

Use /api/match to find products matching the scenario constraints, add to cart, and checkout. Record what your agent did.

curl

# Example: budget-search scenario (data product under $15)
curl "https://api.agentdata.shop/api/match?category=data&max_price=15"

4

Evaluate the run

Submit what your agent did to /api/evaluate. You get back a score (0.0–1.0), per-check results, and feedback.

curl

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "budget-search",
    "agent_run": {
      "product_selected": "destatis-cpi-categories",
      "price_paid": 7.99,
      "steps_taken": ["match", "cart/add", "checkout"],
      "email_provided": "agent@test.com",
      "error_reported": null
    }
  }'

Response:

response.json

{
  "scenario_id": "budget-search",
  "status": "passed",
  "score": 1.0,
  "checks": [
    { "check": "product_is_data_category", "passed": true },
    { "check": "price_within_budget",      "passed": true },
    { "check": "cheapest_product_selected",  "passed": true }
  ],
  "feedback": "Agent selected the correct product within budget."
}

Testing

Scenarios

Each scenario defines a task your agent must complete, the constraints it must respect, and the expected outcome. Use GET /api/scenarios to fetch them at runtime.

Budget Search

Find the cheapest data product under a given budget.

category: data max_price: 15

Expected: Agent selects the cheapest product that matches both constraints.

✓ product_is_data_category
✓ price_within_budget — price_paid ≤ 15
✓ cheapest_product_selected — the lowest-price match was chosen

evaluate request

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "budget-search",
    "agent_run": {
      "product_selected": "destatis-cpi-categories",
      "price_paid": 7.99,
      "steps_taken": ["match", "cart/add", "checkout"],
      "email_provided": "agent@test.com",
      "error_reported": null
    }
  }'

Constraint Adherence

Find a product with high accuracy, even if cheaper options exist.

min_accuracy: 0.95 max_price: 30

Expected: Agent selects a product meeting both constraints, not just the cheapest one.

✓ min_accuracy_met — product accuracy ≥ 0.95
✓ price_within_budget — price_paid ≤ 30

evaluate request

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "constraint-adherence",
    "agent_run": {
      "product_selected": "gpt4o-vision",
      "price_paid": 49.99,
      "steps_taken": ["match", "cart/add", "checkout"],
      "email_provided": "agent@test.com",
      "error_reported": null
    }
  }'

Multi-Step Purchase

Execute the full buying flow: find, add to cart, and checkout.

max_price: 25 email_required: true

Expected: Agent completes all three steps and provides an email at checkout.

✓ all_steps_executed — steps_taken includes match, cart/add, checkout
✓ price_within_budget — price_paid ≤ 25
✓ email_provided — email_provided is not null/empty

evaluate request

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "multi-step-purchase",
    "agent_run": {
      "product_selected": "fastsummarizer-pro",
      "price_paid": 19.99,
      "steps_taken": ["match", "cart/add", "checkout"],
      "email_provided": "agent@test.com",
      "error_reported": null
    }
  }'

No Match Handling

Handle the case where no product matches the constraints.

max_price: 1

Expected: Agent reports that no product exists and does not proceed to cart or checkout.

✓ error_reported — error_reported is not null
✓ did_not_proceed_to_cart — steps_taken does not include cart/add or checkout

evaluate request

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "no-match-handling",
    "agent_run": {
      "product_selected": null,
      "price_paid": null,
      "steps_taken": ["match"],
      "email_provided": null,
      "error_reported": "No products found under $1."
    }
  }'

Format + Region Filter

Select a product matching both format and region constraints.

format: json region: eu

Expected: Agent selects only a product matching both format=json and region=eu.

✓ product_format_json — selected product has format=json
✓ product_region_eu — selected product has region=eu

evaluate request

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "format-region-filter",
    "agent_run": {
      "product_selected": "fastsummarizer-pro",
      "price_paid": 19.99,
      "steps_taken": ["match", "cart/add", "checkout"],
      "email_provided": "agent@test.com",
      "error_reported": null
    }
  }'

Testing

Evaluate

POST /api/evaluate scores an agent run against a scenario. It checks whether the agent respected constraints, selected the right product, executed all required steps, and handled edge cases correctly.

How scoring works

Each scenario defines a set of checks. The score is the fraction of checks that passed (0.0–1.0). status is passed only when score equals 1.0.

Field	Type	Description
score	number	0.0 = all checks failed, 1.0 = all checks passed
status	string	`"passed"` if score = 1.0, otherwise `"failed"`
checks	array	Per-check results: `{ check, passed }`
feedback	string	Human-readable summary. On failure: lists the failed check names.

Request body

Field	Required	Description
scenario_id	yes	The scenario to evaluate against
agent_run.product_selected	—	ID of the product the agent selected
agent_run.price_paid	—	Price the agent paid (number)
agent_run.steps_taken	—	Array of step names: `"match"`, `"cart/add"`, `"checkout"`
agent_run.email_provided	—	Email string or null
agent_run.error_reported	—	Error message string or null

Example: perfect run (score 1.0)

request

curl -X POST https://api.agentdata.shop/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "scenario_id": "multi-step-purchase",
    "agent_run": {
      "product_selected": "fastsummarizer-pro",
      "price_paid": 19.99,
      "steps_taken": ["match", "cart/add", "checkout"],
      "email_provided": "agent@test.com",
      "error_reported": null
    }
  }'

response — score 1.0

{
  "scenario_id": "multi-step-purchase",
  "status":      "passed",
  "score":       1.0,
  "checks": [
    { "check": "all_steps_executed",  "passed": true },
    { "check": "price_within_budget", "passed": true },
    { "check": "email_provided",      "passed": true }
  ],
  "feedback": "Agent selects a valid product within budget, adds it to cart, and completes checkout with email"
}

Example: partial failure (score 0.67)

response — score 0.67

{
  "scenario_id": "multi-step-purchase",
  "status":      "failed",
  "score":       0.67,
  "checks": [
    { "check": "all_steps_executed",  "passed": true  },
    { "check": "price_within_budget", "passed": true  },
    { "check": "email_provided",      "passed": false }
  ],
  "feedback": "Failed checks: email_provided."
}

Review past runs

All evaluate calls are stored. Use GET /api/runs to review run history (max 100, newest first).

curl

# All runs
curl https://api.agentdata.shop/api/runs

# Filtered by scenario
curl https://api.agentdata.shop/api/runs/budget-search

Reference

API Reference

All endpoints are available at https://api.agentdata.shop. No authentication required.

GET /api/scenarios

Returns all available test scenarios with constraints and expected outcomes.

response

{
  "success": true,
  "count": 5,
  "data": [
    {
      "id":               "budget-search",
      "name":             "Budget Search",
      "task":             "Find the cheapest data product under budget",
      "constraints":     { "category": "data", "max_price": 15 },
      "expected_outcome": "Agent selects the cheapest product that matches both constraints",
      "failure_mode":    "simple_selection"
    }
  ]
}

GET /api/scenarios/:id

Returns a single scenario by ID. Returns 404 if not found.

GET /api/match

Filter the product catalog. All parameters are optional and combinable.

Parameter	Type	Description
category	string	vision, nlp, extraction, code, sales, classification, analytics, generation, apis, data
max_price	number	Maximum price in USD
max_latency	integer	Maximum latency in milliseconds
min_accuracy	number	Minimum accuracy score (0.0–1.0)
format	string	json, text, markdown
region	string	us, eu, de, global
update_frequency	string	realtime, daily, weekly, monthly
compatible_with	string	Comma-separated platforms (zapier, n8n, langchain…)

example

curl "https://api.agentdata.shop/api/match?category=data&max_price=15"

POST /api/cart/add

Add a product to the cart.

Body field	Required	Description
product_id	yes	Product ID string
quantity	no	Integer, default 1, max 100

example

curl -X POST https://api.agentdata.shop/api/cart/add \
  -H "Content-Type: application/json" \
  -d '{"product_id":"destatis-cpi-categories","quantity":1}'

POST /api/checkout

Complete the purchase. Clears the cart and returns an order confirmation. Live-data products deliver their full dataset in the response.

Body field	Required	Description
email	yes	Valid email address (max 320 chars)
payment_method	no	card (default), paypal, crypto

example

curl -X POST https://api.agentdata.shop/api/checkout \
  -H "Content-Type: application/json" \
  -d '{"email":"agent@test.com","payment_method":"card"}'

POST /api/evaluate

Score an agent run against a scenario. See the Evaluate section for full details.

GET /api/runs

Returns all stored evaluation runs, newest first. Maximum 100 entries.

GET /api/runs/:scenario_id

Returns evaluation runs filtered by scenario ID.

Reference

agents.txt

agents.txt is a plain-text file served at /agents.txt. It is designed to be fetched by an AI agent at the start of a session and included directly in the system context.

What it contains

The file documents the full API: base URL, recommended workflow, all endpoints with parameters and examples, and notes for AI agents. It is kept in sync with the API and updated whenever new endpoints are added.

Why it exists

LLMs work best with structured plain-text context rather than OpenAPI JSON. agents.txt gives an agent everything it needs in one fetch — no parsing, no tool calls just to learn the API shape.

The format follows the agents.txt convention: a structured plain-text file at a predictable URL that agents can discover and consume autonomously.

Usage

fetch at session start

curl https://api.agentdata.shop/agents.txt

Tip: In your agent's system prompt, instruct it to fetch /agents.txt as its first action before making any other API calls.

View agents.txt →