Simulations API

List suites

GET/v1/simulations/suites

Query parameters

agent_idstring

Returns

200 with the fields below.

enabledbooleanrequired

False if simulations are not turned on for the platform.

suitesarray<object>required

Suites, newest first. At most 200.

9 properties

idstringrequired

The suite's id.

agent_idstringrequired

The agent the suite tests.

namestringrequired

Display name.

publish_gatestringrequired

Whether a failing run blocks publishing the agent: `off`, `warn` or `block`.

case_countintegerrequired

Number of cases in the suite.

descriptionstring

What the suite covers.

latest_runobject

The most recent run. Null if the suite has never been run.

18 properties

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

curl "https://api.dialgem.com/v1/simulations/suites" \
  -H "X-API-Key: $DIALGEM_API_KEY"

Response 200

{
  "enabled": false,
  "suites": [
    {
      "id": "string",
      "agent_id": "agent_123",
      "name": "Example",
      "description": "string",
      "publish_gate": "string",
      "case_count": 0,
      "latest_run": {
        "id": {},
        "suite_id": {},
        "agent_id": {},
        "agent_version": {},
        "status": {},
        "trigger": {},
        "fresh": {},
        "total": {},
        "passed": {},
        "failed": {},
        "errored": {},
        "skipped": {}
      },
      "created_at": "string",
      "updated_at": "string"
    }
  ]
}

Create suite

POST/v1/simulations/suites

Request body application/json

agent_idstringrequired

namestringrequired

descriptionstring

publish_gatestring

Default "off"

Returns

201 with the fields below.

idstringrequired

The suite's id.

agent_idstringrequired

The agent the suite tests.

namestringrequired

Display name.

publish_gatestringrequired

Whether a failing run blocks publishing the agent: `off`, `warn` or `block`.

case_countintegerrequired

Number of cases in the suite.

descriptionstring

What the suite covers.

latest_runobject

The most recent run. Null if the suite has never been run.

18 properties

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

curl -X POST "https://api.dialgem.com/v1/simulations/suites" \
  -H "X-API-Key: $DIALGEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agent_123",
    "name": "Example"
  }'

Response 201

{
  "id": "string",
  "agent_id": "agent_123",
  "name": "Example",
  "description": "string",
  "publish_gate": "string",
  "case_count": 0,
  "latest_run": {
    "id": "string",
    "suite_id": "suite_123",
    "agent_id": "agent_123",
    "agent_version": 0,
    "status": "string",
    "trigger": "string",
    "fresh": false,
    "total": 0,
    "passed": 0,
    "failed": 0,
    "errored": 0,
    "skipped": 0
  },
  "created_at": "string",
  "updated_at": "string"
}

Get suite

GET/v1/simulations/suites/{suite_id}

Path parameters

suite_idstringrequired

Returns

200 with the fields below.

idstringrequired

The suite's id.

agent_idstringrequired

The agent the suite tests.

namestringrequired

Display name.

publish_gatestringrequired

Whether a failing run blocks publishing the agent: `off`, `warn` or `block`.

case_countintegerrequired

Number of cases in the suite.

casesarray<object>required

The suite's cases, in order.

10 properties

idstringrequired

The case's id.

suite_idstringrequired

The suite the case belongs to.

namestringrequired

Display name.

positionintegerrequired

Order of the case within its suite, starting at 0.

callerobjectrequired

How the simulated caller behaves: either a fixed script of lines or a persona with a goal.

dynamic_variablesobjectrequired

Dynamic variables given to the agent for this case.

tool_mocksobjectrequired

What each tool answers during the case, keyed by tool name. Tools are never really called.

checksarray<object>required

What must be true of the conversation for the case to pass.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

descriptionstring

What the suite covers.

latest_runobject

The most recent run. Null if the suite has never been run.

18 properties

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

curl "https://api.dialgem.com/v1/simulations/suites/SUITE_ID" \
  -H "X-API-Key: $DIALGEM_API_KEY"

Response 200

{
  "id": "string",
  "agent_id": "agent_123",
  "name": "Example",
  "description": "string",
  "publish_gate": "string",
  "case_count": 0,
  "latest_run": {
    "id": "string",
    "suite_id": "suite_123",
    "agent_id": "agent_123",
    "agent_version": 0,
    "status": "string",
    "trigger": "string",
    "fresh": false,
    "total": 0,
    "passed": 0,
    "failed": 0,
    "errored": 0,
    "skipped": 0
  },
  "created_at": "string",
  "updated_at": "string",
  "cases": [
    {
      "id": "string",
      "suite_id": "suite_123",
      "name": "Example",
      "position": 0,
      "caller": {},
      "dynamic_variables": {},
      "tool_mocks": {},
      "checks": [
        {}
      ],
      "created_at": "string",
      "updated_at": "string"
    }
  ]
}

Update suite

PATCH/v1/simulations/suites/{suite_id}

Path parameters

suite_idstringrequired

Request body application/json

namestring

descriptionstring

publish_gatestring

Returns

200 with the fields below.

idstringrequired

The suite's id.

agent_idstringrequired

The agent the suite tests.

namestringrequired

Display name.

publish_gatestringrequired

Whether a failing run blocks publishing the agent: `off`, `warn` or `block`.

case_countintegerrequired

Number of cases in the suite.

descriptionstring

What the suite covers.

latest_runobject

The most recent run. Null if the suite has never been run.

18 properties

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

curl -X PATCH "https://api.dialgem.com/v1/simulations/suites/SUITE_ID" \
  -H "X-API-Key: $DIALGEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example"
  }'

Response 200

{
  "id": "string",
  "agent_id": "agent_123",
  "name": "Example",
  "description": "string",
  "publish_gate": "string",
  "case_count": 0,
  "latest_run": {
    "id": "string",
    "suite_id": "suite_123",
    "agent_id": "agent_123",
    "agent_version": 0,
    "status": "string",
    "trigger": "string",
    "fresh": false,
    "total": 0,
    "passed": 0,
    "failed": 0,
    "errored": 0,
    "skipped": 0
  },
  "created_at": "string",
  "updated_at": "string"
}

Delete suite

DELETE/v1/simulations/suites/{suite_id}

Path parameters

suite_idstringrequired

Returns

204 with the resulting object.

curl -X DELETE "https://api.dialgem.com/v1/simulations/suites/SUITE_ID" \
  -H "X-API-Key: $DIALGEM_API_KEY"

Add case

POST/v1/simulations/suites/{suite_id}/cases

Path parameters

suite_idstringrequired

Request body application/json

namestringrequired

callerobjectrequired

checksarray<object>required

dynamic_variablesobject

tool_mocksobject

Returns

201 with the fields below.

idstringrequired

The case's id.

suite_idstringrequired

The suite the case belongs to.

namestringrequired

Display name.

positionintegerrequired

Order of the case within its suite, starting at 0.

callerobjectrequired

How the simulated caller behaves: either a fixed script of lines or a persona with a goal.

dynamic_variablesobjectrequired

Dynamic variables given to the agent for this case.

tool_mocksobjectrequired

What each tool answers during the case, keyed by tool name. Tools are never really called.

checksarray<object>required

What must be true of the conversation for the case to pass.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

curl -X POST "https://api.dialgem.com/v1/simulations/suites/SUITE_ID/cases" \
  -H "X-API-Key: $DIALGEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example",
    "caller": {},
    "checks": [
      {}
    ]
  }'

Response 201

{
  "id": "string",
  "suite_id": "suite_123",
  "name": "Example",
  "position": 0,
  "caller": {},
  "dynamic_variables": {},
  "tool_mocks": {},
  "checks": [
    {}
  ],
  "created_at": "string",
  "updated_at": "string"
}

Start run

POST/v1/simulations/suites/{suite_id}/runs

Path parameters

suite_idstringrequired

Request body application/json

No fields.

Returns

202 with the fields below.

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

curl -X POST "https://api.dialgem.com/v1/simulations/suites/SUITE_ID/runs" \
  -H "X-API-Key: $DIALGEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "fresh": false
  }'

Response 202

{
  "id": "string",
  "suite_id": "suite_123",
  "agent_id": "agent_123",
  "agent_version": 0,
  "status": "string",
  "trigger": "string",
  "fresh": false,
  "total": 0,
  "passed": 0,
  "failed": 0,
  "errored": 0,
  "skipped": 0
}

Update case

PATCH/v1/simulations/cases/{case_id}

Path parameters

case_idstringrequired

Request body application/json

namestring

callerobject

dynamic_variablesobject

tool_mocksobject

checksarray<object>

Returns

200 with the fields below.

idstringrequired

The case's id.

suite_idstringrequired

The suite the case belongs to.

namestringrequired

Display name.

positionintegerrequired

Order of the case within its suite, starting at 0.

callerobjectrequired

How the simulated caller behaves: either a fixed script of lines or a persona with a goal.

dynamic_variablesobjectrequired

Dynamic variables given to the agent for this case.

tool_mocksobjectrequired

What each tool answers during the case, keyed by tool name. Tools are never really called.

checksarray<object>required

What must be true of the conversation for the case to pass.

created_atstring

ISO 8601 creation time.

updated_atstring

ISO 8601 time of the last change.

curl -X PATCH "https://api.dialgem.com/v1/simulations/cases/CASE_ID" \
  -H "X-API-Key: $DIALGEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example"
  }'

Response 200

{
  "id": "string",
  "suite_id": "suite_123",
  "name": "Example",
  "position": 0,
  "caller": {},
  "dynamic_variables": {},
  "tool_mocks": {},
  "checks": [
    {}
  ],
  "created_at": "string",
  "updated_at": "string"
}

Delete case

DELETE/v1/simulations/cases/{case_id}

Path parameters

case_idstringrequired

Returns

204 with the resulting object.

curl -X DELETE "https://api.dialgem.com/v1/simulations/cases/CASE_ID" \
  -H "X-API-Key: $DIALGEM_API_KEY"

List runs

GET/v1/simulations/runs

Query parameters

suite_idstring

limitinteger

Default 30

Returns

200 with the fields below.

runsarray<object>required

Runs, newest first.

18 properties

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

curl "https://api.dialgem.com/v1/simulations/runs" \
  -H "X-API-Key: $DIALGEM_API_KEY"

Response 200

{
  "runs": [
    {
      "id": "string",
      "suite_id": "suite_123",
      "agent_id": "agent_123",
      "agent_version": 0,
      "status": "string",
      "trigger": "string",
      "fresh": false,
      "total": 0,
      "passed": 0,
      "failed": 0,
      "errored": 0,
      "skipped": 0
    }
  ]
}

Get run

GET/v1/simulations/runs/{run_id}

Path parameters

run_idstringrequired

Returns

200 with the fields below.

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 9 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

resultsarray<object>required

One result per case, in order.

14 properties

idstringrequired

The result's id.

case_namestringrequired

The case's name when the run started.

positionintegerrequired

Order of the case within the run, starting at 0.

statusstringrequired

One of `pending`, `passed`, `failed`, `errored` or `skipped`.

cachedbooleanrequired

True if this result was reused from an earlier identical run rather than run again.

transcriptarray<any>required

The simulated conversation, turn by turn.

tool_callsarray<any>required

The tool calls the agent made during the conversation.

variablesobjectrequired

Dynamic variables as they stood at the end of the conversation.

checksarray<any>required

Each check and whether it passed.

model_turnsintegerrequired

Model turns this case used.

Show 4 more parameters

duration_msintegerrequired

How long the case took, in milliseconds.

case_idstring

The case this result is for. Null if the case was deleted since.

reasonstring

Why the case failed, errored or was skipped.

finished_atstring

ISO 8601 time the case finished.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

curl "https://api.dialgem.com/v1/simulations/runs/RUN_ID" \
  -H "X-API-Key: $DIALGEM_API_KEY"

Response 200

{
  "id": "string",
  "suite_id": "suite_123",
  "agent_id": "agent_123",
  "agent_version": 0,
  "status": "string",
  "trigger": "string",
  "fresh": false,
  "total": 0,
  "passed": 0,
  "failed": 0,
  "errored": 0,
  "skipped": 0
}

Cancel run

POST/v1/simulations/runs/{run_id}/cancel

Path parameters

run_idstringrequired

Returns

200 with the fields below.

idstringrequired

The run's id.

suite_idstringrequired

The suite that was run.

agent_idstringrequired

The agent that was tested.

agent_versionintegerrequired

The agent version tested. 0 is the live draft.

statusstringrequired

One of `queued`, `running`, `completed`, `cancelled` or `failed`.

triggerstringrequired

What started the run: `manual` from the dashboard or `api`.

freshbooleanrequired

True if every case was run again even when an identical case had already run.

totalintegerrequired

Number of cases in the run.

passedintegerrequired

Cases that passed.

failedintegerrequired

Cases that ran and did not pass.

Show 8 more parameters

erroredintegerrequired

Cases that could not finish because of an error.

skippedintegerrequired

Cases that were not run, for example after the run was cancelled or the budget ran out.

model_turn_budgetintegerrequired

The most model turns the run may use.

model_turns_usedintegerrequired

Model turns used so far.

budget_exhaustedbooleanrequired

True if the run stopped early because it used its whole turn budget.

created_atstring

ISO 8601 time the run was created.

started_atstring

ISO 8601 time the run started. Null while queued.

finished_atstring

ISO 8601 time the run finished. Null until then.

curl -X POST "https://api.dialgem.com/v1/simulations/runs/RUN_ID/cancel" \
  -H "X-API-Key: $DIALGEM_API_KEY"

Response 200

{
  "id": "string",
  "suite_id": "suite_123",
  "agent_id": "agent_123",
  "agent_version": 0,
  "status": "string",
  "trigger": "string",
  "fresh": false,
  "total": 0,
  "passed": 0,
  "failed": 0,
  "errored": 0,
  "skipped": 0
}