Create system
Create a new system definition that specifies the interface contracts for a component you want to evaluate.
A system acts as a template that defines three key contracts through JSON Schemas:
- Input Schema: What data your system accepts (e.g., user queries, context documents)
- Output Schema: What data your system produces (e.g., responses, confidence scores)
- Config Schema: What parameters can be adjusted (e.g., model selection, temperature)
This separation lets you evaluate any system as a black box, focusing on its interface rather than implementation details.
Authorizations
Bearer authentication header of the form Bearer <token>
, where <token>
is your auth token.
Path Parameters
The ID of the system's Project.
"314"
Body
Response
System created successfully
A System Under Test (SUT) defines the interface to a component or service you want to evaluate.
It specifies three contracts through schemas:
- inputSchema: The structure of data the system accepts.
- outputSchema: The structure of data the system produces.
- configSchema: The parameters that modify system behavior.
This abstraction lets you evaluate any system as a black box, focusing on its interface rather than implementation details. It's particularly useful for systems with variable outputs or complex internal state.
Systems are templates - to run evaluations, pair them with a SystemConfig that provides specific parameter values.