System versions

Create system version

Create a new version for a system.

Each version contains specific parameter values that match the system’s configSchema - things like model parameters, thresholds, or processing options. Once created, versions cannot be modified, ensuring stable reference points for evaluations.

When creating a system version:

The config object is validated against the parent system’s configSchema.
System versions with validation errors are still stored, with errors included in the response.
Validation errors indicate fields that don’t match the schema but don’t prevent creation.
Having validation errors may affect how some evaluation metrics are calculated.

POST

systems

{systemId}

configs

import Scorecard from 'scorecard-ai';

const client = new Scorecard({
  apiKey: process.env['SCORECARD_API_KEY'], // This is the default and can be omitted
});

const systemVersion = await client.systems.versions.create('12345678-0a8b-4f66-b6f3-2ddcfa097257', {
  config: { temperature: 0.1, maxTokens: 1024, model: 'gpt-4-turbo' },
  name: 'Production (Low Temperature)',
});

console.log(systemVersion.id);

{
  "id": "5fa85f64-5717-4562-b3fc-2c963f66afa7",
  "systemId": "12345678-0a8b-4f66-b6f3-2ddcfa097257",
  "name": "Production (Low Temperature)",
  "config": {
    "temperature": 0.1,
    "maxTokens": 1024,
    "model": "gpt-4-turbo"
  }
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

systemId

string

required

The ID of the system to create the system version for.

Example:

"12345678-0a8b-4f66-b6f3-2ddcfa097257"

Body

application/json

Response

201

application/json

System version created successfully

A SystemVersion defines the specific settings for a System Under Test.

System versions contain parameter values that determine system behavior during evaluation. They are immutable snapshots - once created, they never change.

When running evaluations, you reference a specific systemVersionId to establish which system version to test.

System versions will be validated against the system's configSchema, with non-conforming values generating warnings.

List system versionsRetrieve a paginated list of system versions for a specific system. System versions provide concrete parameter values for a System Under Test, defining exactly how the system should be configured during an evaluation run.

import Scorecard from 'scorecard-ai';

const client = new Scorecard({
  apiKey: process.env['SCORECARD_API_KEY'], // This is the default and can be omitted
});

const systemVersion = await client.systems.versions.create('12345678-0a8b-4f66-b6f3-2ddcfa097257', {
  config: { temperature: 0.1, maxTokens: 1024, model: 'gpt-4-turbo' },
  name: 'Production (Low Temperature)',
});

console.log(systemVersion.id);

{
  "id": "5fa85f64-5717-4562-b3fc-2c963f66afa7",
  "systemId": "12345678-0a8b-4f66-b6f3-2ddcfa097257",
  "name": "Production (Low Temperature)",
  "config": {
    "temperature": 0.1,
    "maxTokens": 1024,
    "model": "gpt-4-turbo"
  }
}

Overview

Projects

Testsets

Testcases

Metrics

Runs

Records

Scores