> ## Documentation Index > Fetch the complete documentation index at: https://docs.scorecard.io/llms.txt > Use this file to discover all available pages before exploring further. # Create Metric > Create a new Metric for evaluating system outputs. The structure of a metric depends on the evalType and outputType of the metric. ## OpenAPI ````yaml post /projects/{projectId}/metrics openapi: 3.1.0 info: title: Scorecard API description: REST API for Scorecard version: 1.0.0 servers: - url: https://api2.scorecard.io/api/v2 security: - ApiKeyAuth: [] paths: /projects/{projectId}/metrics: post: summary: Create Metric description: >- Create a new Metric for evaluating system outputs. The structure of a metric depends on the evalType and outputType of the metric. operationId: createMetric parameters: - in: path name: projectId description: The ID of the Project to create the Metric in. schema: type: string example: '314' required: true requestBody: content: application/json: schema: anyOf: - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: ai description: AI-based evaluation type. guidelines: type: string description: Guidelines for AI evaluation on how to score the metric. promptTemplate: type: string description: >- The complete prompt template for AI evaluation. Should include placeholders for dynamic content. evalModelName: type: string default: gpt-4o description: The AI model to use for evaluation. temperature: type: number minimum: 0 maximum: 2 default: 0 description: The temperature for AI evaluation (0-2). outputType: type: string const: int description: Integer output type. passingThreshold: type: integer minimum: 1 maximum: 5 default: 4 description: >- The threshold for determining pass/fail from integer scores (1-5). required: - name - evalType - promptTemplate - outputType description: A Metric with AI evaluation and integer output. title: AI int metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: human description: Human-based evaluation type. guidelines: type: string description: Guidelines for human evaluators. outputType: type: string const: int description: Integer output type. passingThreshold: type: integer minimum: 1 maximum: 5 default: 4 description: >- The threshold for determining pass/fail from integer scores (1-5). required: - name - evalType - outputType description: A Metric with human evaluation and integer output. title: Human int metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: heuristic description: Heuristic-based evaluation type. guidelines: type: string description: Guidelines for heuristic evaluation logic. outputType: type: string const: int description: Integer output type. passingThreshold: type: integer minimum: 1 maximum: 5 default: 4 description: >- The threshold for determining pass/fail from integer scores (1-5). required: - name - evalType - outputType description: A Metric with heuristic evaluation and integer output. title: Heuristic int metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: ai description: AI-based evaluation type. guidelines: type: string description: Guidelines for AI evaluation on how to score the metric. promptTemplate: type: string description: >- The complete prompt template for AI evaluation. Should include placeholders for dynamic content. evalModelName: type: string default: gpt-4o description: The AI model to use for evaluation. temperature: type: number minimum: 0 maximum: 2 default: 0 description: The temperature for AI evaluation (0-2). outputType: type: string const: float description: Float output type (0-1). passingThreshold: type: number minimum: 0 maximum: 1 default: 0.9 description: >- Threshold for determining pass/fail from float scores (0.0-1.0). required: - name - evalType - promptTemplate - outputType description: A Metric with AI evaluation and float output. title: AI float metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: human description: Human-based evaluation type. guidelines: type: string description: Guidelines for human evaluators. outputType: type: string const: float description: Float output type (0-1). passingThreshold: type: number minimum: 0 maximum: 1 default: 0.9 description: >- Threshold for determining pass/fail from float scores (0.0-1.0). required: - name - evalType - outputType description: A Metric with human evaluation and float output. title: Human float metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: heuristic description: Heuristic-based evaluation type. guidelines: type: string description: Guidelines for heuristic evaluation logic. outputType: type: string const: float description: Float output type (0-1). passingThreshold: type: number minimum: 0 maximum: 1 default: 0.9 description: >- Threshold for determining pass/fail from float scores (0.0-1.0). required: - name - evalType - outputType description: A Metric with heuristic evaluation and float output. title: Heuristic float metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: ai description: AI-based evaluation type. guidelines: type: string description: Guidelines for AI evaluation on how to score the metric. promptTemplate: type: string description: >- The complete prompt template for AI evaluation. Should include placeholders for dynamic content. evalModelName: type: string default: gpt-4o description: The AI model to use for evaluation. temperature: type: number minimum: 0 maximum: 2 default: 0 description: The temperature for AI evaluation (0-2). outputType: type: string const: boolean description: Boolean output type. required: - name - evalType - promptTemplate - outputType description: A Metric with AI evaluation and boolean output. title: AI boolean metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: human description: Human-based evaluation type. guidelines: type: string description: Guidelines for human evaluators. outputType: type: string const: boolean description: Boolean output type. required: - name - evalType - outputType description: A Metric with human evaluation and boolean output. title: Human boolean metric - type: object properties: name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: heuristic description: Heuristic-based evaluation type. guidelines: type: string description: Guidelines for heuristic evaluation logic. outputType: type: string const: boolean description: Boolean output type. required: - name - evalType - outputType description: A Metric with heuristic evaluation and boolean output. title: Heuristic boolean metric examples: Create AI Metric: value: name: Response Accuracy description: Evaluates if the response is factually accurate outputType: boolean evalType: ai guidelines: Check if the response contains factually correct information promptTemplate: >- Please evaluate if the following response is factually accurate: {{outputs.response}} evalModelName: gpt-4o temperature: 0.1 summary: Created AI Metric description: Response after successfully creating an AI metric. Create Human Metric: value: name: Response Quality description: Human evaluation of response quality outputType: int evalType: human guidelines: Rate the quality of the response on a scale of 1-5 passingThreshold: 3 summary: Create Human Metric description: >- Request to create a human-evaluated metric with integer scoring. Create Heuristic Metric: value: name: Response Length description: Checks if response meets length requirements outputType: boolean evalType: heuristic guidelines: Response should be between 50-200 characters summary: Create Heuristic Metric description: >- Request to create a rule-based metric that uses algorithmic validation. Create Simple Heuristic Metric: value: name: Non-Empty Response description: Checks if response is not empty guidelines: Checks if response is not empty outputType: boolean evalType: heuristic summary: Create Simple Heuristic Metric description: >- Request to create a simple heuristic metric without guidelines. responses: '201': description: Metric created successfully content: application/json: schema: $ref: '#/components/schemas/Metric' examples: Created AI Metric: value: id: '456' name: Response Accuracy description: Evaluates if the response is factually accurate outputType: boolean evalType: ai guidelines: >- Check if the response contains factually correct information promptTemplate: >- Please evaluate if the following response is factually accurate: {{response}} evalModelName: gpt-4o temperature: 0.1 summary: Created AI Metric description: Response after successfully creating an AI metric. '401': $ref: '#/components/responses/UnauthenticatedError' '500': $ref: '#/components/responses/ServiceError' x-codeSamples: - lang: JavaScript source: |- import Scorecard from 'scorecard-ai'; const client = new Scorecard({ apiKey: 'My API Key', }); const metric = await client.metrics.create('314', { evalType: 'ai', name: 'Response Accuracy', outputType: 'boolean', promptTemplate: 'Please evaluate if the following response is factually accurate: {{outputs.response}}', description: 'Evaluates if the response is factually accurate', evalModelName: 'gpt-4o', guidelines: 'Check if the response contains factually correct information', temperature: 0.1, }); console.log(metric); - lang: Python source: |- from scorecard_ai import Scorecard client = Scorecard( api_key="My API Key", ) metric = client.metrics.create( project_id="314", eval_type="ai", name="Response Accuracy", output_type="boolean", prompt_template="Please evaluate if the following response is factually accurate: {{outputs.response}}", description="Evaluates if the response is factually accurate", eval_model_name="gpt-4o", guidelines="Check if the response contains factually correct information", temperature=0.1, ) print(metric) - lang: cURL source: |- curl https://api2.scorecard.io/api/v2/projects/$PROJECT_ID/metrics \ -X POST \ -H "Authorization: Bearer $SCORECARD_API_KEY" components: schemas: Metric: anyOf: - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: ai description: AI-based evaluation type. guidelines: type: string description: Guidelines for AI evaluation on how to score the metric. promptTemplate: type: string description: >- The complete prompt template for AI evaluation. Should include placeholders for dynamic content. evalModelName: type: string default: gpt-4o description: The AI model to use for evaluation. temperature: type: number minimum: 0 maximum: 2 default: 0 description: The temperature for AI evaluation (0-2). outputType: type: string const: int description: Integer output type. passingThreshold: type: integer minimum: 1 maximum: 5 default: 4 description: >- The threshold for determining pass/fail from integer scores (1-5). required: - id - name - description - evalType - guidelines - promptTemplate - evalModelName - temperature - outputType - passingThreshold description: A Metric with AI evaluation and integer output. title: AI int metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: human description: Human-based evaluation type. guidelines: type: string description: Guidelines for human evaluators. outputType: type: string const: int description: Integer output type. passingThreshold: type: integer minimum: 1 maximum: 5 default: 4 description: >- The threshold for determining pass/fail from integer scores (1-5). required: - id - name - description - evalType - guidelines - outputType - passingThreshold description: A Metric with human evaluation and integer output. title: Human int metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: heuristic description: Heuristic-based evaluation type. guidelines: type: string description: Guidelines for heuristic evaluation logic. outputType: type: string const: int description: Integer output type. passingThreshold: type: integer minimum: 1 maximum: 5 default: 4 description: >- The threshold for determining pass/fail from integer scores (1-5). required: - id - name - description - evalType - guidelines - outputType - passingThreshold description: A Metric with heuristic evaluation and integer output. title: Heuristic int metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: ai description: AI-based evaluation type. guidelines: type: string description: Guidelines for AI evaluation on how to score the metric. promptTemplate: type: string description: >- The complete prompt template for AI evaluation. Should include placeholders for dynamic content. evalModelName: type: string default: gpt-4o description: The AI model to use for evaluation. temperature: type: number minimum: 0 maximum: 2 default: 0 description: The temperature for AI evaluation (0-2). outputType: type: string const: float description: Float output type (0-1). passingThreshold: type: number minimum: 0 maximum: 1 default: 0.9 description: Threshold for determining pass/fail from float scores (0.0-1.0). required: - id - name - description - evalType - guidelines - promptTemplate - evalModelName - temperature - outputType - passingThreshold description: A Metric with AI evaluation and float output. title: AI float metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: human description: Human-based evaluation type. guidelines: type: string description: Guidelines for human evaluators. outputType: type: string const: float description: Float output type (0-1). passingThreshold: type: number minimum: 0 maximum: 1 default: 0.9 description: Threshold for determining pass/fail from float scores (0.0-1.0). required: - id - name - description - evalType - guidelines - outputType - passingThreshold description: A Metric with human evaluation and float output. title: Human float metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: heuristic description: Heuristic-based evaluation type. guidelines: type: string description: Guidelines for heuristic evaluation logic. outputType: type: string const: float description: Float output type (0-1). passingThreshold: type: number minimum: 0 maximum: 1 default: 0.9 description: Threshold for determining pass/fail from float scores (0.0-1.0). required: - id - name - description - evalType - guidelines - outputType - passingThreshold description: A Metric with heuristic evaluation and float output. title: Heuristic float metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: ai description: AI-based evaluation type. guidelines: type: string description: Guidelines for AI evaluation on how to score the metric. promptTemplate: type: string description: >- The complete prompt template for AI evaluation. Should include placeholders for dynamic content. evalModelName: type: string default: gpt-4o description: The AI model to use for evaluation. temperature: type: number minimum: 0 maximum: 2 default: 0 description: The temperature for AI evaluation (0-2). outputType: type: string const: boolean description: Boolean output type. required: - id - name - description - evalType - guidelines - promptTemplate - evalModelName - temperature - outputType description: A Metric with AI evaluation and boolean output. title: AI boolean metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: human description: Human-based evaluation type. guidelines: type: string description: Guidelines for human evaluators. outputType: type: string const: boolean description: Boolean output type. required: - id - name - description - evalType - guidelines - outputType description: A Metric with human evaluation and boolean output. title: Human boolean metric - type: object properties: id: type: string description: The ID of the Metric. name: type: string description: The name of the Metric. description: type: - string - 'null' default: null description: The description of the Metric. evalType: type: string const: heuristic description: Heuristic-based evaluation type. guidelines: type: string description: Guidelines for heuristic evaluation logic. outputType: type: string const: boolean description: Boolean output type. required: - id - name - description - evalType - guidelines - outputType description: A Metric with heuristic evaluation and boolean output. title: Heuristic boolean metric description: >- A Metric defines how to evaluate system outputs against expected results. ApiError: type: object properties: code: type: string message: type: string details: type: object additionalProperties: true x-stainless-any: true required: - code - message - details description: An API error. responses: UnauthenticatedError: description: Error indicating that the request is not authenticated. content: application/json: schema: $ref: '#/components/schemas/ApiError' examples: Authentication failure: value: code: UNAUTHORIZED message: Invalid or missing authentication token details: {} summary: Authentication failure description: >- Error returned when authentication credentials are invalid or missing. ServiceError: description: >- An internal service error indicating an issue with the Scorecard service. content: application/json: schema: $ref: '#/components/schemas/ApiError' examples: Internal error: value: code: INTERNAL_ERROR message: An unexpected error occurred while processing your request. details: {} summary: Internal error description: Generic error when an unexpected internal issue occurs. securitySchemes: ApiKeyAuth: type: http scheme: bearer bearerFormat: starts with ak_ ````