> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scorecard.io/llms.txt
> Use this file to discover all available pages before exploring further.

# MCP Server Integration

> Use AI assistants as your evaluation companion with Scorecard's Model Context Protocol server

export const DarkLightImage = ({lightSrc, caption, alt, darkSrc = null, width = "1000"}) => {
  const getAbsoluteUrl = src => {
    if (src.startsWith('http://') || src.startsWith('https://')) {
      return src;
    }
    const currentUrl = typeof window !== 'undefined' ? window.location.origin : '';
    if (currentUrl.includes('.mintlify.app')) {
      const subdomain = currentUrl.split('.')[0].replace('https://', '');
      return `https://mintlify.s3.us-west-1.amazonaws.com/${subdomain}${src.startsWith('/') ? '' : '/'}${src}`;
    } else if (currentUrl === 'https://docs.scorecard.io') {
      return `https://mintlify.s3.us-west-1.amazonaws.com/scorecard-d65b5e8a${src.startsWith('/') ? '' : '/'}${src}`;
    } else {
      return `${currentUrl}${src.startsWith('/') ? '' : '/'}${src}`;
    }
  };
  const content = <>
      <img className="block dark:hidden" width={width} src={getAbsoluteUrl(lightSrc)} alt={alt} />
      <img className="hidden dark:block" width={width} src={getAbsoluteUrl(darkSrc || lightSrc.replace('light', 'dark'))} alt={alt} />
    </>;
  if (caption) {
    return <Frame caption={caption}>{content}</Frame>;
  } else {
    return content;
  }
};

## Overview

Scorecard's MCP (Model Context Protocol) server lets you manage projects, create testsets, configure metrics, run evaluations, and analyze results through natural language in any MCP-compatible client.

## Available Tools

The MCP server exposes \~45 tools covering metrics, scores, systems, annotations, and documentation search.

<Frame caption="Scorecard MCP server tools listed in Claude Code.">
  <img src="https://mintcdn.com/scorecard-d65b5e8a/GEa9A9d0Z6w_aCnI/images/mcp-tools-overview.png?fit=max&auto=format&n=GEa9A9d0Z6w_aCnI&q=85&s=da4955847cf5724aef5715c006bc6153" alt="Scorecard MCP server tool listing showing ~45 available tools across Metrics, Scores, Systems, Annotations, and Docs." width="3016" height="1166" data-path="images/mcp-tools-overview.png" />
</Frame>

## Setting Up the MCP Server

### Claude Code

Add the Scorecard remote MCP server with a single command:

```bash theme={null}
claude mcp add --transport http scorecard https://mcp.scorecard.io/mcp
```

Complete the OAuth authentication flow in your browser when prompted. Verify the connection:

```bash theme={null}
claude mcp list
```

You should see `scorecard: https://mcp.scorecard.io/mcp (HTTP) - ✓ Connected`.

### Claude Desktop

Go to Claude Desktop settings and click the "Connectors" tab. Click "Add custom connector" and paste the URL: `https://mcp.scorecard.io/mcp`. Click "Add", then "Connect" to login to Scorecard.

<DarkLightImage lightSrc="/images/claude-desktop-mcp-light.png" caption="Adding the Scorecard MCP connector in Claude Desktop." />

### Local configuration

You can run the MCP server locally via npx:

```sh theme={null}
export SCORECARD_API_KEY="your_api_key"
npx -y scorecard-ai-mcp@latest
```

For clients with a configuration JSON:

```json theme={null}
{
  "mcpServers": {
    "scorecard_ai": {
      "command": "npx",
      "args": ["-y", "scorecard-ai-mcp", "--client=claude", "--tools=dynamic"],
      "env": {
        "SCORECARD_API_KEY": "ak_MyAPIKey"
      }
    }
  }
}
```

## Examples

### Create a project and testset

```
Create a new Scorecard project called "Support Bot Eval". Then create a testset
called "Support Scenarios" with 10 testcases. Each testcase should have:
- inputs: "customerMessage" and "category" (billing, technical, or product)
- expected: "idealResponse"
```

### Create metrics

```
Create two metrics in the "Support Bot Eval" project:
1. "Response Accuracy" (integer 1-5) - How well does the response answer the question?
2. "Tone" (boolean) - Is the response professional and empathetic?
```

### Analyze results

```
Show me the latest run results for the "Support Bot Eval" project.
Which testcases scored lowest on Response Accuracy?
```

### Generate testcases from a codebase

In Claude Code, you can combine file access with the MCP server:

```
Read the API routes in src/api/ and generate 20 testcases covering
the edge cases for each endpoint. Add them to the "API Tests" testset
in project 1234.
```

### Iterate on metrics

```
The "Response Accuracy" metric is too lenient — update the prompt template
to penalize responses that miss key details from the ideal response.
```

## Technical Details

* Built on the [Model Context Protocol](https://modelcontextprotocol.io/) standard
* Compatible with any MCP client (Claude Code, Claude Desktop, Cursor, and more)
* Secured with OAuth authentication
* Open source: [github.com/scorecard-ai/scorecard-mcp](https://github.com/scorecard-ai/scorecard-mcp)