Skip to main content
Scorecard uses a hierarchical structure to organize your evaluation work: Organizations contain multiple Projects, and each Project contains your testsets, metrics, runs, and results.

​
Organizations

Organizations are the top-level entity in Scorecard that enables team collaboration and workspace isolation. All members of an organization can access all projects within that organization. API keys are scoped to a specific organization, not user or project.
Screenshot of the organization member management dialog with a few users.Screenshot of the organization member management dialog with a few users.

Organization member management dialog with a few users.

​
Projects

Each project serves as a container for:
  • Testsets: Collections of Testcases for evaluation
  • Metrics: Custom evaluation criteria and scoring methods
  • Records: Evaluation executions and their results, traces and logs
Screenshot of the projects overview page with two projects.Screenshot of the projects overview page with two projects.

Projects overview page for an organization.

​
Creating a New Project

To create a project, click “Create Project” in the projects menu and go through the steps.
Choose a descriptive project name for the system or use case you’re evaluating, such as “Customer Support Chatbot” or “Product Recommendation Engine.”
Screenshot of the create project modal in the UI.Screenshot of the create project modal in the UI.

Creating a new project.