Skip to main content
Scorecard uses a hierarchical structure to organize your evaluation work: Organizations contain multiple Projects, and each Project contains your testsets, metrics, runs, and results.

​
Organizations

Organizations are the top-level entity in Scorecard that enables team collaboration and workspace isolation. All members of an organization can access all projects within that organization. API keys are scoped to a specific organization, not user or project.

​
Projects

Each project serves as a container for:
  • Testsets: Collections of Testcases for evaluation
  • Metrics: Custom evaluation criteria and scoring methods
  • Records: Evaluation executions and their results, traces and logs

​
Creating a New Project

To create a project, click “Create Project” in the projects menu and go through the steps.
Choose a descriptive project name for the system or use case you’re evaluating, such as “Customer Support Chatbot” or “Product Recommendation Engine.”