Scorecard uses a hierarchical structure to organize your evaluation work: Organizations contain multiple Projects, and each Project contains your testsets, metrics, runs, and results.
Organizations
Organizations are the top-level entity in Scorecard that enables team collaboration and workspace isolation.
All members of an organization can access all projects within that organization. API keys are scoped to a specific organization, not user or project.
Projects
Each project serves as a container for:
- Testsets: Collections of Testcases for evaluation
- Metrics: Custom evaluation criteria and scoring methods
- Records: Evaluation executions and their results, traces and logs
Creating a New Project
To create a project, click “Create Project” in the projects menu and go through the steps.
Choose a descriptive project name for the system or use case you’re evaluating, such as “Customer Support Chatbot” or “Product Recommendation Engine.”
