Features
Organizations & Projects
Manage your team’s evaluation work with Organizations and Projects
Scorecard uses a hierarchical structure to organize your evaluation work: Organizations contain multiple Projects, and each Project contains your testsets, metrics, runs, and results.
Organizations
Organizations are the top-level entity in Scorecard that enables team collaboration and workspace isolation. All members of an organization can access all projects within that organization. API keys are scoped to a specific organization, not user or project.Organization member management dialog with a few users.
Projects
Each project serves as a container for:- Testsets: Collections of Testcases for evaluation
- Metrics: Custom evaluation criteria and scoring methods
- Runs: Evaluation executions and their results
- Prompts and Systems: The AI system versions being evaluated
- Traces: Live traces and logs of AI system calls
Projects overview page for an organization.
Creating a New Project
To create a project, click “New Project” in the top right of the Projects page and fill in the project details.Choose a descriptive project name for the system or use case you’re evaluating, such as “Customer Support Chatbot” or “Product Recommendation Engine.”
Creating a new project.