Playground

Playground overview.

​

Getting Started

​

1. Select Your Test Data (Left Panel)

• Click the testset dropdown at the top of the left panel
• Select a testset that contains the data you want to test your prompt against
• The first testcase will be automatically selected

• Click on individual testcases to select them
• Hold Shift and click to select multiple testcases
• Selected testcases have a blue left border
• Hover over the info icon to see the full testcase data
• Testcases with a green flask icon have been tested

​

2. Edit Your Prompt (Middle Panel)

• Select a prompt from the dropdown in the header
• If you don’t have prompts yet, create one first in the Prompts section

• The left sidebar shows all versions of your selected prompt
• Click any version to switch to it
• Versions with unsaved changes show a save indicator
• The production version is marked with a badge

• Use the “Prompt Templates” tab to write your prompts
• The editor supports Jinja syntax for dynamic content
• Insert variables from your testcase data like {{variable_name}}
• Add multiple messages by clicking ”+ Add Message”
• Set message roles (System, User, Assistant) using the dropdown
• Remove messages with the minus button

• Switch to the “Model Settings” tab
• Choose your AI model (GPT-4, Claude, etc.)
• Adjust parameters like temperature, max tokens, and top-p
• These settings affect how the AI generates responses

​

3. Preview and Test (Right Panel)

• The “Template Preview” tab shows how your prompt looks with real data
• Variables are automatically replaced with values from selected testcases
• This helps you verify your Jinja templating is working correctly

• Click “Try” to test your prompt on selected testcases
• Click “Kickoff Run” to create a full run for the entire testset, which will appear in your run history
• Results appear in the “Results” tab automatically

• The “Results” tab shows AI responses for each testcase
• See response time, token count, and full output
• Click the completion badge to open a detailed results modal
• Green indicates successful completion, yellow shows partial results

​

Key Features

​

Jinja Templating

Hello {{name}}, your order #{{order_id}} is {{status}}.

​

Multi-testcase Testing

• Test individual testcases for quick iteration
• Run all testcases for comprehensive evaluation
• Kickoff full runs that are tracked in your run history
• Compare results across different prompt versions

​

Version Management

• Save new versions of your prompts with the “Save” button
• Switch between versions to compare performance
• Publish versions to production when ready

​

Real-time Preview

• See exactly how your prompt will look with real data
• Catch templating errors before running tests
• Understand how variables are populated

​

Best Practices

1. Start Small: Select one testcase first to quickly iterate on your prompt
2. Use Variables: Leverage Jinja templating to make prompts dynamic and reusable
3. Test Thoroughly: Run all testcases before publishing to production
4. Save Versions: Create new versions when making significant changes
5. Monitor Results: Check response times and token usage to optimize costs

​

Common Workflows