Score Configs

A Score Configuration defines the structure and validation rules for evaluation scores used when annotating traces or observations.

Create a Score Configuration

Use Add New Score Config to define a custom evaluation metric.

Fields

1. Name – Specifies the name of the evaluation metric.

Examples:

• Accuracy
• Quality
• Relevance

2. Data Type – Defines the type of value accepted by the score configuration. Supported types include:

a. Numeric

Accepts numeric values within a defined range.

Shows additional fields:

• Minimum – Lower bound of the score
• Maximum – Upper bound of the score

Examples:

• 0 to 1
• 1 to 5

b. Categorical

Allows users to select from predefined categories.

Displays:

• Value – Numeric value or identifier
• Label – User-friendly display name shown during annotation

Use Add Category to define multiple options.

Example:

Value	Label
0	Poor
1	Average
2	Good

c. Boolean

Represents a binary evaluation (true/false).

Standard convention:

• 1 → True
• 0 → False

d. Text

Accepts free-form text input and no additional fields are displayed.

Used for:

• Comments
• Explanations
• Qualitative feedback

3. Description – Provides additional context about the score measures and how it should be interpreted.

Availability

After a score configuration is created, it becomes available in the Annotate panel, where it can be applied to traces and observations.