Operations · Intermediate
Curating and scoring conversation logs for model evaluation
Evaluate large volumes of prompt-response pairs, tag quality issues, and filter datasets for training
- When you askProvide dataset source and scoring rules
- ThenEvaluate conversations and label tags
- ResultCurated dataset draft ready for review
Inspect massive batches of conversational text logs to detect unwanted behaviors, label dialogues, and extract clean data subsets. The bot evaluates prompts and completions against custom criteria, tags issues, and prepares revised datasets for review.
Prompt to start with
BotBentoRecommended
Examine conversational logs pulled from the specified database or files. Score each conversation based on criteria such as sycophancy, adherence to instructions, and factual accuracy. Apply descriptive labels and flags to low-performing interactions. Generate a cleaned export draft with problematic entries removed or tagged. Never delete any records or overwrite existing datasets without explicit user confirmation.