---
name: review-new-inference-benchmark-runs-against-a
description: "Spot throughput and latency changes before they reach the team. A recurring review bot that checks newly added benchmark outputs against a saved baseline, highlights notable throughput or latency changes, and prepares a concise regression note for a maintainer to review before anything is shared."
license: CC-BY-4.0
compatibility: "Needs access to: GitHub, Docs, Notes."
metadata:
  author: "BotBento Editorial"
  version: "2026-11-17"
  source: "https://botbento.com/usecases/review-new-inference-benchmark-runs-against-a/"
  kind: workflow
  hermes:
    tags: ["benchmarks", "performance", "review", "inference"]
    category: ops
---
# Review new inference benchmark runs against a stored baseline

Spot throughput and latency changes before they reach the team

## When to use

A recurring review bot that checks newly added benchmark outputs against a saved baseline, highlights notable throughput or latency changes, and prepares a concise regression note for a maintainer to review before anything is shared.

## Steps

1. On a schedule: Weekly benchmark review
2. Compare latest benchmark run to baseline
3. Draft regression summary for approval

## Instructions

Each week, read the newest benchmark result files in the repository and compare key throughput and latency numbers with the stored baseline. Flag any metric that moved in an unexpected direction. Draft a short summary listing the changed metrics and the likely cause. Do not post, send, or open issues; present the draft for approval first.

## Source

Summarized in our own words from [Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium](https://news.ycombinator.com/item?id=50533270) (Hacker News, checked 2026-11-17). Any results described there are the authors’ own.

From the BotBento use-case directory: https://botbento.com/usecases/review-new-inference-benchmark-runs-against-a/. Licence: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
