Reading the Learning panel
The Learning panel sits above the knowledge list on a project's Knowledge tab. It answers one question: is this project's knowledge loop actually compounding, or does it only look busy?
Four numbers, each a headline over the last 30 days with a daily sparkline under it.
| Metric | What it counts |
|---|---|
| Repeat error rate | How often an error signature seen in the last 7 days comes back. A loop that is working should make repeats rarer. |
| Recall hit rate | How often a recall returned something an agent then reported using, via recall_used. |
| Knowledge precision | Of the entries promoted to trusted, how many were later contradicted. |
| Candidate to trusted (p50) | Median time an entry waits between being written and being confirmed. A review backlog shows up here. |
Under them, a point-in-time count of trusted and candidate entries.
What these numbers are not
They are agent-reported telemetry, not ground truth. Every input comes from
agents describing their own work: an agent decides an event is an error, an
agent decides a recalled entry was useful, an agent decides a failure refutes an
entry. The panel claims a direction, not a measurement of reality, and it is
labelled that way on screen.
There is no comparison against other projects or other RelayRoom installs, and there are no external benchmarks anywhere in the panel. A number here means something about this project's own history and nothing else.
Why a metric sometimes refuses to show a number
Below a sample threshold the panel renders "not enough data" instead of a percentage. A ratio computed from a handful of observations is exactly the kind of false confidence this whole feature exists to prevent, and rounding it to a reassuring percentage would be the first dishonest thing the system did.
| Metric | Shows a number once there are |
|---|---|
| Knowledge precision | 20 trusted entries |
| Recall hit rate | 50 recall queries in the window |
| Repeat error rate | 20 error events in the window |
| Candidate to trusted p50 | 20 promotions in the window |
A quiet project stays below these and honestly says so. That is the intended behaviour, not a setup problem.
Why the last 14 days are marked provisional
Precision depends on a late, negative signal. An entry is promoted, and only later does a contradiction arrive to say it was wrong. So the most recent stretch always looks better than it will turn out to be: the bad news is still in flight.
RelayRoom handles that in two places rather than papering over it:
- The sparkline marks the most recent 14 days provisional.
- The precision headline is computed from settled days only - days whose 14-day contradiction window has already closed.
The daily rollup also recomputes the trailing 14 days on every run and backfills them, so a day's precision figure keeps improving in accuracy as its window closes. The trusted and candidate counts are the opposite kind of number - a snapshot of that day's stock - so they are deliberately left out of that recomputation. A past day's stock stays the past's.
The normalization version
The panel displays a normalization version alongside the metrics. The thresholds above, the 30-day window, and each metric's definition are part of a contract; if one of them changes, the series before and after the change are not the same measurement.
Rather than let a definition change silently bend a line, the version number changes with it, so a jump in the chart has a visible cause. If you are comparing this month against last, check that the version did not move between them.
What is not here
No claim about how much the loop improves anything. RelayRoom does not have enough real-world data to make one honestly, and a number invented to fill that gap would undermine every other number on the page. What the panel gives you is what it measured and how it measured it; the conclusion is yours.