Related: https://github.com/ClickHouse/ClickHouse/pull/120927

Changelog category (leave one):

• CI Fix or Improvement (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

A master_head performance shard now fails on a performance dashboard confirmed_regression only when the shard's own report also reports that query as slower.

Description

120927 made a master_head shard fail when the performance dashboard classifies any of that shard's slowdowns as confirmed_regression, and removed the in-tree verdict from the decision. The dashboard classifies a single measurement episode; the in-tree verdict carries a per-query bar built from a month of master runs of that exact query (1.5x the p99 of abs(diff), compare.sh:1221). So a chronically noisy micro benchmark reaches the dashboard's highest tier while the job's own report does not call it changed.

On #117122 arm 6/6, three of the four dashboard confirmed_regression rows that shard measured sit 2.5-3.5x below their own historical bar (diff 0.229 vs 0.57, 0.261 vs 0.91, 0.222 vs 0.85); the fourth clears it, and the shard reds on all four. Of the 28 reds in the first 10.5 h, 19 are on shards whose own report published no slower query at all.

The fix makes the two judges agree: a confirmed_regression blocks only when the same (test, query_index) is also slower in this job's own published verdict, recomputed from report/all-query-metrics.tsv, the file the gate already reads and which exports changed_threshold for exactly this purpose. The dashboard is not called when there is no candidate, and a corrupt row fails the check instead of being skipped. pr-120176's tpcds #71 +2225% still fails, checked against the live API.

It also restores the > 10 slower floor that #120927 dropped from master_head (it fired twice in the preceding week); happy to drop those two lines on request. Nothing currently runs ci/tests/, so I will remove the new file there on request.

Not fixed here: the accountability set comes from this attempt's unseeded 10-query sample while the dashboard run merges every attempt, so a re-run can drop a standing verdict.

Replay over every dashboard-gate red in the window (click to expand)

Each row's in-tree verdict is read from that job's own published `report.html`, and intersected with the live dashboard's `confirmed_regression` set for the shard's arch. 28 rows: **19 go green, 8 stay red, 1 inconclusive** (its report was overwritten by a re-run).

The 8 that stay red, i.e. where both judges name the same query:

| PR | shard | query both judges flag |
|---|---|---|
| #120874 | amd 1/6, arm 1/6 | `order_by_read_in_order #1` |
| #119598 | arm 6/6 | `optimize_time_filter_with_preimage #6` |
| #50053 | arm 6/6 | `optimize_time_filter_with_preimage #2`, `#6` |
| #90740 | arm 3/6 | `bitmap_cardinality #1` |
| #110102 | arm 3/6 | `bitmap_cardinality #4` |
| #117122 | arm 6/6 | `timeseries_to_grid_aggregation_functions #7` |
| #121042 | amd 1/6 | `order_by_read_in_order #1` |

In all 19 that go green the dashboard names a query the shard's own report did not publish as slower.

Workflow [PR]
Sync PR [sync-upstream/pr/121090]
<!-- CI automatic block end :ci_links: -->