· 4 min read
A dashboard release checklist before a BI change goes live
I use a short BI release checklist to compare outputs, review filter and definition changes, confirm sign-off, and keep a rollback path before business-critical dashboard changes go live.
· 5 min read
by Berhan Turkkaynagi
A business-critical dashboard change is not ready just because the pull request looks tidy.
Before I approve a BI change, I want one small release artifact that says which slice was checked, what number should move, who signed off, and how I roll it back if the live result lands wrong.
It plays the same role for BI that the deployment summary I want for dbt Core promotions plays upstream: one compact artifact that makes scope and approval visible before promotion, so the leadership review is not the first serious review of the release.
Problem
A dashboard change can look small in code review and still change the number leadership sees in production.
One filter tweak, one rewritten chart calculation, or one quiet definition note can change the story leaders see at 08:00 ET. If nobody compares outputs on a trusted slice, records the expected delta, and names the rollback path, the team ends up explaining the number during the review instead of shipping it with confidence.
Default approach
- Freeze the release scope first: which tiles, metrics, filters, and date windows are actually changing.
- Compare before-and-after outputs on one validation slice the business owner already trusts, using the same date window and filters on both sides.
- Read the filter logic and metric-definition changes line by line in the pull request or release note, not just in chart screenshots.
- Write down the expected visible difference before the dashboard is republished.
- Get explicit owner sign-off on the changed slice and the definition note.
- Keep one rollback path ready before I call the release safe, including the revert step and the prior comparison snapshot.
This is release evidence, not polish review. A dashboard can look cleaner and still be less trustworthy if the release note never says which filter changed or why the number moved.
Example: the dashboard release checklist in the pull request
Here is the kind of release artifact I want attached to a revenue dashboard pull request before it goes live:
Dashboard release checklist
| Field | Value |
|---|---|
| Dashboard | executive revenue scorecard |
| PR | #418 |
| Change owner | analytics engineering |
| Business owner | finance analytics lead |
| Validation slice | 2025-11-24 to 2025-12-02, US enterprise orders only |
| Change summary | Exclude sandbox and QA orders from the revenue filter Update the revenue mix chart to use settled net revenue Update the seven-day trend chart denominator to match the settled view |
| Review step | What I confirm | Status |
|---|---|---|
| Before/after output comparison | Headline revenue: $4,218,330 -> $4,201,980 (-0.39%), expected from sandbox-order removal Revenue mix by segment: enterprise share changes from 61.2% -> 61.5%, expected Seven-day trend: two days move because the denominator now matches settled revenue | pass |
| Filter logic review | New exclusion: order_source in ('sandbox', 'qa') No change to refunded-order handling No date-window change outside the stated validation slice | pass |
| Definition note | Dashboard note added: revenue mix and seven-day trend now use settled net revenue instead of booked gross revenue Release note linked in the PR summary for downstream readers | pass |
| Owner sign-off | Finance analytics lead reviewed the changed slice at 16:10 ET Expected visible differences match the release note | approved |
| Rollback path | Revert PR #418 and republish the previous dashboard definition Keep the prior validation-slice snapshot in the release note for comparison | ready |
That checklist earns its space because each line changes the next decision.
The before-and-after slice tells me whether the delta is understood. The filter review catches quiet scope changes. The definition note keeps the dashboard aligned with how the business talks about the metric. Owner sign-off makes the change explicit. The rollback path keeps the release safe if the live result still surprises us.
If the release still lands wrong after that, I move into incident mode and keep The incident note template I wish every analytics team used open beside the debugging work. The point of the checklist is to make that handoff rare.
Tradeoffs
- Breaks when: teams force the full checklist onto low-risk dashboards and make routine edits slower than they need to be → Mitigation: keep the strict version for executive and business-critical dashboards, and use a lighter release note for low-risk BI work.
- Breaks when: there is no stable validation slice or baseline report to compare against → Mitigation: create one trusted slice before the release review starts, even if the first version is manual and narrow.
- Breaks when: the filter or metric definition is still disputed when the release review starts → Mitigation: hold the go-live and resolve the definition in writing before anyone debates chart polish.
- Breaks when: an urgent hotfix needs to land during an active incident → Mitigation: collapse the checklist to the trust boundary, owner approval, and rollback steps first, then backfill the release note after the number is safe again.
Close
Next step: For one business-critical dashboard, write the release checklist you want attached to its next logic or filter change before it reaches leadership.
The release conversation feels calmer when the expected delta and rollback path are visible before anyone is defending a surprising card.