The source-trust scorecard I use before promoting a table to production

The source-trust scorecard is the promotion record between a valid contract and a production dependency.

The dangerous moment is the promotion step. The columns land, row volume looks normal, and the first schema checks pass. Then a downstream model inherits key churn, late corrections, or an owner-response gap nobody reviewed before the table entered the daily dashboard path.

My default is to separate the source contract from the promotion decision. The contract says what the source is supposed to do. The source-trust scorecard records what the source has actually done since ingestion started and whether that behavior is safe enough to depend on.

Problem

I do not want the first real test of a source table to happen after a critical model depends on it.

That is the trap with contract-compliant sources. The agreement may define row meaning, cadence, key rules, accepted values, and an owner, but the observation window can still show behavior that would make production noisy. A table can arrive late twice in two weeks. A vendor can merge catalogs and rekey items. A correction can restate data outside the expected window. A new status can appear before downstream logic knows what to do with it.

Those facts do not always make the source unusable. They do mean I need a promotion record that says which uses can move forward, which ones need guardrails, and which ones stay blocked.

Default approach

I use the scorecard after the initial source contract exists and before the table becomes a dependency for a critical model, dashboard, or recurring review.

The review stays small on purpose:

• Check cadence reliability against the promised landing pattern.
• Check key stability against the joins, deduplication, and history the downstream model needs.
• Check whether corrections are predictable, bounded, and explainable.
• Check null and domain risk on fields the downstream logic treats as required.
• Check whether late arrivals are normal, visible, and compatible with the reporting window.
• Check whether the source owner responds fast enough when evidence shows a defect.
• Write the promotion decision as promote, promote with guardrails, or hold.

Every score needs a reference: a check result, run log, source-owner note, issue, or review record. If I cannot point to the evidence, I treat the score as an opinion and keep it out of the promotion decision.

Example: a vendor inventory source that needs guardrails

Here is the kind of source that makes the scorecard useful: a vendor inventory feed that already has a lightweight source contract.

The table lands with expected columns. Row volume stays near the normal range. Basic schema checks pass. A shallow review would promote it into both the daily replenishment model and the executive inventory snapshot.

The observation window says something narrower:

Source trust scorecard
Source table: vendor_inventory_snapshot
Candidate use: daily available-to-promise and replenishment models
Observation window: 14 daily loads after initial source contract
Promotion decision: promote with guardrails
Decision owner: analytics engineering, reviewed with inventory source owner

1) Cadence reliability
- Expected: daily landing by 05:30 ET
- Observed evidence: 12 of 14 loads landed on time; 2 arrived 45-70 minutes late with source-owner notice
- Risk note: acceptable for noon planning refresh, not safe for early-morning executive snapshot
- Score: guardrail

2) Key stability
- Expected: one row per source_item_id, location_id, snapshot_date
- Observed evidence: 0 duplicate rows on stable keys, but 3% of items were rekeyed after vendor catalog merge
- Risk note: downstream joins need a rekey bridge before production promotion
- Score: guardrail

3) Correction pattern
- Expected: corrections can restate the last 72 hours only
- Observed evidence: two corrections landed inside 72 hours; one correction restated a five-day-old partition
- Risk note: publish recent days as preliminary until correction window is proven or exception is explained
- Score: guardrail

4) Null/domain risk
- Expected: item_id, location_id, on_hand_qty, and status are populated with accepted values
- Observed evidence: location_id null rate stayed below 0.2%; status introduced HOLD without prior notice
- Risk note: HOLD needs an accepted-value rule and owner-confirmed downstream handling
- Score: guardrail

5) Late-arrival pattern
- Expected: late rows are rare and visible through load metadata
- Observed evidence: late rows cluster around vendor weekend reconciliation and affect Monday snapshots
- Risk note: Monday planning slice needs a freshness and late-row annotation before release
- Score: guardrail

6) Owner response
- Expected: source owner acknowledges production-blocking defects same business day
- Observed evidence: owner answered cadence and status questions same day; rekey explanation took two days
- Risk note: key churn is the only unresolved promotion blocker
- Score: guardrail

Decision
- Promote with guardrails after adding the rekey bridge, accepted-value rule for HOLD, and preliminary label for recent correction windows.
- Hold executive-facing snapshots until the next observation window shows cadence and key behavior are stable.

That decision is not a clean promote, and it is not a full hold.

The source is useful for the noon replenishment path after the rekey bridge and HOLD handling are in place. Recent partitions need a preliminary label until correction behavior is clearer. The executive snapshot waits because early-morning cadence and key stability are still the two ways this table can put the wrong number in front of the wrong room.

The scorecard keeps that judgment visible. It shows why one production use can move forward while another stays blocked, and it gives the next reviewer the evidence behind the boundary.

Tradeoffs

• Breaks when: teams use the scorecard to block exploratory analysis or every low-risk staging table → Mitigation: reserve it for sources that will feed critical models, dashboards, recurring reviews, or other downstream commitments.
• Breaks when: scoring becomes subjective gatekeeping or a permanent grade → Mitigation: record the exact check, run log, source-owner note, issue, or review evidence behind each score, then revisit the score when source behavior changes.
• Breaks when: teams wait for a perfect source and delay useful delivery → Mitigation: allow promote with guardrails when the limits are explicit, such as preliminary labels, blocked dashboard surfaces, rekey bridges, or owner-confirmed correction windows.
• Breaks when: the scorecard duplicates the source contract and creates two places for the same rule → Mitigation: let the contract define expectations, and let the scorecard record observed behavior against those expectations during the promotion window.

Close

Next step: Before one critical model or dashboard depends on a new source, score the source against observed cadence, key stability, corrections, null risk, late arrivals, owner response, and the promote/guard/hold decision.

The scorecard is complete only when the downstream owner can see why the table moves forward, moves with guardrails, or stays held.