Operational freshness
Is the delivery current?
Measure elapsed time from the latest successful load. Do not use the transaction date as a substitute.
Data operations
Freshness is not the latest business date.
A historical dataset can be complete and current as a delivery even when its latest transaction date is old.
Visual model
Test recency and coverage separately.
Operational freshness checks when the pipeline loaded the data. Period completeness checks whether the declared reporting window is intact.
Load recencyPeriod completenessAudit history
Operational freshness
Time since the latest successful idempotent load.
Separate rules
Period completeness
Source, FX, and marts cover the declared period.
EvidenceObserved + expected values
→
AuditRun and rule history
→
ConsumersGrafana + Streamlit
Period completeness
Is the declared window intact?
Test source dates, FX coverage, mart boundaries, and expected monthly partitions.
Shared contract
Calculate once.
Store monitoring facts in governed views. Dashboard tools read the result instead of rebuilding the rules.
Recovery evidence
Prove the control can fail.
The first live check detected a 243.662-hour stale load. A controlled reload restored a 6/6 pass.
Where I used it
Airflow audit record
Each run writes overall status and rule-level evidence to MySQL.
View architecture →Grafana operations view
The primary local dashboard shows current health, run history, and failed checks.
View evidence →Streamlit inspection view
A read-only Python application exposes the same observed and expected values.
View evidence →Implementation evidence
Four views. Two consumers. One definition.
vw_freshness_latest_run and vw_freshness_latest_results support the current state. History views support trend and audit analysis. Grafana and Streamlit have no write action.