Every data pipeline looks clean in the architecture diagram. Then it meets five departments' worth of spreadsheets, each with its own definition of "customer," and the diagram stops being useful.
The failure mode nobody diagrams
Pipelines don't usually break from too little data — they break from ambiguous data that looks valid. A department renames a column, a date format quietly shifts from DD/MM to MM/DD, or two teams both call something "revenue" while meaning different things.
| Failure type | How often we saw it | Typical detection lag |
|---|---|---|
| Schema drift | Very common | Days |
| Silent unit mismatch | Common | Weeks |
| Definition conflict (e.g. "active user") | Common | Months |
| Straightforward missing data | Rare | Hours |
Governance before pipelines, not after
The teams that avoid month-two breakage build a data contract with each source system before writing a single pipeline — an explicit, versioned definition of what each field means, owned by the source team, not the analytics team.
That's data quality incidents per month after introducing contracts at month zero versus the incident rate we saw on projects that skipped this step. The gap isn't subtle.
The dashboard is the easy part
By the time you're building the dashboard, the hard work is already done or it isn't. Teams that treat the warehouse and governance layer as the real deliverable — and the dashboard as a thin final step — are the ones whose pipelines are still running cleanly a year later.