Cowbell · 2023 - Present

Enterprise Data Foundation

A reusable data foundation for risk, products, analytics and AI, not one-off pipelines. Ingestion, medallion warehouse layers, domain marts and operational distribution from a single architectural spine.

The problem

Every team was building its own pipelines for the same data. Business logic was duplicated, lineage was unclear, and new products meant new plumbing.

Key decisions

  • Separate ingestion from transformation so sources can change without breaking models.
  • Keep raw data for traceability before any business logic is applied.
  • Build reusable domain marts instead of team-specific extracts.
  • Decouple analytical and operational workloads.
  • Treat data distribution as a platform capability, not a per-project afterthought.

Outcome

One foundation now serves risk intelligence, analytics, products, operational applications and AI workloads without each team rebuilding pipelines.

AWSRedshiftGlueS3KinesisdbtTerraform

Architecture flow

  1. Data sources
  2. Ingestion, streaming and replication
  3. Bronze: raw and traceable
  4. Silver: standardized
  5. Gold: curated business models
  6. Domain data marts
  7. Operational distribution, APIs, search and AI