The problem

Forecasts are only as good as the data and compute behind them. Large enterprise datasets had to be prepared, fed to models and turned into predictions on a reliable schedule.

Key decisions

  • Separate data preparation from model computation so each scales on its own.
  • Make every prediction run repeatable and traceable back to its inputs.
  • Ship through fully automated delivery: no manual steps from commit to production.
  • Build on shared platform abstractions rather than per-service tooling.

Outcome

Prediction workloads ran at enterprise scale on a platform where releases across many microservices were fully automated, the delivery standard I have brought to every platform since.

AzurePySparkSynapseMachine LearningCI/CD

Architecture flow

  1. Enterprise sales, supply and calendar data
  2. Ingestion and preparation
  3. Feature computation with PySpark
  4. Model computation
  5. Predictions and forecast outputs
  6. Supply-chain planning applications