Building a Data Stack From the Ground Up

Situation

Data was scattered across a CRM, a billing system, an operational platform, and a set of spreadsheets maintained by different people on different cadences. There was no source of truth for any operational question. Every meaningful question — how many active engagements, how much was outstanding, what the close rate looked like by segment — required someone to manually pull from three places and reconcile, often with inconsistent answers depending on who pulled what. Leadership wanted a real data foundation but had been told it would require ripping out and replacing the systems they already had.

Approach

We built the stack on Microsoft Fabric. Scheduled scripts pulled data from the source systems — including from systems without clean APIs, using extraction techniques described in our other case work — and loaded it into a Lakehouse organized by domain. Pipeline notebooks then cleaned and modeled the data into well-shaped tables, joinable with SQL or directly consumable by Power BI. The output flowed into multiple consumption surfaces: dashboards for leadership, embedded reports in PowerPoint and SharePoint for board and investor reviews, scheduled PDF distributions for stakeholders who wanted email-delivered summaries, and operator-facing views for front-line staff. The same underlying data layer served everyone, so the numbers were always consistent regardless of which surface someone was looking at. We have built variants of this pattern on Snowflake, Postgres/Supabase, and AWS Redshift depending on the right environment for the engagement.