Data pipelines,
built with precision.

The product needed live Profit & Loss, Balance Sheet, and Cash Flow data pulled from members' QuickBooks Online accounts and surfaced inside a financial dashboard. I evaluated two approaches, then designed, built, and shipped the one that gave us full control.

Two approaches, evaluated

End-to-end architecture

How it works, in detail

The service handles the full OAuth 2.0 lifecycle with Intuit, then stores each member's access & refresh tokens Fernet-encrypted in a Postgres (Cloud SQL) database — keyed by a stable member identifier rather than a raw company ID. Report endpoints (/reports/pnl, /reports/balance-sheet, /reports/cash-flow) fetch from the QBO API with a 24-hour cache to stay well within rate limits, and are protected against cross-site abuse via a nonce/source cookie scheme. Cleaned output lands in Postgres analytics marts, modeled in dbt for the dashboard to read.

CI/CD & deployment

Work flowed through a dedicated staging branch with pull-request review before promotion. The service builds and deploys as a Google Cloud service (Cloud Run), so releases are containerized, repeatable, and rollback-friendly — no manual server babysitting. Secrets and DB credentials are injected at runtime, never committed.

A member-based organization was moving off Nimble AMS (plus a legacy Salesforce instance) onto a brand-new Salesforce org spanning Sales Cloud, Revenue Cloud, and Service Cloud. My role: design and run the data migration — extraction, transformation, field mapping, and load — at a scale that broke past the original estimates.

The ETL process

Scale & the hard problems

Profiling revealed the true migration volume was around 5 million records — far beyond the original 1M assumption — which reshaped the whole batching and performance strategy. The Global Person ID model was the highest-priority blocker: get identity resolution wrong and every downstream relationship breaks. I mapped the join logic carefully, verified against real data rather than assumptions, and flagged the count breach early so scope and timeline could be re-planned.

Tooling

Built on SSIS and KingswaySoft for the heavy ETL lifting, with Salesforce Data Loader for targeted loads — a stack I've used across multiple PostgreSQL→Salesforce and legacy→Salesforce migrations.

Supported a migration from Salesforce CPQ to Revenue Cloud Advanced (RCA), focused on the data side: extracting CPQ objects, transforming them, and mapping fields into the new revenue model. Coordinated across a stack that included SSIS, KingswaySoft, Workato, Sage Intacct, and Snowflake, keeping quote-to-revenue continuity intact through the cutover.

A product team needed a new tracking attribute to flow all the way from raw source data into the reporting layer, without breaking any downstream model. I designed the change through dbt's layered architecture rather than patching the mart directly — so the field stays traceable and testable at every step.

Layered flow

How it was built

The new field was added at the staging layer first, verified against real production rows (not assumed sample values), then threaded through the intermediate layer where the core business joins live, and finally exposed in the mart that the dashboard reads. Each layer got its own YAML documentation and dbt tests, and the change went through a senior-reviewed pull request before merging to the shared branch.

Key lesson carried forward: never guess example values in YAML descriptions or tests — always pull a real row from the warehouse to confirm the field behaves as expected before documenting it.

Leadership needed to compare marketing performance across 100+ locations fairly — but each location had different session volumes, call-tracking setups, and campaign spends. The challenge wasn't just pulling data; it was building one consistent methodology so every location could be compared like-for-like.

Data flow

The methodology problem

The trickiest part was defining Conversion Rate consistently: calls, sales, website enquiry forms, and email leads all needed to be counted — and double-counted paths excluded — the same way, everywhere. I documented the exact formula and every exclusion rule so any location's number could be trusted at a glance, then handed the full methodology over in writing for the reporting team to maintain independently.

Operational teams needed dashboards that updated in near real time, without waiting on a nightly batch job. I connected Snowflake directly to Looker Studio for live KPI visibility, then separately led a workstream migrating an existing reporting suite off legacy tooling and onto Power BI — keeping both platforms in sync with the same underlying warehouse models.

Architecture

The Power BI migration was designed so it didn't disrupt existing report consumers — the underlying Snowflake models stayed the single source of truth, with both BI tools pointed at the same governed layer instead of duplicating logic in each front end.

Founded and run a content platform helping aspiring data analysts across India prepare for interviews — covering SQL, Python, and career guidance at scale. Rather than writing every article by hand, I built an automated content pipeline so the site could grow consistently.

Content pipeline

The platform covers SQL fundamentals, Python for analytics, and career guides tailored to the Indian job market, with content scaled through the automation workflow rather than manual publishing — freeing time to focus on quality and monetization strategy (currently working toward Google AdSense approval).