Scaling Billions of Rows of Ad Data from 2,000+ Sources at Salesforce

What it takes to unify fragmented data, scale metadata, and serve insights at enterprise speed

🎉 35,000 Subscribers! Let’s Celebrate with 20% Off 🎉

Byte-Sized Design just hit 35,000 subscribers, thank you for being part of this growing community of engineers who care about clarity, real-world learning, and great system design.

To celebrate, wevare offering 20% off your first year of Byte-Sized Design Premium for a limited time until the 10th of May.

Why Upgrade?

Byte-Sized Design Premium is your shortcut to:

• Real-world system design case studies from top tech companies

• Weekly premium editions with simplified concepts and clear takeaways

• 73 of the best engineering blogs, curated for your growth

• Principal FAANG engineer–written templates for interviews and architecture docs

• Full access to our entire archive

• More advanced editions launching soon

If you’ve been waiting to upgrade, now’s your moment. The price will go up as new features roll out—lock in the discount now.

👇
Upgrade and Save 20% Today

Thanks again for helping Byte-Sized Design grow. We’ve got more ambitious editions coming your way. Let’s jump into today’s exciting topic!

---

TL;DR

Salesforce reimagined how marketers interact with advertising data. Marketing Intelligence tackles one of the thorniest problems in digital marketing: unifying fragmented ad data from platforms like Google, Meta, LinkedIn, and Snapchat, and transforming it into real-time, AI-powered insights.

This was a deep technical challenge involving schema harmonization, metadata scalability, cloud-native replatforming, and automation across thousands of dynamic data sources. In this Byte-Sized Design edition, we break down how Salesforce built a system that ingests billions of ad events, provisions pipelines automatically, and makes marketers productive in minutes instead of weeks.

---

📖 What Will We Dive Into Today?

• Why Cross-Channel Marketing Data Is a Technical Nightmare

• How Salesforce Built Auto-Provisioning Data Pipelines at Scale

• Why Metadata Scalability Became the Hardest Problem

• The Real Role of Agentic Automation (And Where It Breaks)

• How They Tuned Query Performance Over Billions of Rows

• Lessons for Your Next Data Platform Migration

---

💥 The Problem: Data Fragmentation at Enterprise Scale

Marketers today spend tens to hundreds of millions in ad budgets, often spread across dozens of platforms. Each one has different schemas, update intervals, access patterns, and reporting quirks.

That data fragmentation creates three problems:

1. Ingesting and Normalizing Data Is Manual and Slow
   Connecting 1,000+ ad accounts used to take weeks, often via professional services and hand-rolled ETLs.

2. Insights Are Delayed or Missed Entirely
   Without harmonized data, performance analysis is delayed. Budget reallocation decisions come too late.

3. Non-Technical Users Are Locked Out
   Tools required deep SQL knowledge or help from data engineers to build anything meaningful.

Salesforce aimed to flip the script:
👉 One-click integrations, harmonized schemas, and real-time feedback loops—all usable by a marketing manager, not a data engineer.

---

⚙️ Architecture Overview: Auto-Provisioning Pipelines on Salesforce Data Cloud