Handbook

User shares photo:
📷 Family photo with location: "Home address: 123 Main St"

User's expectation:
✅ Visible to friends on Facebook
✅ Maybe suggest nearby friends
❌ NOT used to target ads based on home location
❌ NOT shared with third-party apps
❌ NOT used to track family members' movements
❌ NOT visible on other Meta apps without permission

Question: Can Meta guarantee this?

Old approach (doesn't scale):
Write policies → Train developers → Hope they follow rules → Audit manually

PAI approach (scales):
Write policies → Embed in infrastructure → Impossible to violate → Automatic verification

2018: GDPR in Europe
- Requires "purpose limitation"
- Data can only be used for stated purposes
- Violations: Up to 4% of global revenue ($4.6B+ for Meta)

2020: CCPA in California
- Similar requirements
- Growing globally

2023: 100+ privacy laws worldwide
- Each with different requirements
- Manual compliance: impossible at scale

Meta's challenge:
├─ 3.8 billion users
├─ 100+ products (Facebook, Instagram, WhatsApp, etc.)
├─ Millions of data assets (tables, APIs, models)
├─ Billions of lines of code
├─ Thousands of engineers
└─ Code changes: 1,000+ per day

Manual privacy tracking:
❌ Impossible
❌ Error-prone
❌ Doesn't scale
❌ Can't verify

User shares photo with location: "At Disneyland with kids"

Allowed uses (explicit consent):
✅ Show photo to friends
✅ Suggest friends nearby
✅ Add to family memories

NOT allowed (no consent):
❌ Target location-based ads to the user
❌ Track children's locations
❌ Share with third-party advertisers
❌ Use for surveillance purposes

Challenge: How do you enforce this across billions of lines of code?

Simple example:

User shares photo with location (SOURCE)
    ↓
Web Endpoint
    ↓
Database Table
    ↓
Data Warehouse
    ↓
AI Model
    ↓
Ad Targeting System (SINK)

Question: Does location data from photos reach the ad targeting system?

Assets to track:
├─ Web endpoints: 100,000+
├─ Database tables: 1,000,000+
├─ Data warehouse tables: 10,000,000+
├─ AI models: 100,000+
├─ Logging systems: 10,000+
└─ Microservices: 50,000+

Technology stacks:
├─ Languages: Hack, C++, Python, JavaScript, Rust, Java
├─ Data systems: MySQL, RocksDB, Presto, Spark, Hive
├─ Processing: Web servers, batch jobs, streaming pipelines
└─ ML platforms: PyTorch, TensorFlow, custom frameworks

Code changes:
├─ 1,000+ commits per day
├─ 5,000+ engineers making changes
├─ Assets constantly being created/modified/deleted
└─ No way to manually track

Manual data lineage:
├─ Engineers document data flows
├─ Create diagrams in slides/wikis
├─ Update spreadsheets
└─ Hope it stays accurate

Reality:
❌ Documentation immediately out of date
❌ Engineers forget to update
❌ Takes weeks to trace one data flow
❌ Incomplete (only ~10% coverage)
❌ Can't verify accuracy

Week 1: Start investigation
- Interview Photos team: "Where does location data from photos go?"
- Photos team: "Uh... to the database, some logs, maybe some APIs?"
- Manually read code to find all references

Week 2: Trace database
- Find 67 downstream tables that read from photo_metadata
- Each has 20-100 downstream consumers
- Need to check each one

Week 3: Trace APIs
- Find 25 APIs that expose photo/location data
- Who calls these APIs? Need to search all code
- Each API has 50+ callers

Week 4: Trace logs
- Location data might be in 300+ log tables
- Each log feeds into data warehouse
- Data warehouse has 1,000s of downstream tables

Week 8: Still not done
- Found 10,000+ potential data flows
- 90% are false positives
- Still missing actual flows
- Code has changed since week 1 (already out of date!)

Result: 
- 2 months spent
- Incomplete and inaccurate
- Out of date immediately
- Costs $50,000+ per investigation
- Impossible to scale

While you're documenting:
├─ New tables are created
├─ New code is deployed
├─ New data flows are added
└─ Documentation is already wrong

It's like mapping a city where streets
change every minute!

Input: location = "Disneyland, Anaheim, CA"

Transformation 1:
location → {metadata: {location: "Disneyland, Anaheim, CA", lat: 33.8, lon: -117.9}}

Transformation 2:
{locations: ["Disneyland", "Universal Studios"]} → {nearby_count: 2}

Transformation 3:
"Disneyland, Anaheim, CA" → "LA_AREA_THEME_PARK" (category code)

Question: Are these the same data?
Manual tracking: Impossible to tell

Direct flow (easy to track):
photo.location → nearby_friends_feature

Indirect flow (nearly impossible to track):
photo.location 
  → tmp_table_abc123
  → aggregated_location_features
  → model_training_dataset
  → user_interest_model
  → ad_targeting_service
  → third_party_advertisers

Manual tracking: Would take months to discover this chain

To manually track location data:
├─ Review: 1,000,000+ lines of code
├─ Interview: 500+ engineers
├─ Check: 100,000+ assets
├─ Trace: 1,000,000+ potential flows
└─ Time: 6-12 months (minimum)

But:
├─ Code changes daily
├─ Engineers leave/join
├─ Systems evolve
└─ Impossible to keep up

Cost: $5M+ per privacy requirement
Timeline: Months to years
Accuracy: 50-60% at best

GDPR requirement: Location data must be purpose-limited

To implement:
1. Find all places location data is used
2. Add privacy controls at those places
3. Verify controls work

Without lineage:
❌ Can't find all places (manual search takes months)
❌ Miss hidden flows (transformed data)
❌ Can't verify (no visibility)

Result: Can't comply with GDPR
Risk: $4.6B fine (4% of global revenue)

2018: Developer adds new feature

Code:
def show_targeted_ads(user_id):
    photos = get_user_photos(user_id)
    # Use ALL photo metadata including location
    return target_ads_by_location(photos)

Oops: Just used children's location data for ad targeting!

Without lineage:
- Nobody knows location data leaked to ads
- No alerts
- No way to discover
- Privacy violation goes undetected

With lineage:
- Automatic detection: "Photo location data flowing to ad targeting"
- Alert: "Potential privacy violation - children's data"
- Block: "Cannot deploy code that violates policy"

Product team: "We want to add a new feature"

Privacy team: "First, we need to verify data flows"

Manual review:
- Week 1-4: Document current data flows
- Week 5-8: Trace new data flows
- Week 9-12: Write privacy review
- Week 13-16: Get approvals
- Week 17-20: Implement controls

Total: 5 months before feature can launch

Result:
- Product team frustrated
- Innovation slowed
- Competitors move faster
- Company loses market share

Auditor: "How do you ensure religion data from Dating 
          doesn't reach your ad system?"

Without lineage:
Company: "We have policies and training"
Auditor: "Can you prove it?"
Company: "Uh... we trust our engineers?"
Auditor: "That's not acceptable"
Company: "We can review code manually?"
Auditor: "How long would that take?"
Company: "6-12 months... per data type..."
Auditor: "Not acceptable. You're not compliant."

Result: Can't pass audit, can't operate

Create massive documentation:
├─ Data dictionaries
├─ Flow diagrams
├─ Spreadsheets
└─ Wikis

Reality:
- Documentation created: 10,000 pages
- Time to create: 1 year
- Cost: $2M
- Out of date: Immediately
- Engineers read it: 5%
- Actual use: Near zero

Result: $2M wasted ❌

Require privacy review for every code change

Reality:
- Code changes per day: 1,000+
- Time per review: 2 hours
- Reviewers needed: 250 full-time
- Cost: $50M per year
- Bottleneck: Everything slows down
- Engineers bypass process

Result: Creates bureaucracy, doesn't scale ❌

Train all engineers on privacy rules

Reality:
- Engineers trained: 5,000
- Time per training: 4 hours
- Cost: $2M per year
- Engineers remember: 20%
- Engineers follow rules: 30%
- Mistakes still happen: Daily

Result: Doesn't prevent violations ❌

"Just ask the team that owns the code"

Reality:
Engineer: "Uh, I think it goes to these 5 tables..."
[Actually goes to 50+ tables]

Engineer: "I don't know, I didn't write this code"
[Original author left 2 years ago]

Engineer: "It's in the logs somewhere"
[Logs go to 100+ downstream systems]

Result: Incomplete, inaccurate ❌

Old thinking:
"Privacy is about policies and training"

New thinking:
"Privacy is about infrastructure and automation"

Old approach:
- Write privacy policies
- Train developers
- Review code manually
- Hope nothing breaks

New approach (PAI):
- Build privacy into infrastructure
- Automatic data lineage
- Automatic control enforcement
- Automatic verification

1. Automated data lineage collection
   ✅ Zero manual work
   ✅ Tracks all data flows
   ✅ Updates in real-time
   ✅ Handles all tech stacks

2. High accuracy
   ✅ 90%+ precision
   ✅ Minimal false positives
   ✅ Catches transformations
   ✅ Finds indirect flows

3. Complete coverage
   ✅ All programming languages
   ✅ All data systems
   ✅ All processing types
   ✅ Millions of assets

4. Developer-friendly
   ✅ Easy to query
   ✅ Visual tools
   ✅ Fast results
   ✅ Actionable insights

5. Scalable
   ✅ Handles billions of data flows
   ✅ Processes in real-time
   ✅ Cost-effective
   ✅ Reliable

6. Verifiable
   ✅ Can prove compliance
   ✅ Audit trail
   ✅ Continuous monitoring
   ✅ Alerts for violations

Technical challenges:
├─ Multiple programming languages
├─ Different data paradigms (SQL, APIs, models)
├─ Transformed data (hard to match)
├─ Millions of assets
├─ Billions of data flows
├─ Real-time updates
├─ Minimal performance impact
└─ Must be 90%+ accurate

Organizational challenges:
├─ 5,000+ engineers to support
├─ Can't disrupt existing systems
├─ Must integrate with all tech stacks
├─ Gradual rollout (can't migrate everything at once)
└─ Adoption across hundreds of teams

Timeline: 3+ years
Investment: $50M+
Team size: 100+ engineers

User shares family photo with location: "Home, 123 Main St"

Allowed uses:
✅ Show photo to friends
✅ Create family memories album
✅ Suggest nearby friends once

Not allowed:
❌ Target ads based on home location
❌ Track family/children's locations over time
❌ Share with third-party apps
❌ Use for location-based marketing

Question: How do you enforce this across millions of lines of code?
Answer: You need data lineage!

Engineer adds new ad targeting feature:
def target_local_ads(user_id):
    photos = get_user_photos(user_id)  # Oops! Includes home location
    locations = extract_locations(photos)  # Now location leaked
    return show_nearby_business_ads(locations)

Result: Privacy violation, nobody knows ❌

Data lineage system detects:
- Photo location data flowing from Photos → ad targeting
- Automatic alert: "PRIVACY VIOLATION DETECTED"
- Block deployment: "Cannot deploy code that violates policy"
- Alert teams: "Fix privacy violation before deploying"

Result: Privacy protected automatically ✅

COPPA (Children's Online Privacy Protection Act):
- Data from users under 13 has strict restrictions
- Location data especially sensitive
- Cannot be used for behavioral advertising
- Cannot be shared with third parties
- Cannot be used to track children

Example: Kid posts photo from school
- School location must be protected
- Cannot be used for ads
- Cannot be shared
- Must be deleted after X days

Challenge: Need to track all flows of children's location data
across all Meta products

Scale:
- 100M+ children on Meta platforms
- Billions of photos with locations
- Data in millions of assets
- Need 100% accuracy (regulatory requirement)

Manual tracking: Impossible
Automated lineage: Essential

Without data lineage:
├─ Can't implement privacy controls
├─ Can't verify compliance
├─ Risk: $4.6B+ in fines
├─ User trust destroyed
└─ Product development slowed

With data lineage:
├─ Automatic privacy enforcement
├─ Continuous compliance verification
├─ Risk: Minimized
├─ User trust maintained
└─ Fast product iteration

Data lineage is not optional—it's essential

Meta's data lineage system:

Collection methods:
├─ Static code analysis (simulates execution)
├─ Privacy Probes (runtime instrumentation)
├─ SQL analysis (batch processing)
├─ Config parsing (AI systems)
└─ Integrated across all tech stacks

Key technologies:
├─ Automatic payload matching
├─ Cross-language support
├─ Real-time processing
├─ Graph database for relationships
└─ Developer-friendly query tools

Results:
✅ Millions of assets tracked
✅ Billions of data flows mapped
✅ 90%+ accuracy
✅ Real-time updates
✅ Developer-friendly
✅ Enabled PAI at scale

Engineering time saved: 90%+
Privacy violations prevented: 100s per year
Cost: $50M investment, $500M+ value

The Question: "Where does location data from photos go?"

Without data lineage:
├─ Manual code review: 6 months
├─ Incomplete results: 60% coverage
├─ Inaccurate: 50+ false positives per true flow
├─ Out of date: Immediately
└─ Cost: $5M per investigation

With data lineage:
├─ Automated query: 30 seconds
├─ Complete results: 95%+ coverage
├─ Accurate: 90%+ precision
├─ Always up to date: Real-time
└─ Cost: $0 per query (after building system)