Handbook

📚 This is Part 2 of the “Scaling Your API” Series

• Part 1: Performance & Infrastructure ← - How to handle more users and traffic
• Part 2 (this page): Design & Architecture - How to organize teams and design better systems
• Part 3: Choosing the Right Database → - Database selection for your API
• Part 4: Load Balancing & High Availability → - Keeping your API always available
• Part 5: Monitoring & Performance → - Tracking and improving API performance

What This Guide Is About

In Part 1, we talked about how to make your systems handle more users—like adding more servers and making things faster.

But there’s another challenge: how do you organize hundreds of people and keep everything working together smoothly?

When companies like PayPal, Amazon, or Netflix grow, their biggest challenge isn’t just handling traffic. It’s making sure that:

• Hundreds of teams can work without stepping on each other’s toes
• Everything still works together seamlessly
• Customers get a simple, consistent experience

This guide shares 10 practical strategies that big companies use. Don’t worry—we’ll explain everything in simple terms!

The Big Picture: From One System to Many Small Ones

The Problem with One Big System

Imagine your entire company’s software is like one giant machine. Everything is connected to everything else.

What goes wrong:

• When you want to change one part, you might break other parts
• Everyone has to coordinate on every change
• It takes forever to make updates
• If one thing breaks, everything breaks

The Solution: Break It Into Smaller Pieces

Instead of one giant system, break it into smaller, independent pieces. Each piece handles one job:

• One piece handles user accounts
• Another handles payments
• Another handles orders
• Another sends emails

Why this helps:

• Each team can work on their piece without bothering others
• You can update one piece without touching the rest
• If one piece breaks, the others keep working
• Teams can choose the best tools for their specific job

But now you have a new challenge: With hundreds of small pieces, how do you keep everything organized and consistent?

Let’s look at 10 strategies that solve this.

Strategy 1: Manage Your APIs Like an Investment Portfolio

What This Means

Think about how investment firms manage money. They don’t just buy random stocks. They look at their entire portfolio and make strategic decisions:

• Which investments are most important?
• Which ones aren’t performing well?
• Where should we focus our resources?

Do the same with your APIs (the connections between your systems).

Why This Matters

Without portfolio thinking:

• Three different teams might all build a “user login” system because no one knows the others are doing it
• You spend equal effort on everything, even unimportant stuff
• Old, broken systems stick around forever because no one plans to replace them

With portfolio thinking:

• You can see when teams are duplicating work and combine efforts
• You focus your best people on the most important systems
• You have a plan to retire old systems properly

Real-Life Example

Think about it like managing a store:

Bad way: Every department orders whatever they want. You end up with 5 different cash register systems because no one coordinated.

Good way: One person oversees all systems, makes sure departments share what makes sense, and plans upgrades systematically.

Strategy 2: Design for Customers, Not Your Company Structure

The Problem

Most companies accidentally build systems that reflect their org chart instead of what customers actually want.

Example: If you have separate teams for “User Management,” “Payments,” and “Orders,” you might build three separate systems that customers have to deal with individually.

But customers don’t care about your team structure! They just want to “buy something.”

How to Fix It

Design your systems around what customers want to do, not how your company is organized.

Example: Buying Something Online

Bad approach (org-chart focused):

• Customer calls “user-team/check-if-user-valid”
• Then calls “inventory-team/check-stock”
• Then calls “payment-team/process-card”
• Then calls “order-team/create-order”
• Then calls “notification-team/send-email”

Good approach (customer focused):

• Customer says “checkout” with all their information
• Behind the scenes, your system coordinates all those teams
• Customer only sees one simple action

Real-Life Example

Think about ordering at a restaurant:

Bad way: You have to go separately to the meat counter, vegetable stand, and baker, then assemble your meal yourself.

Good way: You tell the waiter what you want. They coordinate with everyone in the kitchen. You just get your meal.

Your systems should be the waiter, not force customers to be the restaurant manager.

Strategy 3: Speak Two Languages—Business and Technical

The Challenge

Your systems serve two very different audiences:

Business people think about:

• “Can we accept payments in Europe?”
• “Can customers track their shipments?”
• “What’s the cost per transaction?”

Technical people think about:

• “What data format does this use?”
• “How do we call this system?”
• “What happens if it fails?”

The Solution

Create two different explanations of the same system:

For business stakeholders:

Payment Processing System

What it does: Let customers pay with credit cards and digital wallets

Business value:
- Accept payments in 50+ countries
- Support 15+ payment methods
- Secure and compliant with regulations
- Works 99.99% of the time

Cost: $0.02 per transaction plus 2.9%

For technical teams:

Payment API

Endpoint: POST /payments

Send this data:
- Amount (number)
- Currency (USD, EUR, etc.)
- Payment method token
- Customer ID

You'll get back:
- Payment ID
- Status (succeeded/failed)
- Timestamp

Real-Life Example

Think about buying a car:

Salesperson talks about: Safety, comfort, fun to drive, looks nice

Mechanic talks about: Engine size, horsepower, transmission type, fuel efficiency

Both are talking about the same car, just in ways that matter to different people.

Strategy 4: Design First, Build Second

The Old Way (Build First)

Most teams jump straight into building:

1. Team starts writing code
2. Spends weeks building the system
3. Finally shows it to users
4. Users say “This doesn’t work for us!”
5. Team has to rebuild everything (expensive and slow!)

The Better Way (Design First)

Create a detailed plan and fake version first:

1. Team designs what the system will do (on paper)
2. Creates a fake version users can try
3. Gets feedback quickly (cheap to change!)
4. Improves the design based on feedback
5. Only then starts building the real thing

Why This Is Better

Real-Life Example

Think about building a house:

Bad way: Start construction, build for months, then show the owner. They hate the kitchen layout. Now you have to tear down walls.

Good way: Show the owner blueprints and a 3D model. They want changes? Just redraw. Much cheaper than rebuilding!

Strategy 5: Keep the Complexity Hidden

The Problem: Showing Too Much

Too often, systems expose all their internal messy details to users.

Bad example - exposing internal mess:

• /database-sync/users
• /cache-layer/user-profiles
• /legacy-system-adapter/update

Users see all your internal plumbing and have to understand it!

Good example - clean and simple:

• /users (create a user)
• /users/123 (get user 123)
• /users/123 (update user 123)

Users just see clean, simple operations.

The Solution: Client Advocate

Have someone whose job is to represent the users’ perspective:

• Reviews designs and asks “Why do users need to know this?”
• Simplifies complicated processes
• Hides internal complexity
• Tests if it’s actually easy to use

Real-Life Example

Think about using an ATM:

Bad design: Show users the complex backend—”Step 1: Connect to database cluster #3. Step 2: Run fraud check algorithm. Step 3: Send request to Federal Reserve…”

Good design: Show users a simple screen—”Enter PIN → Choose Amount → Take Cash”

Users don’t need to know about all the complex stuff happening behind the scenes!

Strategy 6: Build Systems That Last and Grow

The Problem with Constant Breaking Changes

Bad approach: Every year, force everyone to update:

• 2020: Version 1.0 works
• 2021: Version 2.0 - Version 1.0 stops working!
• 2022: Version 3.0 - Version 2.0 stops working!
• Everyone constantly has to update their code

The Better Way: Add, Don’t Break

Instead of changing existing things, just add new optional things:

Example:

Version 1.0 (2020):

User information:
- ID number
- Name
- Email

Version 1.0 (2021) - Added features, didn’t break anything:

User information:
- ID number (still works!)
- Name (still works!)
- Email (still works!)
- Profile picture (NEW! Optional!)
- Theme preference (NEW! Optional!)

Old systems that only know about ID, Name, and Email? They still work perfectly!

New systems can use the new features if they want.

When DO You Need a New Version?

Only make a breaking change when absolutely necessary:

• Complete redesign of the entire system
• Removing something that nobody uses anymore
• Switching to a completely different technology

Real-Life Example

Think about phone updates:

Good approach (like iOS): Apps built for iOS 12 still work on iOS 17, but new apps can use iOS 17 features.

Bad approach: Every year, forcing everyone to rewrite their apps from scratch.

Strategy 7: Make Things Flow Naturally

Part A: Use Clear, Simple Names

Use names that describe what things are (nouns), not what they do (verbs).

Bad names (confusing):

• /createUser
• /getUserData
• /updateUserEmail
• /deleteUser

Good names (clear):

• /users (to create a user)
• /users/123 (to get user 123)
• /users/123 (to update user 123)
• /users/123 (to delete user 123)

Clear naming pattern: /users for the collection, /users/123 for one specific user.

Part B: Design for Complete Actions

People don’t want to make 10 separate calls. They want to complete one task.

Example: Buying something online

Bad design (10 separate steps):

1. Get shopping cart
2. Check shipping options
3. Calculate tax
4. Validate payment method
5. Apply coupon code
6. Reserve inventory
7. Process payment
8. Create order
9. Send confirmation email
10. Get order details

Takes forever! 10 separate operations!

Good design (one complete action): Send everything at once: Shopping cart, shipping choice, payment info, coupon

Get everything back at once: Order confirmation, tracking number, email sent

One operation, much faster!

Real-Life Example

Think about ordering coffee:

Bad way: “Can I have a cup?” Wait for answer. “Can you put coffee in it?” Wait. “Can you add milk?” Wait. “Can you heat it?” Wait.

Good way: “One hot latte, please.”

Design your systems to let people express complete intentions, not tiny atomic steps.

Strategy 8: Find the Right Size

The Problem: Too Small or Too Big

Too small (too many calls): To get one user’s information:

• Call /user to get basic info
• Call /user/profile to get profile
• Call /user/preferences to get settings
• Call /user/permissions to get access rights
• 5 separate calls for one user!

Too big (too much data): One call to /user-complete returns:

• 50 fields of data
• Tons of stuff you don’t need
• Takes forever to send
• Wastes bandwidth

The Right Balance

Default: Give what most people need

Get user 123:
Returns:
- ID
- Name
- Email
- Account status

Optional: Let people ask for more if they want

Get user 123 with extra information:
Returns everything above, PLUS:
- Full profile
- Settings
- Order history
(Only if requested!)

Real-Life Example

Think about buying a computer:

Too small: Buying individual transistors and assembling from scratch (way too much work!)

Too big: Everything permanently welded together - can’t upgrade or fix anything (inflexible!)

Just right: Logical components (CPU, RAM, storage) that work together but can be upgraded independently.

Strategy 9: Automate Quality Checks

The Old Way: Manual Reviews

Old approach:

1. Team creates a design
2. Wait for meeting with architecture team
3. Get feedback 2 weeks later
4. Make changes
5. Wait for another meeting…

Problems:

• Takes forever
• Inconsistent (depends on who reviews)
• Doesn’t scale (need more reviewers as company grows)
• Expensive (uses senior people’s time for routine checks)

The Better Way: Automated Checks

Set up automated systems that check for common issues instantly:

What can be automated:

• Naming conventions (are names clear and consistent?)
• Security basics (is authentication required?)
• Size limits (are responses too large?)
• Breaking changes (did you break existing features?)
• Documentation (is everything explained?)

When it runs:

1. Developer submits design
2. Automated checks run instantly (seconds)
3. If all checks pass → Auto-approved!
4. If minor issues → Warnings shown
5. If major issues → Must fix before proceeding

Benefits:

Real-Life Example

Think about car manufacturing:

Old way: Inspect every single part by hand. Slow, inconsistent, expensive.

New way: Automated sensors and tests catch 95% of issues instantly. Humans only check special cases.

Strategy 10: Teach and Build Champions

The Challenge

Even with great tools and processes, you need people to actually use them. That requires changing culture.

Four Ways to Build Culture

1. Education Programs

Make learning easy and accessible:

• Monthly workshops on best practices
• Lunch-and-learn sessions
• Training videos
• Style guides and examples
• Decision-making guides

2. Champions Program

Find enthusiastic people in each team:

• They learn best practices first
• They teach their teammates
• They provide local support and advice
• They share feedback with central team
• They adapt global patterns to local needs

3. Success Stories

Share wins to inspire others:

• “How Team X reduced their calls by 70%”
• “Team Y’s smooth migration - lessons learned”
• “How Team Z deployed with zero downtime”

Show metrics:

• Fewer problems reported
• Happier developers
• Faster time to launch
• Better system performance

4. Make It Easy to Do the Right Thing

Provide tools and templates:

• Starter templates (quick start with best practices)
• Automated code generators (less manual work)
• Fake test systems (test before building)
• Interactive documentation (learn by trying)
• Checking tools (verify your work)

Real-Life Example

Think about teaching healthy eating:

Bad way: Just hand out nutrition guidelines and hope people follow them.

Good way:

• Offer cooking classes (education)
• Share success stories of people who got healthier (inspiration)
• Make healthy food easily available (remove barriers)
• Build a supportive community (champions help each other)

Putting It All Together

All 10 strategies work together:

Your Roadmap: Where to Start

Phase 1: Foundation (Months 1-3)

• List all your current systems
• Create a style guide (how things should be named and organized)
• Start designing before building for new projects
• Set up basic automated checks
• Decide on versioning approach

Phase 2: Process (Months 4-6)

• Use design-first for all new systems
• Create a design review process
• Set up fake test systems
• Assign someone to represent users
• Define how systems move from new to old to retired

Phase 3: Scale (Months 7-9)

• Build a central place to find all systems
• Create templates for common patterns
• Launch training program
• Identify and train champions
• Implement portfolio management

Phase 4: Optimize (Ongoing)

• Measure if things are getting better
• Gather feedback from teams
• Improve based on pain points
• Share success stories
• Keep improving

How Mature Are You?

Common Mistakes to Avoid

❌ Mistake 1: Reviews Become Bottlenecks

Problem: Every design needs approval from a committee. Takes 2 weeks. Work piles up. Solution: Automate 80% of checks. Humans only review special cases.

❌ Mistake 2: Over-Complicating the Design Process

Problem: Design phase involves too many people and takes forever. Solution: Set a time limit (1-2 weeks max). Improve based on real use.

❌ Mistake 3: Ignoring Old Systems

Problem: Focus only on new systems. Old systems create messy experiences. Solution: Make a plan to update or retire old systems.

❌ Mistake 4: Forcing Changes Without Buy-In

Problem: Central team orders changes. Teams resist and find workarounds. Solution: Involve teams early. Use champions. Show value first.

❌ Mistake 5: Treating Everything the Same

Problem: Apply heavy process to everything, even small internal tools. Solution: Risk-based approach - more oversight for critical systems, lighter touch for simple ones.

Key Takeaways

1. Think Strategically: Manage all your systems as a portfolio, not isolated projects. Avoid duplication, focus on important ones, plan for retirement.

2. Design for Users: Build what customers need, not what matches your org chart.

3. Serve Both Audiences: Business people and technical people need different information about the same systems.

4. Design First: Plan and get feedback before building. Much cheaper to change a plan than rebuild.

5. Hide Complexity: Don’t make users understand your internal mess. Show them simple, clean operations.

6. Build to Last: Add new features without breaking old ones. Only make breaking changes when absolutely necessary.

7. Think in Complete Actions: Let users accomplish complete tasks, not force them through 10 tiny steps.

8. Find the Right Balance: Not too many calls (annoying) and not one giant call (wasteful). Just right!

9. Automate Quality: Manual reviews don’t scale. Automate routine checks, humans handle special cases.

10. Build Culture: Invest in training, find champions, share wins, and make good practices easy.

Real Success Stories

Amazon: Everything is an API

• Every team provides clean interfaces to their systems
• Teams can work independently
• Result: Built AWS - one of the biggest cloud platforms

Stripe: Make Developers Happy

• Obsessively focus on user experience
• Best documentation in the industry
• Never break existing integrations
• Result: Developers love using Stripe

PayPal: Strategic Portfolio

• Hundreds of systems managed strategically
• Design before building
• Central marketplace to discover systems
• Result: Faster development, less duplication

Netflix: Automated Everything

• Automated quality checks
• Self-service tools for teams
• Strong training culture
• Result: Fast innovation with consistency

Summary

Growing from a small company to a large one isn’t just about handling more users (that was Part 1). It’s about organizing hundreds of people so they can work together effectively without stepping on each other’s toes.

The 10 strategies in this guide help you:

• Keep things organized (portfolio management)
• Focus on what matters to users (customer-centric design)
• Let teams work independently while staying consistent (standards + autonomy)
• Build for the long term (extensibility, not breaking changes)
• Scale your processes (automation, champions, education)

Remember the pattern:

1. Start with strategy (portfolio thinking, customer focus)
2. Apply design principles (design-first, hide complexity, right-size)
3. Automate what you can (quality checks, tools)
4. Build culture (education, champions, success stories)

Whether you’re managing 10 systems or 1,000, these patterns will help you grow smoothly and sustainably.

Continue Learning

• ← Back to Part 1: Performance & Infrastructure - Learn about handling more traffic and users
• Related Topics:
  • Breaking big systems into smaller pieces
  • Security best practices
  • Different types of system designs

Questions? These strategies work at companies like PayPal, Amazon, and Netflix, but every company is unique. Adapt them to your situation.

Good luck building great systems! 🚀