Module 01 · Lesson 8 of 14Free

Design tradeoffs: making and explaining decisions

Level

Intermediate

Topic

System design fundamentals

Reading time

18 min

The festival sale begins on Kartly. Traffic reaches 1,000 requests a second, and product descriptions stop loading. One engineer proposes saving recent results in memory; another wants a larger database.

Both changes could help. How do you choose a design when improving one property creates a cost somewhere else?

For this lesson’s scenario, a test database handles 600 description reads a second within the agreed latency target. Checkout work already has its own reserved allowance. These are illustrative test results, not PostgreSQL limits.

A benefit needs a constraint

A tradeoff is a choice that improves something you value at the expense of something else you value. Here, fewer database reads may cost freshness, memory and recovery work.

But not every improvement requires accepting a loss. Suppose a query accidentally reads unused columns. Removing them might reduce work without changing the result. Investigate such improvements before accepting a harder compromise.

Start with the product questions from Functional and non-functional requirements. For this scenario, the team agrees:

The distinctions between freshness, availability and durability come from Core system properties. Here, your task is to decide which property matters for this particular operation.

A description that briefly shows an older material specification is different from charging an older price. The team must approve each behavior separately. “Some stale data is acceptable” is too broad to guide an implementation.

Google’s SRE guidance makes a related point: reliability investment must reflect business needs and the costs of achieving it. The target is not maximum reliability at any price.^{1Embracing RiskMarc Alvidrez, Site Reliability Engineering, O’Reilly Media, 2016}

These constraints narrow the choices. They do not yet tell you which feasible design costs least.

Compare mechanisms, not product names

The simplest implementation reads a description from PostgreSQL for every request. Each result comes from that request’s database read. There is no second copy for the app to manage.

Under our assumed workload, each product-page request needs one description read. The database therefore receives the same rate as incoming product-page requests.

One alternative is a local cache: a bounded collection of recent description results in the app server’s memory. Another is a shared cache: a separate service holding those results.

Both can avoid repeated reads, but their operational consequences differ. Local memory disappears with the app. A shared cache adds another service to operate. AWS describes both benefits and limitations of these approaches, including empty caches after app starts.^{2Caching challenges and strategiesJas Chhabra, Amazon Builders’ Library, 2019}

The app owns the local cache. Each entry stores a product ID, description text and an expiration time. On a usable entry, the app returns the text. Otherwise, it reads PostgreSQL and saves the completed result.

The app checks expiration on every lookup and caps total memory use. Under this scenario’s policy, an entry stops being reusable after 30 seconds. That is a reuse rule, not proof that the text reflects every seller update within that interval.

We leave implementation details to Caching patterns. The important decision here is accepting a second copy of descriptions to reduce repeated database work.

Now you need evidence that this cost buys enough capacity.

Calculate the margin you need

Use the workload estimates from Capacity estimation, then expose the assumptions that control the result.

Assume a representative sale test finds that 90% of description requests reuse local entries. A cache hit means a request uses a saved result without reading PostgreSQL.

These calculations assume one database read per miss. They exclude retries and extra database operations. Record those assumptions beside the estimate.

Notice what the calculation does not establish. It does not prove a page latency target or predict every short burst. It only estimates description-read demand for a given hit rate.

Avoid collapsing this into “the cache makes the system ten times faster.” In the assumed warm workload, it cuts description reads by a factor of ten. Other request work still exists.

You can compare plausible conditions without inventing a precise forecast:

A good decision should survive the plausible range, or state how it behaves outside that range. The empty-cache row is therefore part of the decision, not an exception to discuss later.

Failure changes the preferred option

Suppose the app restarts during the sale. Its local entries disappear. Every request initially needs PostgreSQL.

AWS warns about this transition: a system that works with a warm cache can overwhelm its dependencies when cached results disappear. Testing without cached results exposes that dependency.^{2Caching challenges and strategiesJas Chhabra, Amazon Builders’ Library, 2019}

In Kartly’s hypothetical restart, shoppers see descriptions stall. Some retry. If the app accepts all that work, it has already broken the requirement to stay within browsing’s database allowance.

The tempting response is to wait until database capacity becomes free. Waiting is useful for short disturbances. Sustained excess demand is different.

A queue stores work until it can proceed. In this model, any finite queue eventually fills if arrivals keep exceeding departures. A larger queue only postpones that moment.

For this scenario, make the admission limiter app-local because there is one app server. Reject excess requests immediately rather than adding a waiting queue. Choose the in-flight bound from representative database tests.

A shopper then sees a clear “Description temporarily unavailable” message instead of an indefinite spinner. Monitor rejected requests, active database calls and database latency. Recover by admitting new work as capacity becomes free; do not retain rejected work for later replay.

This protects a resource boundary, not a guarantee that checkout always succeeds. Shared hardware and other database work still require validation.

The decision is now explicit about who pays: some browsing requests fail so checkout retains its reserved capacity.

Make uncertainty visible in the record

An architecture decision record, or ADR, preserves the context, alternatives and consequences of an important choice. Microsoft recommends recording confidence and superseding an accepted record rather than rewriting its history.^{3Maintain an architecture decision record (ADR)Microsoft, Microsoft Learn, Azure Well-Architected Framework, 2026}

For Kartly, a useful record could fit on one screen:

The 50% trigger is a scenario policy, not a mathematical limit. It starts a review before reaching the calculated boundary. It does not automatically justify a shared cache.

A low hit rate might mean shoppers request more distinct products. A larger cache will not necessarily help. First determine whether entries disappear too early, memory fills, or results simply do not repeat.

The record should also name evidence that would overturn the decision. For example: an improved direct query meets peak demand, or product policy no longer permits reused descriptions.

Keeping accepted records and linking replacement decisions lets future engineers distinguish a mistaken assumption from a changed requirement.^{3Maintain an architecture decision record (ADR)Microsoft, Microsoft Learn, Azure Well-Architected Framework, 2026}

Your explanation should therefore end with a condition, not a verdict: “We choose this because these constraints hold; we will reconsider when these measurements change.”

Common mistakes

Treating requirements as weighted preferences. A high performance score cannot compensate for using an unapproved price. Remove infeasible options before ranking them.

Calling the newest design the best. A shared cache may solve a future coordination problem. That does not establish a benefit for today’s single app server.

Counting only infrastructure cost. Include maintenance, recovery work and engineering time. Reliability improvements consume both machines and opportunities to build other features.^{1Embracing RiskMarc Alvidrez, Site Reliability Engineering, O’Reilly Media, 2016}

Describing only normal operation. “Most requests hit memory” does not explain a restart. State what happens when the assumed benefit disappears.

Changing the priority without saying so. Serving expired descriptions trades freshness for availability. It requires a product decision, not merely a fallback branch.

Where this shows up

• Design: a fast product catalog page applies these questions to cache placement and freshness.
• Design: an e-commerce checkout separates acceptable browsing compromises from payment and order requirements.

Kartly’s next question is whether to add capacity to its current machine or distribute work across machines, explored in Vertical and horizontal scaling.

Check your understanding

Summary