Cost and Architectural Economics

The architecture works. The economics do not.

Consider a multi-tenant analytics product. Customers send events, build dashboards, and query their history. A plausible first design accepts each event into a durable log, writes it to a fast analytical store, indexes every field a customer might filter, retains the raw event for a year, replicates the data to a second region, and exports another copy to a warehouse. Every dashboard reads detailed data so any query remains possible.

Nothing in that paragraph is obviously absurd. Each choice buys something: durability, query freedom, low latency, recovery, or analysis. Together, however, they may create a product whose cost grows faster than its value.

Suppose the service receives an average of 100,000 one-kilobyte events per second. Those are modeled numbers, not production measurements. Even so, they make the shape visible:

raw ingest per day
  = 100,000 events/second × 1 KB/event × 86,400 seconds
  = 8.64 TB/day

30 days of raw events
  = 259.2 TB

That is before replicas, indexes, materialized views, backups, warehouse exports, telemetry, or storage-engine overhead. The exact provider price is not the important discovery. The multiplier is.

Cost belongs in the system design at this moment, while the architecture can still change. Adding “use cheaper storage” after drawing every copy would miss the design question: which representations must exist, where, for how long, and for whose benefit?

Choose a unit that can carry a decision

The business sells analytics to tenants, so begin with the cost of serving one tenant for one month. That unit is more useful than “our database bill” because it can inform a price tier, quota, retention policy, noisy-neighbor control, or product limit.

A tenant-month is still too coarse for diagnosis. Decompose it into the work that tenant causes:

tenant-month cost
  = ingest and validation
  + retained raw and derived data
  + dashboard and ad hoc query work
  + export and network movement
  + allocated baseline capacity
  + vendor charges
  + operational and support burden

This is a model, not an invoice. Its job is to reveal why two tenants with the same seat count may have radically different economics. One sends small, regular events and reads a few repeated dashboards. Another sends large high-cardinality payloads, scans a year of history, refreshes custom dashboards every few seconds, and exports raw data across regions. Averaging them together hides the tenant that determines capacity and support load.

The subunits help with attribution. Cost per million accepted events exposes ingest and retention. Cost per dashboard refresh exposes scan volume, fan-out, cache reuse, and concurrency. Cost per export exposes bytes read and moved. The product unit remains the tenant-month; the subunits explain it.

In an interview, the model can be stated without pretending to know a current price sheet:

I would model one tenant-month, then attribute it by accepted event, retained
gigabyte, dashboard query, and export. The likely multipliers are payload
volume times retention, the number of derived copies, and the amount of data
scanned per query. That tells us where an architectural change could matter.

Follow one event until the copies become visible

Take one accepted event through the first design:

client
  -> ingestion service
  -> durable log
  -> hot analytical store
  -> indexes and materialized views
  -> warehouse export
  -> regional replica and backup
  -> the platform's own logs, metrics, and traces

Every arrow has more than a storage cost. It may consume serialization and compression work, network bandwidth, vendor-metered operations, encryption and privacy review, schema compatibility, replay logic, monitoring, and someone’s attention during failure. A raw event copied into six systems also creates six retention and deletion obligations.

The right response is not to prohibit copies. The durable log may be necessary for recovery. A regional replica may be justified by the recovery objective or data-residency rules. A materialized view may turn a common dashboard from an expensive scan into a predictable lookup. The question for each copy is what promise it serves.

In this product, most customers repeatedly read recent dashboards. They rarely scan old raw events, but they do expect the common charts to be fast and accurate. That observation suggests a different shape:

accepted event
  -> durable, compressed raw log
  -> streaming rollups for common dimensions
  -> hot recent aggregates for dashboards
  -> colder partitioned detail for investigation and replay

explicit side paths
  -> bounded customer export
  -> required backup or regional copy
  -> sampled, compact operational telemetry

Now the hot store holds what the interactive product repeatedly reads rather than every representation the system can produce. Raw history remains available, but an old, wide query may become asynchronous. Fields are indexed because a supported query needs them, not because they arrived in the payload. Long retention, high-resolution data, and large exports can become explicit plan features instead of invisible subsidies.

This change spends compute at ingestion to create rollups and spends some storage on derived aggregates. In return, it reduces repeated raw scans and makes dashboard latency and query cost more predictable. Precomputation is not inherently cheaper; it wins here because the same views are read many times. For a product dominated by one-off investigation, the balance could reverse.

Preserve the promises that justify the bill

Cost reduction becomes false economy when it erases the reason the system exists. The analytics product might permit approximate exploratory charts while requiring exact billing totals. It might serve older investigations slowly while promising that recent operational dashboards are fresh. Those different promises deserve different storage, computation, and validation.

The design can therefore classify work by consequence:

• Billing aggregates stay exact, auditable, and reproducible from durable inputs.
• Common recent dashboards receive predictable low-latency service.
• Exploratory queries may use sampling when the interface says so.
• Long-range raw scans run asynchronously with tenant budgets.
• Deletion and residency rules follow the data into every retained copy.

This is disciplined spending, not indiscriminate thrift. Removing a replica that the recovery objective depends on is not an optimization. Neither is weakening audit history, silently sampling a money calculation, or allowing one customer’s query to exhaust capacity shared by everyone else.

Cost controls often improve other properties when they are attached to a real limit. An event-size bound constrains storage and protects ingestion. A query budget limits spend and contains noisy neighbors. Expiring unused indexes reduces write amplification and privacy exposure. Suppressing duplicate work through idempotency saves provider calls and improves correctness. But each control still needs product judgment; a blunt quota can make a paid feature unreliable just as easily as it can protect the service.

Design for the shape of demand

The average ingest rate does not describe the capacity problem. Assume the 100,000-event-per-second baseline rises fivefold at the top of each hour when customers flush batches. Dashboard traffic also surges at the start of the business day, while compaction and customer exports can move within a deadline.

Buying fixed capacity for the full combined peak leaves expensive resources idle. Depending entirely on instant autoscaling assumes workers, databases, and vendors can all expand before queues and latency become harmful. A more credible design gives each kind of demand a policy.

Reserve or commit only the stable base when the workload and architecture are predictable enough to use it. Absorb short ingest bursts in a durable queue, then scale consumers against queue age rather than queue length alone. Isolate exports and compaction from interactive queries, give them deadlines and pause controls, and keep warm capacity where startup time would violate the latency promise. Apply tenant budgets before a single burst becomes the shared system’s capacity plan.

The dependencies matter. Adding consumers does not help if the analytical store, object store, or enrichment vendor is already at its safe concurrency limit. A queue converts a request spike into backlog; it does not abolish the work. The design must show the rate at which backlog can drain and the age at which the product promise fails.

A concise explanation might be:

I would reserve the measured baseline, queue the short ingest burst, and
autoscale consumers against backlog age within the downstream write limit.
Exports and compaction use separate pools because they can yield to dashboard
traffic. I would not reserve the modeled five-times peak until measurements
show that it is sustained and predictable.

Count the people and contracts in the architecture

The visible infrastructure bill is only part of total cost of ownership. The system also needs upgrades, incident response, capacity planning, security review, privacy controls, migrations, customer support, and engineers who understand its failure modes.

Suppose a managed analytical database charges by stored data and query work. A self-managed engine appears cheaper under a simple compute-and-disk comparison. The comparison is incomplete until it includes:

• the work to make ingestion, backups, restoration, upgrades, and failover production-safe;
• on-call load and the specialist knowledge needed to diagnose corruption, skew, compaction stalls, and bad query plans;
• integration with tenant isolation, authorization, audit, observability, billing, residency, and deletion;
• the vendor’s metered units, minimum commitments, reliability boundaries, export path, and behavior at the product’s projected workload;
• the opportunity cost of delaying product work, and the migration cost if the first choice stops fitting.

The managed service may still be expensive. The custom system may still be right. A capability central to the product’s differentiation, or a requirement the market cannot meet, can justify ownership. A commodity engine that a small team would struggle to operate can justify a visible vendor premium.

There is also a useful middle ground. The product can use managed storage and execution while retaining its own ingestion contract, tenant budgets, rollup definitions, idempotency, cost attribution, fallback behavior, and portable raw data. Buying an engine does not mean outsourcing the product’s correctness or economic controls.

Licensing deserves the same architectural attention. Per-event, per-query, per-gigabyte, per-host, and per-seat contracts reward different designs. If one user action fans out into ten metered vendor operations, that amplification belongs in the cost model. So do commitment terms and the time and bandwidth required to export retained data. “Lock-in” is too vague unless the answer names what would be difficult to move: data, query semantics, operational skills, integrations, or a contract.

Make one economic choice visible in the interview

Cost should appear when an estimate first reveals a multiplier and when two architectures spend differently. It need not become a tour of every possible saving.

For the analytics design, a complete answer could sound like this:

The product unit is a tenant-month. I would attribute it by accepted events,
retained bytes, dashboard query work, and exports. At the modeled ingest rate,
30 days of raw input is already about 259 TB before replicas and indexes, so
the expensive path is data retention, derived copies, and repeated query
scans—not the stateless API layer.

I would keep compressed raw events in one durable replay path, build exact
rollups for common dashboards, keep recent aggregates hot, and move older
detail to partitioned colder storage. Long raw scans and exports become
bounded asynchronous work. Required recovery and residency copies remain
explicit; fields are indexed only for supported queries.

I would reserve the stable baseline, queue the top-of-hour ingest burst, and
isolate exports and compaction from interactive queries. The build-versus-buy
decision includes metering, staffing, operations, compliance, and data exit,
not just machine cost. I would preserve exact billing, deletion, and freshness
promises even where a cheaper approximation is available.

This gives the interviewer several productive directions. They can make one tenant dominate query load, require low-latency global dashboards, impose a residency boundary, remove the durable raw log, change the workload from repeated dashboards to ad hoc investigation, or reveal a vendor charge on every scanned byte. Each variation changes the model rather than inviting a memorized list of optimizations.

Practice the multiplier

Take a design you already know and choose one unit the product or user feels: an uploaded minute, a delivered message, a search query, a workflow execution, or a tenant-month. Write the rough equation for that unit. Include retained data, fan-out, retries, derived copies, vendor operations, and human ownership where they genuinely apply.

Then find the first architectural choice that changes the dominant multiplier. Do not begin with a discount. Change retention, placement, precomputation, batching, isolation, or the amount of work the product promises. State the latency, reliability, correctness, privacy, or audit boundary the change must preserve.

For a harder variation, make demand bursty and make one dependency impossible to scale quickly. Decide what is reserved, queued, autoscaled, throttled, isolated, or deliberately overprovisioned. Finally, replace one component with a managed service and explain which costs disappear, which merely move into a contract, and which responsibilities remain yours.

The most useful recovery sentence is short:

Let me identify the product unit, find the multiplier, and compare the
architectural levers against the promise we cannot break.

That sentence returns a vague cost discussion to the design. Once the economics are credible, the next question is whether the system can reach that architecture without a dangerous cutover. Evolution and Migration takes up that transition.