Security Principles Engineers Can Apply

An organization owner asks for a Delete organization button. The first design sounds straightforward: authenticate the owner, enqueue a job, delete the tenant’s records, and log success. Each step is plausible. Together they could let a stolen session erase a company, let a confused worker cross a tenant boundary, or leave the product claiming deletion while copies remain.

Security principles help only when they change this design. They are not ten slogans to paste into a review. They are different questions pressed against the same path: who may begin it, where authority is checked, how much power the worker receives, what happens when a dependency fails, what evidence remains, and whether the system can recover before destruction becomes final.

Start at the irreversible act

Put the final purge behind a narrow deletion service. The product API may request deletion; it may not delete tenant data directly. The deletion service accepts an immutable organization identifier and a signed request reference, verifies that the request is still authorized, places the organization in a recoverable quarantine period, and later coordinates deletion across registered data stores.

That boundary makes least privilege concrete. The product API can create and inspect a deletion request, but cannot purge storage. The deletion worker can delete records for the organization named in one approved request, but cannot enumerate every tenant or grant itself new permissions. Its database role cannot migrate schemas; its object-store role is limited to tenant prefixes; its credentials are short-lived.

The implementation rule is simple to state and hard to fake: every capability is scoped by action, resource, and duration. The review question is, “Which permission would this component lose if we narrowed it to the deletion it is performing?” The answer should be visible in IAM policy, database grants, denied-action tests, and the runtime identity inventory—not inferred from the component’s name.

Bad: give the existing background-worker role database administration and object-store delete access because it already runs trusted jobs.

Better: give a dedicated worker a request-bound identity and narrowly scoped deletion capabilities, then test that it cannot read another tenant, alter schema, or invoke unrelated administration APIs.

Check authority where the damage occurs

The owner role shown in a user interface does not authorize a later purge. Sessions expire. Ownership changes. Accounts are suspended. Queued messages are replayed. Complete mediation requires the system to decide at every protected transition: when the request is created, when a second approver confirms it if policy requires one, when quarantine begins, and immediately before each store is purged.

Place those checks at the service that changes state, using the canonical organization identifier from the approved request rather than a caller-supplied tenant field. Give cached policy an explicit maximum age and invalidation path. Make request identifiers idempotent so a retry resumes the same deletion instead of creating a second one.

The review question is, “If the organization ID, owner, request state, or queued message changes, which check refuses the operation?” Verification needs object-level authorization tests, stale-policy tests, replay tests, and samples of denied transitions in production. A gateway check alone is not evidence that the worker mediates its own authority.

Make the dangerous path absent by default

Secure by default asks what happens before a team remembers to configure anything. A new service template should expose no destructive administration route, grant no cross-tenant worker role, accept no default credential, and emit the security events required for privileged actions. A newly registered data store should be treated as deletion-incomplete until it implements the deletion contract and passes a drill.

The implementation belongs in generated templates, infrastructure modules, policy libraries, and service-registration gates. The review question is, “What destructive capability appears when a team takes every default?” Inspect a freshly generated project and a newly provisioned environment; do not prove the default by showing a mature service that engineers hardened afterward.

This principle also reduces attack surface. One deletion API and one registered worker path are easier to defend than separate purge scripts, support-console buttons, database procedures, and cloud-storage commands. Inventory every route to the same effect, remove the redundant ones, and narrow what remains by caller, organization, operation, and environment. Repeat that inventory after migrations and major feature changes. A forgotten support script is part of the attack surface even when it has no public endpoint.

Require independent reasons for the action to succeed

For a high-impact deletion, separation of duties may require two distinct decisions: an organization owner initiates the request and another authorized owner confirms it, or a support-assisted request requires customer authorization plus an internal approver. The person who implements a temporary bypass cannot accept its risk. A production administrator using break-glass access cannot approve their own access afterward.

Separation is useful only when the second actor sees the organization, scope, consequence, and evidence and can stop the action. Two clicks by people sharing the same compromised session, credential, or incentive are ceremony, not separation. The implementation must bind both decisions to the same immutable request and record distinct identities and timestamps. Review who can occupy both roles; in operation, sample approvals and alert when role combinations or unusual timing collapse the intended independence.

Now apply defense in depth, but demand that the layers fail differently. Strong reauthentication helps prevent a stolen passive session from initiating deletion. A second approval resists one compromised owner. A quarantine delay contains mistakes before purge. Immutable audit events support detection and investigation. Rate limits constrain automation. Tested restoration supports recovery during the quarantine window.

Five checks in the same API process are not five layers. Ask, “Which single compromise or mistaken assumption defeats all of these controls?” Exercise the answer: revoke an owner after initiation, replay a job, compromise a worker credential in a test environment, and attempt to bypass the quarantine. The observed failures show whether the layers are independent.

Decide failure behavior before the dependencies fail

The deletion path depends on identity, policy, queues, key management, storage, audit delivery, and the registry of systems that hold tenant data. Fail safe means each dependency failure has a deliberate state transition that preserves the security property at risk.

If authorization or approval state cannot be read, do not begin or advance deletion. If a key cannot be obtained, do not substitute an unmanaged key or silently skip encrypted storage. If one registered store is unavailable, retain the request in a retryable, visibly incomplete state; do not report success. If the central log sink is unavailable, append the critical event to a durable restricted buffer and halt before final purge if the action cannot be audited safely.

“Fail closed” is not a universal synonym for stopping the whole product. Blocking customer reads because the audit exporter is slow may create a needless outage. The design must name the protected operation, the unavailable control, and the safe degraded behavior. Reviewers should ask, “What state remains after this dependency times out, and can retrying duplicate or bypass the decision?” Fault-injection tests and incomplete-deletion alerts should prove the answer.

Design as though one foothold is already lost

Assume breach begins after a control has failed. Suppose an attacker has one owner’s session, one worker token, or code execution in one workload. The architecture should still put another boundary in the way: reauthentication and independent approval for the session, request-bound permissions for the worker, tenant and network segmentation around the workload, and rapid credential revocation everywhere.

The review question is, “From this foothold, what can the attacker reach next without another decision?” Trace the path rather than answering “the network is segmented.” In operation, test credential rotation, inspect attempted cross-tenant and cross-service access, and practice containment of the deletion worker without disabling unrelated customer traffic.

Assuming breach changes observability from general logging into evidence about that path. Make security observable by emitting an event for every deletion transition: actor and delegated identity, exact action, organization, request state, decision and reason, approval references, source, correlation identifier, and policy version. Protect the event stream’s integrity and retention. Exclude tokens, secrets, unrestricted personal data, and deleted payloads; telemetry must not become another copy of the tenant.

The useful review question is, “Can an investigator reconstruct who attempted what, under which authority, and what the system decided?” Prove it by tracing a sampled request across services, reconciling initiated requests with quarantines and purges, and alerting on repeated denials, approval anomalies, stalled stores, and audit gaps. A log line that says deletion failed proves almost nothing.

Recover before finality, verify after it

Deletion cannot be both instantly irreversible and recoverable. Design for recovery forces that conflict into policy. Before final purge, use a quarantine period appropriate to the data and threat: block normal access, preserve a tightly controlled restoration path, tell the customer when deletion becomes final, and define who may cancel it. After final purge, recovery means restoring trusted service configuration and proving deletion coverage—not resurrecting data the organization was promised would be erased.

The implementation needs a deletion ledger, a registry of covered stores, tested restore permissions for the quarantine state, and explicit completion criteria for final purge. Backups must follow the declared deletion and retention policy; an indefinite recoverable copy would defeat the action’s purpose. Reviewers should ask, “At each stage, what can be restored, by whom, until when, and how do we know the final state is trustworthy?”

Run both sides of the drill. Restore a quarantined organization without crossing tenant boundaries, then complete a modeled deletion and reconcile the ledger against every registered store, search index, cache, export, and backup policy. Record unresolved copies as failures, not as successful deletion with footnotes.

Principle card: from advice to proof

The principles now describe one system rather than ten independent aspirations. Use this reference to carry the same move into another design. The architecture pattern is a starting point, not automatic proof; the review question and operating evidence test whether it is real.

When a principle produces only a control name, ask again. A useful answer changes the architecture, constrains the implementation, gives the reviewer a falsifiable question, and leaves an operating signal. Chapter 3 turns those answers into the minimum bar a production system must clear.