Production Data Systems Handbook / Chapter 20
Graph Systems
Use graph systems when traversal is the workload, and evaluate fan-out, depth, provenance, temporal validity, freshness, and operational risk.
Preparing audio…
Audio edition
Graph Systems
The Account That Connected to Everyone
A risk analyst begins with an account flagged after a disputed payment. The transactional database answers the first questions cleanly: which cards the account used, which devices and network addresses appeared at login, which addresses received orders. Then the investigation changes shape. The analyst needs every account connected through those identifiers during the last fourteen days, including paths that pass through another account, while excluding common identifiers that join thousands of unrelated people.
One more join can answer one more known path. It cannot conveniently discover which path will matter, how far the useful neighborhood extends, or which arrangement of weak observations forms a suspicious pattern. This is where a graph system may earn its place. The reason is not that accounts, cards, and devices have relationships. The reason is that following and judging those relationships has become the workload.
That distinction also provides the rejection test. If the important questions have fixed depth, stable shapes, and bounded results, ordinary tables, recursive queries, closure tables, materialized paths, or an analytical job may remain simpler. A graph store deserves another backup, access-control surface, migration plan, and on-call burden only when traversal pressure outweighs them.
Write the Traversal Before Choosing the Store
“Fraud is a graph problem” is a theme, not a workload. The investigation becomes designable when it is written as a query:
Starting from one flagged account, find accounts reachable within three steps through recent device, network, card, address, referral, recovery-email, or support-contact observations. Suppress or discount common identifiers, return the strongest paths first, and preserve the evidence for every connection.
That sentence makes several choices visible. The start is an account rather than an unrestricted search. Depth is capped. Edge types and a time window constrain expansion. Common identifiers need different treatment. The result is ranked paths with evidence, not an unexplained cluster.
The same discipline sharpens other graph-shaped work. A deployment tool might ask for every downstream service and dataset reachable from one changed schema. A permissions service might need the paths through which a user inherits access. A recommender might explore user-item neighborhoods, although offline graph computation or precomputed features may serve the product better than an online traversal. A knowledge graph may exist chiefly to connect governed facts across vocabularies rather than to answer low-latency path queries.
For any of these, name the starting nodes, permitted directions and edge types, expected and worst-case fan-out, useful depth, filters, freshness, result semantics, latency, and concurrency. “Find dependencies” is still too vague if one engineer means an offline inventory and another means an authorization decision on every request.
An Edge Is a Claim With a History
The fraud graph can represent accounts, devices, network addresses, cards, shipping addresses, emails, referral codes, and support cases as nodes. Its edges should describe observations precisely: USED_DEVICE, OBSERVED_AT_IP, USED_CARD, SHIPPED_TO, REFERRED, RECOVERED_WITH, or CONTACTED_FROM. RELATED_TO only hides the question the system will later be asked to answer.
In a property graph, nodes and edges can carry labels and properties. An OBSERVED_AT_IP edge might carry a source event identifier, first-seen and last-seen times, the time it was ingested, and a confidence or status value. Those properties let the traversal select recent evidence, distinguish verified from inferred connections, and return a proof path an analyst can inspect.
RDF expresses facts as subject-predicate-object triples and is often useful when shared vocabularies, interoperability, ontologies, and inference dominate the design. Property-graph and RDF models overlap in capability, but they tend to expose different centers of work. The former often brings edge properties, traversal performance, and query safety to the foreground; the latter often makes predicate meaning and semantic governance impossible to postpone. Neither removes the need to define identity, ownership, retention, and deletion.
Time needs particular care. “This account used this device” may mean the event occurred on Tuesday, while the graph learned about it on Thursday after a delayed import. One timestamp records the event; another records when the system observed it. For relationships that persist, a validity interval says when the claim was true. Investigations may need all three. An edge can also expire, be disputed, or be withdrawn when its source is deleted. Treating every edge as timeless truth turns old observations into current accusations.
Provenance is therefore part of the data model, not decoration for an audit screen. Operators need to trace an edge to a source table, event, scan, manual assertion, model inference, or third-party feed. That link supports correction, appeal, source deletion, graph rebuild, and an honest answer to the simplest investigative question: why did these accounts appear together?
Make the Relational Design Compete
The graph proposal should have to beat the best ordinary design, not a caricature of joins. An org chart that answers “who manages this employee?” and “who reports to this manager?” fits a parent reference and an index. A product taxonomy may fit a parent-child table or a materialized path. A small service map may fit adjacency rows plus a cached closure. A graph-shaped dashboard refreshed hourly for analysts may belong in the warehouse.
The answer changes when path shapes become variable and repeated. Matrix reporting, delegated authority, historical assignments, and policy inheritance can make the org chart a traversal workload. A dependency model that spans services, jobs, queues, tables, dashboards, owners, deploys, and incidents may outgrow a collection of special-purpose joins. If application code keeps inventing traversal, precomputed paths drift from their sources, and each new relationship requires another bespoke query, the relational system may already be paying graph costs without graph tooling.
Compare the complete systems: source ownership, update path, query latency, bulk loading, backup and restore, rebuild, authorization, migration, cost, and the team’s ability to diagnose a bad result. Familiar storage is not automatically cheaper, and expressive traversal is not automatically worth operating.
Follow the Query Into a Supernode
Return to the flagged account. Its first steps reach a card, two devices, a shipping address, and a network address. One device leads to three other accounts. The network address is a public VPN exit and leads to thousands. Without a degree check, the second step can dominate the query, produce a dramatic but meaningless cluster, and consume the shared service while doing so.
That high-degree connector is a supernode. Common addresses, public IP ranges, global roles, root organizations, popular products, central services, widely used packages, and generic “unknown” nodes create the same problem. Some are useful facts, but they are rarely useful at the same weight or expansion policy as a rare card or recovery email.
The query needs policy as well as syntax. It may exclude known public networks from expansion, segment them, lower their evidence weight, cap neighbors per edge type, or show that a path was truncated. Depth caps, fan-out limits, timeouts, result limits, edge-type allowlists, and cost estimates are not restrictions pasted on after launch. They define what the graph is allowed to mean under production load.
Partitioning makes this more than a local query concern. Graph neighborhoods do not politely end at storage boundaries. A traversal can cross partitions at each step, and a supernode can scatter work across many of them. The design should test representative topology—including high-degree nodes and concurrent investigations—rather than only small, evenly distributed samples. Analyst exploration can run in a more isolated lane; an online product or authorization path needs tighter bounds and predictable failure behavior.
Authorization must travel with the traversal. Filtering only the returned nodes can still expose restricted intermediate relationships, counts, or path shapes. Access checks may be needed before expansion, on edges and intermediate nodes, and again on results. A permissions graph raises the stakes further: stale or unexplained edges can grant or deny access, so update and delete targets must match the risk of the decision.
Decide Where Authority Lives
The transactional records in the fraud example should remain authoritative unless the graph is deliberately designed as a primary store. A derived graph then needs more than an ingestion job. It needs source links, a freshness target, idempotent backfills, late-event handling, delete propagation, drift detection, and a rebuild procedure that can produce a new version while the old one still serves.
Bulk loading is part of launch design because a first load is only the smallest rebuild. Teams should know how long a full reconstruction takes, how changes arriving during it are caught up, how old and new versions are compared, and how traffic returns to the previous version when entity resolution or edge construction is wrong. If the graph is authoritative, it inherits the full obligations of primary data: invariants, transaction requirements, backups, restore tests, migrations, access control, and incident ownership.
Observability should describe graph behavior rather than generic host health alone. Track traversal depth and fan-out by edge type, high-degree nodes, slow query shapes, timeouts, truncated results, edge age, ingestion lag, rebuild duration, and the callers responsible for expensive work. Storage grows with edges and their properties; query cost grows with expansion; governance cost grows with every source and relationship meaning. A healthy process and a fast disk cannot reveal that the graph is returning stale or poorly sourced paths.
Vocabulary also needs an owner. Someone must decide who can introduce an edge type, what it means, whether its direction is significant, how it changes, and which consumers depend on it. A shared graph assembled from many systems is not a neutral integration layer. It concentrates ambiguity unless ownership becomes clearer at the same time.
Record the Graph-Fit Decision
Before the store becomes a product dependency, write one review record:
User or operator decision:
Traversal query in one sentence:
Starting nodes and request rate:
Allowed edge types and directions:
Depth and expected/peak/worst fan-out:
Filters, ranking, proof path, and result limit:
Freshness, expiry, and deletion targets:
Authoritative sources and edge provenance:
Valid time, observed time, and confidence rules:
Supernodes and expansion policy:
Relational, analytical, search, or precomputed alternative:
Online and exploratory isolation:
Timeout, truncation, and authorization behavior:
Bulk load, rebuild, comparison, and rollback path:
Monitoring, cost, incident, and vocabulary owners:
Complete it for the fraud investigation. Then change one assumption: a shared network address is revealed to be a public VPN exit. Trace how its degree changes query cost, evidence weight, the analyst’s explanation, and any result already used in a decision. This is the moment a graph design becomes credible. It can admit that a technically valid path may still be weak evidence.
Now apply the same record to an org chart, a recommendation engine, and a permissions graph. The org chart may remain relational until historical and delegated relationships become central. The recommender may need offline graph computation more than online graph serving. The permissions graph may justify traversal while demanding the strictest freshness, proof, and deletion behavior.
A graph system has earned its place when the review produces concrete traversal pressure, explicit edge meaning, and bounded production behavior. If it produces only relationship vocabulary, keep the simpler store. The lesson carries into the next class of specialized systems as well: an access path is valuable only when the team can explain the facts behind its answers and operate the transformations between source and result.
Continue reading
Full table of contents