Neo4j & Cypher¶

03:20 AM. The risk API's p99 latency alarm fires. One Cypher query — MATCH (u:User {user_id: $id})-[:USED]->(d:Device)<-[:USED]-(u2:User) RETURN u2 LIMIT 20 — is pinning a CPU core and stalling other Bolt sessions. PROFILE shows the first operator is AllNodesScan.

What's missing from this query?

A. An index on User.user_id so the planner can seek instead of scan every node. B. A LIMIT clause — there already is one, so that isn't it. C. A bound on traversal depth — the pattern is effectively unbounded. D. A read replica to spread Bolt sessions across.

Pick one before reading on. The fraud graph is modelled; what's needed now is bounded traversals on the payment path and investigations in a language that looks like the picture. Neo4j is the operational graph store in this academy — Cypher patterns, indexes to find the start, PROFILE to see expands. It is not where you run last week's GMV.

Start with the situation¶

• Risk API: 2-hop shared device/IP for this user_id, < 50 ms, LIMIT.
• Analyst: 3-hop neighbourhood of a bad IP, minutes OK, still bounded.
• CDC upserts from the payment Postgres.

Recommendations use Cypher only to read precomputed ALSO_BOUGHT or to debug a neighbourhood — not to compute similarity online.

Why the obvious approach breaks¶

Cypher is easy to write and easy to make cartesian. Indexes are easy to forget (label scan of all :Transaction). Variable-length paths are easy to unbounded-star. The database will try. The cluster will pause.

You must treat Cypher like CQL: every query has a start key, a type list, a depth cap, and a LIMIT.

Build the mental picture¶

ASCII art is the language:

(node:Label {prop: value})-[:TYPE]->(other)

MATCH is declare a pattern. WHERE filters. RETURN projects. The planner’s job is to pick a start (index) and expand. If it starts with AllNodesScan, you lost.

Under the hood¶

graph TD
    C["Cypher"]
    P["Planner / cost"]
    I["Index seek on label+prop"]
    E["Expand by type+direction"]
    F["Filter / hash distinct"]
    C --> P --> I --> E --> F

Storage: nodes, relationships, property store, label/type tokens. Index-free adjacency after the start: expand walks relationship chains on the node.

Indexes and constraints are how MATCH (u:User {user_id: $id}) is O(log n) not O(|Users|).

Transactions: Bolt session writes are transactional; CDC should batch modestly. Do not open a transaction that upserts 10 million edges.

Memory: transaction heap + page cache. A supernode expand pulls a huge chain into the expander. Serving and GDS on the same box contend — see algorithms.

Put it to work¶

Run locally¶

docker run --name neo4j -p 7474:7474 -p 7687:7687 \
  -e NEO4J_AUTH=neo4j/password \
  -e NEO4J_PLUGINS='["graph-data-science"]' \
  neo4j:5

Browser: http://localhost:7474. Bolt: 7687.

Constraints and indexes first¶

CREATE CONSTRAINT user_id IF NOT EXISTS
FOR (u:User) REQUIRE u.user_id IS UNIQUE;
CREATE CONSTRAINT device_id IF NOT EXISTS
FOR (d:Device) REQUIRE d.device_id IS UNIQUE;
CREATE CONSTRAINT ip_addr IF NOT EXISTS
FOR (i:IP) REQUIRE i.address IS UNIQUE;
CREATE CONSTRAINT txn_id IF NOT EXISTS
FOR (t:Transaction) REQUIRE t.txn_id IS UNIQUE;
CREATE CONSTRAINT merchant_id IF NOT EXISTS
FOR (m:Merchant) REQUIRE m.merchant_id IS UNIQUE;

CREATE INDEX ip_flag IF NOT EXISTS FOR (i:IP) ON (i.flag);
CREATE INDEX txn_ts IF NOT EXISTS FOR (t:Transaction) ON (t.ts);

Without the uniqueness constraints, MERGE is guesswork and rings split.

MERGE from CDC (idempotent upsert)¶

MERGE (u:User {user_id: $uid})
  ON CREATE SET u.name = $name
MERGE (d:Device {device_id: $did})
  ON CREATE SET d.fingerprint = $fp
MERGE (ip:IP {address: $addr})
MERGE (m:Merchant {merchant_id: $mid})
  ON CREATE SET m.name = $mname
MERGE (t:Transaction {txn_id: $tid})
  SET t.amount = $amount, t.ts = datetime($ts)
MERGE (u)-[:MADE]->(t)
MERGE (t)-[:ON]->(d)
MERGE (t)-[:FROM]->(ip)
MERGE (t)-[:AT]->(m)
MERGE (u)-[r:USES]->(d)
  SET r.last_seen = datetime($ts)
MERGE (u)-[s:USES]->(ip)
  SET s.last_seen = datetime($ts)

MERGE on a non-indexed property is a scan. Always MERGE on the constrained id.

Pattern matching — fraud serving¶

// 1 hop: devices for user
MATCH (u:User {user_id: $uid})-[:USES]->(d:Device)
RETURN d.device_id, d.fingerprint;

// 2 hop: other users sharing a device, skip supernodes
MATCH (u:User {user_id: $uid})-[:USES]->(d:Device)<-[:USES]-(other:User)
WHERE other <> u AND coalesce(d.flag, '') <> 'supernode'
RETURN DISTINCT other.user_id
LIMIT 100;

// 3 hop: merchants in that neighbourhood (bounded)
MATCH (u:User {user_id: $uid})-[:USES]->(d:Device)<-[:USES]-(other:User)
WHERE coalesce(d.flag, '') <> 'supernode'
WITH DISTINCT other LIMIT 200
MATCH (other)-[:MADE]->(t:Transaction)-[:AT]->(m:Merchant)
WHERE t.ts > datetime() - duration('P90D')
RETURN m.merchant_id, count(*) AS n
ORDER BY n DESC
LIMIT 20;

Variable-length — always capped¶

MATCH (ip:IP {address: $addr})
WHERE ip.flag IS NULL OR ip.flag <> 'supernode'
MATCH (ip)<-[:FROM]-(:Transaction)<-[:MADE]-(u:User)
MATCH path = (u)-[:USES|MADE|FROM|ON|AT*1..3]-(other:User)
RETURN DISTINCT other.user_id, min(length(path)) AS hops
LIMIT 500;

*1..3 is the product. * is an incident.

Shortest path (investigation)¶

MATCH (a:User {user_id: $a}), (b:User {user_id: $b})
MATCH path = shortestPath((a)-[:USES|MADE|FROM|ON|AT*..6]-(b))
RETURN [n IN nodes(path) | labels(n)[0] + ':' + coalesce(n.user_id, n.device_id, n.address, n.txn_id)] AS steps,
       length(path) AS hops;

Still cap *..6. Unbounded shortestPath on a connected-world graph is a CPU bomb.

Aggregation (small neighbourhoods only)¶

MATCH (t:Transaction)-[:AT]->(m:Merchant {merchant_id: $mid})
WHERE t.ts > datetime() - duration('P1D')
RETURN count(t) AS n, sum(t.amount) AS amt;

Company-wide totals: ClickHouse.

Python driver¶

from neo4j import GraphDatabase

driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))

CYPHER_2HOP = """
MATCH (u:User {user_id: $uid})-[:USES]->(d:Device)<-[:USES]-(other:User)
WHERE other <> u AND coalesce(d.flag,'') <> 'supernode'
RETURN DISTINCT other.user_id AS user_id
LIMIT 100
"""

def shared_device_users(uid: str) -> list[str]:
    with driver.session() as session:
        recs = session.run(CYPHER_2HOP, uid=uid)
        return [r["user_id"] for r in recs]

Parameterise. Do not f-string Cypher.

EXPLAIN and PROFILE¶

EXPLAIN
MATCH (u:User {user_id: $uid})-[:USES]->(d:Device)
RETURN d;

Want NodeIndexSeek / NodeUniqueIndexSeek, then Expand.
Reject AllNodesScan, NodeByLabelScan on :Transaction, cartesian ValueHashJoin of two disconnected MATCH.

PROFILE
MATCH (u:User {user_id: $uid})-[:USES]->(d:Device)<-[:USES]-(o:User)
RETURN o.user_id;

Read db hits and rows on each Expand. If Expand rows jump 1 → 5,000,000, you hit a supernode.

Where teams get caught¶

How it fails¶

Availability: Neo4j Causal Cluster (or Aura) — read replicas for analysts, write on primary. Do not run WCC on the primary at noon.

What happened next¶

It was A. PROFILE opened with AllNodesScan, which is the planner saying it had no way to find the starting node and read every node in the graph to locate one. An index on User.user_id turns that first operator into a seek and the query into the bounded 2-hop expand it was always meant to be.

The LIMIT was already there and did not help, because it applies after the rows exist. Limiting the output of a scan still costs the scan.

Every Cypher pattern has this shape: find the start, then expand. The expand was already bounded — [:USED] twice, nothing recursive — so the entire incident lived in the first operator, and PROFILE named it in one line.

Check your understanding¶

Reference¶

Behaviour at the next orders of magnitude, the trade-offs, the alternatives, and what to check when inheriting someone else's version of this — kept here rather than in the walkthrough above.

How to investigate¶

1. PROFILE the exact parameterised query from production (same $addr).
2. Degree of the start node and of the first expand type.
3. Query log / query.log slow queries.
4. Page cache hit ratio — if GDS just ran, cache is cold.
5. Lock / stopped queries: SHOW TRANSACTIONS.

Scale: 10× / 100× / 1000׶

10×. One instance, constraints, 90-day edges, replicas for analysts.

100×. Causal cluster; separate GDS box or gds.graph.project off-peak; prune edges; risk API may cache 2-hop sets in Redis.

1000×. Serving is ego-net in KV; Neo4j is investigation + CDC subset (recent, sampled types). Sharding graphs is still painful (fabric / sharding by tenant). Do not expect linear scale-out like Cassandra.

Trade-offs¶

Cypher + native expand is the point. You pay for clustering ops, RAM for hot graph, and a second consistency story vs Postgres.

Aura vs self-managed: same modelling, less pager, less control of GDS memory.

Alternatives¶

Apply¶

Every production Cypher review: start label + indexed property, typed relationships, max depth, LIMIT, supernode filter, PROFILE attached as screenshot.