Real-Time Streaming Risk Pipeline

Fraud & Ring Detection Engine

Scores every transaction in real time against three independent fraud signals — an online-learning model, a Neo4j ring detector, and a Qdrant synthetic-identity check — then merges them into one ALLOW / REVIEW / DECLINE decision. Twelve containers, one docker compose up.

The problem

A single fraud signal is either too noisy to act on or too conservative to catch anything. A behavioral model flags an odd transaction but can't see that six accounts share one device; a graph finds the ring but not the one-off anomaly; neither notices a synthetic identity reusing the same fingerprint. The engineering problem is running all three concurrently over one event stream, at transaction speed, and still returning a decision when one of them is down.

Architecture & approach

1. A PaySim-grounded simulator emits payment events into a Redis Stream. Three consumer groups read the same stream in parallel and produce independent signals, so adding a detector never slows the ones already there.

2. Behavioral signal: a River online logistic classifier does predict_proba_one then learn_one on every transaction's ground-truth label — the model adapts as fraud patterns drift instead of going stale between retrains.

3. Relational signal: accounts, devices and IPs are upserted into Neo4j, and a scheduled Louvain community-detection pass surfaces dense clusters of accounts sharing devices and IPs. A community counts as a ring only at size ≥ 3 and intra-community density ≥ 0.4 — the size floor drops coincidental pairs, the density floor drops communities Louvain groups for modularity that aren't actually collusive.

4. Identity signal: each transaction is feature-hash embedded and looked up in Qdrant by cosine nearest neighbor. The 0.92 threshold is deliberately high — genuinely distinct fingerprints rarely exceed ~0.9, so it targets real near-duplicate reuse rather than manufacturing false positives.

5. A NestJS orchestrator consumes the scored stream and enriches each event by calling the graph and similarity services over HTTP, each wrapped in an opossum circuit breaker. The merge policy refuses to auto-decline on one soft signal: a very high ML score declines alone, a merely suspicious score declines only when corroborated by a structural signal, and anything mildly suspicious goes to human review.

6. Fail-open with degraded scoring, implemented rather than improvised: when an enrichment service goes slow or dies the breaker trips, the orchestrator drops that signal, stamps degraded: true on the decision, and keeps deciding on what's left. No transaction stalls waiting on a dead dependency.

7. Every application service is stateless — all shared state lives in Redis, Neo4j or Qdrant, never a process global — so replicas scale horizontally and Redis consumer groups hand disjoint entries to each one with no double-processing.

8. Decisions stream to a Next.js dashboard over WebSocket with auto-reconnect: live feed, flagged panel, force-directed ring visualization, and throughput/latency metrics. The whole stack — Redis, Neo4j, Qdrant, five services and a k6 runner — comes up from a single compose file with nothing installed on the host.

Stack

• Python
• FastAPI
• River
• NestJS
• TypeScript
• Redis Streams
• Neo4j
• Qdrant
• Next.js
• k6
• Docker Compose