Entity-Relation Graph Mining & GraphRAG Pipeline

Entity graphs and community summaries answer corpus-wide questions that chunk retrieval cannot.

Part 1 of 5

Basics

The whole idea first, then the terms the rest of this page uses.

1.1Whole idea

Chunk retrieval answers 'what does this contract say?' but not 'what are the main risks across all contracts?'. GraphRAG extracts entities and relations with an LLM, groups them into communities, summarises each one, and answers from those summaries.

Why it matters

Entity graphs and community summaries answer corpus-wide questions that chunk retrieval cannot.

1.2Key concepts

One idea per card, in plain words.

01Triple

One fact as subject, relation and object: 'Acme supplies steel'.

02Entity resolution

Merges 'Acme Corp' and 'ACME' into one node. Errors here damage everything downstream.

03Community detection

Groups densely linked entities. Production systems use Leiden; the code uses Louvain's first phase.

04Global search

Maps the question over community summaries, then reduces the partial answers into one.

Part 2 of 5

Question

What exactly you must answer, and what to settle before you start.

2.1Question

Interviewer

Analysts ask 'What are the main supplier risks across 20,000 contracts?'. Chunk RAG returns ten arbitrary snippets. Build the pipeline that can answer.

2.2Clarify first

Each answer removes a guess.

You askAnswer
How often do contracts change?A few hundred a week, so incremental updates matter.
Budget for indexing?LLM extraction runs per chunk, so estimate cost before committing.
Part 3 of 5

Detailed answer

How it works, and how to build or explain it.

3.1Step by step

The mechanism, in the order you would explain it.

  1. Extract triples

    Ask an LLM for entities and relations per chunk, in a strict schema.

  2. Resolve entities

    Normalise names, then merge by similarity. Keep source chunk ids on every fact.

  3. Store the graph

    In Neo4j: MERGE (s:Entity {name: $s}) MERGE (o:Entity {name: $o}) MERGE (s)-[:REL {type: $r}]->(o).

  4. Find communities

    Move each entity into the community that most raises modularity, until nothing moves.

  5. Summarise communities

    Give the LLM each community's facts, and store the report with its embedding.

  6. Answer by map-reduce

    Ask each relevant summary for a partial answer, rank them, and combine the best.

3.2Code

Working code for the core of it.

Finding communities for GraphRAG

A global question needs summaries of groups of entities. This finds the groups.

graph-communities.tsTypeScriptCopy
1export interface Triple {2  subject: string;3  relation: string;4  object: string;5}6 7/** Entity graph behind GraphRAG: deduplicated triples, plus communities to summarise. */8export class KnowledgeGraph {9  private edges = new Map<string, Set<string>>();10  private triples = new Map<string, Triple>();11 12  add(t: Triple): void {13    const [s, o] = [t.subject.trim().toLowerCase(), t.object.trim().toLowerCase()];14    this.triples.set(`${s}|${t.relation}|${o}`, { subject: s, relation: t.relation, object: o });15    for (const [a, b] of [[s, o], [o, s]]) {16      if (!this.edges.has(a)) this.edges.set(a, new Set());17      this.edges.get(a)!.add(b);18    }19  }20 21  /** Louvain's local-moving phase: each entity joins the community that most raises modularity. */22  communities(maxRounds = 20): string[][] {23    const nodes = [...this.edges.keys()].sort();24    const degree = (n: string) => this.edges.get(n)!.size;25    const twoM = nodes.reduce((sum, n) => sum + degree(n), 0);26    const home = new Map(nodes.map((n) => [n, n]));27    const total = new Map(nodes.map((n) => [n, degree(n)])); // sum of degrees per community28 29    for (let round = 0; round < maxRounds; round++) {30      let moved = false;31      for (const n of nodes) {32        const current = home.get(n)!;33        total.set(current, total.get(current)! - degree(n));34        const links = new Map<string, number>([[current, 0]]);35        for (const m of this.edges.get(n)!) links.set(home.get(m)!, (links.get(home.get(m)!) ?? 0) + 1);36 37        let best = current;38        let bestGain = links.get(current)! - (total.get(current)! * degree(n)) / twoM;39        for (const [c, k] of [...links].sort((x, y) => x[0].localeCompare(y[0]))) {40          const gain = k - (total.get(c)! * degree(n)) / twoM;41          if (gain > bestGain + 1e-12) {42            best = c;43            bestGain = gain;44          }45        }46        total.set(best, total.get(best)! + degree(n));47        home.set(n, best);48        if (best !== current) moved = true;49      }50      if (!moved) break;51    }52    const groups = new Map<string, string[]>();53    for (const n of nodes) groups.set(home.get(n)!, [...(groups.get(home.get(n)!) ?? []), n]);54    return [...groups.values()].sort((a, b) => b.length - a.length);55  }56 57  /** The facts inside one community: the input to its LLM summary. */58  factsFor(community: string[]): string[] {59    const inside = new Set(community);60    return [...this.triples.values()]61      .filter((t) => inside.has(t.subject) && inside.has(t.object))62      .map((t) => `${t.subject} ${t.relation} ${t.object}`);63  }64}
64 linesUTF-8TypeScript
this.triples.set(...)
Lowercases and dedupes, since the extractor repeats itself.
k - (total.get(c)! * degree(n)) / twoM
The modularity gain: links into a community, minus what chance would give.
factsFor(community: string[])
The facts to give the summariser for one group.

Tested by npm run test:modules.

Part 4 of 5

Q&A

What interviewers ask next. Answer out loud, then open.

4.1Interview questions

What does it cost?
Far more than embeddings: at least one LLM call per chunk, plus community reports. Pilot on a sample and extrapolate.
Local or global search?
Local search starts from entities in the question and suits specific facts. Global search reads community summaries and suits themes.
How do you update it?
Re-extract changed documents, update affected communities, and regenerate only their summaries.
Part 5 of 5

Extra depth

Edge cases, counterpoints and what to read next.

5.1Edge cases and counterpoints

Includes the strongest argument against this approach.

Show detailsHide details
Provenance
Keep chunk ids on every triple. Without them you cannot cite or audit an answer.
Counterpoint
Most user questions are specific, and hybrid RAG answers them more cheaply. Build GraphRAG only when corpus-wide questions truly matter.