networkx vs apache arrow and cypher

[adsharma]

Plan: Migrate graphify from networkx to Apache Arrow

1. Motivation

The current codebase stores its entire knowledge graph in a single graph.json
file (NetworkX node_link_data JSON). This works but has known limitations:

• No graph-native queries — every read loads the full JSON into a NetworkX
  Graph in memory, then walks it with BFS/DFS in Python. There is no
  MATCH, WHERE, or OPTIONAL matching.

• No incremental reads — you cannot ask "which nodes have community 5"
  without deserializing the entire graph.

  Similar issues here.
  Similar discussion about scalability, large repos: Are there reliable benchmarks showing Graphify improves coding agent performance on large repos? #1328, Establish a Knowledge Base #1340

  Full plan: https://gist.github.com/adsharma/c0fd0c9163f1fc60669cacfe955e152f

[safishamsi]

Let's chat
Feel free to book a time slot as per your convenience!

https://graphify-labs.cal.com/safi/30min?layout=mobile