-
-
Notifications
You must be signed in to change notification settings - Fork 11.2k
Plan: Migrate graphify from networkx to Apache Arrow
1. Motivation
The current codebase stores its entire knowledge graph in a single graph.json
file (NetworkX node_link_data JSON). This works but has known limitations:
-
No graph-native queries — every read loads the full JSON into a NetworkX
Graphin memory, then walks it with BFS/DFS in Python. There is no
MATCH,WHERE, orOPTIONALmatching. -
No incremental reads — you cannot ask "which nodes have community 5"
without deserializing the entire graph.Similar issues here.
Similar discussion about scalability, large repos: Are there reliable benchmarks showing Graphify improves coding agent performance on large repos? #1328 , Establish a Knowledge Base #1340Full plan: https://gist.github.com/adsharma/c0fd0c9163f1fc60669cacfe955e152f
All reactions
Replies: 1 comment
Let's chat
Feel free to book a time slot as per your convenience!