We have 3 nodes of 60GB Machines where 50% i.e. 30GB is for Heap and 30 GB is for File Cache. We have 3 indexes of 1 Primary and 2 Replicas each. We have calculated the KNN Graph Memory required for them is around 28GB in total (across 3).
What should we do once the graph grows out of that?
Should we Scale Horizontally by adding more Nodes of 60GB RAM or should we Scale Vertically by Adding Machines with higher RAM per node?
Also, where does the KNN Graph reside? in Heap or File Cache? Should I give more RAM to Heap or file Cache?