Multiple distance types in same index?

k-NN
#1

I think I know that the answer is “no” but just wanted confirmation / suggestions on how I could solve this problem.

Suppose I have some data that is of the following format:

{
  md5 : xxxx, #unique id
  float_vector: xxxxxx,
  binary_vector: xxxx,
  other_data : { 
     .....
   }
}

I would like to be able to do a knn query either using Euclidean distance on the float vector or Hamming distance on the binary vector.

Based on the docs, I don’t think this is possible as the distance type is per-index. So what might be a way to solve this? For example, would I ingest the main data in one index, the md5/float_vector in another index and md5/binary_vector in a third index. Then to do the search I do it in the appropriate index and then look up full records in the original index?

#2

@gdd314596 apologies for the delay in responding. You are right. It is not possible to define space type per field. We already created an issue here. You can watch the above issue to get latest updates.

Currently, approximate-knn is supported only for l2 and cosnisnesimil, while custom scoring approach support l2, cosisnesimil and Hamming bit spaces.
In your case, one possible option is to create knn index with setting, “knn.space_type”: “l2” and use approximate-knn approach for knn query using Euclidean distance on the float vector, and use custom scoring (pre-filter approach) for Hamming distance on the binary vector like mentioned here.

Please let us know did the above suggestion helps you.

#3

Yes thanks! I was able to figure out how to combine l2/Hamming after realizing that that you didn’t need to index as l2. Thanks again