a file parser that embedds some text fileds in vectors
an elasticsearch ingest pipeline that process additionnel transformations on other numerical/date fields
I’d like to use Knn for a classification use case. i.e. computing the similarity score of each embedded vector against a set of labelised vectors and adding a label to this vector.
Is it possible to do this using some script processor in ingest pipeline ?
@wassim.dhib apologies for responding late. I believe script processor is used to perform an operation on individual document like add fields, replace score, etc. from same index.
In your case, I will build another pipeline as , for every document which contains embedded vector, i will use those vector as query vector for search query on index containing labelised vectors , calculate the label from the search result and update document with label.