How to update indexed data from ingestion pipeline

Hello! I need to update a field of several records already indexed based on a value from a new record still not indexed? Is there any way to implement it?

The table below describes my index with current docs. The yellow line would be a new doc arriving.

RULE: As far the latest entry is received (NEW “not indexed entry”) with STATE=OFF, all respective entries of Group 1 (FROM EXISTING INDEX) must be set COLOR=RED

image

Could you please let me know any example how to implement?

Thanks

Marcelo

This looks like you need to setup an ingest pipeline. Best to go through the docs and see if all your requirements are fulfilled.

Hi Tony, I tried, but I could have access only to docs in the pipeline still to be indexed. I could not query index later on to update old docs. Is there any way to implement a script (painless) for this purpose? As far I could figure out, ctx points to received docs and I have not way to point out to a target elastic index for other docs update in the sequence of ingestion. Am I right conceptually or there is a misunderstanding?

Have a look through this https://www.elastic.co/guide/en/elasticsearch/reference/7.9/docs-update-by-query.html#docs-update-by-query-api-ingest-pipeline. It may help, or the actual update by query will possibly be your solution.