Anomaly Detection for Continuous Ingesting Data

Hi, Currently I have ingesting a 20-25GB of daily data in opendistro. And I have created multiple detectors and set aggregation method avg but its getting failed after a couple of hours and got message not have enough data to initialize detector. But when i set aggregation method SUM. The detectors running successfully. Please let me know where I am having mistake. What would be the settings for Detector Interval, Window delay and window size for it ?

AD will detect realtime streaming data. Can you check the date range your data? For SUM, it will return 0 if no data found, but for AVG, it will return null if no data found. So if you are using SUM, the detector can run successfully as it can get 0 even no data.

Hi,
I’m getting following error message for the detector I have created: " The detector is not initialized because no sufficient data is ingested.
Make sure your data is ingested correctly. If your data source has infrequent ingestion, increase the detector time interval and try again."

Also I have shared the screenshot of detector settings. Window Size: 8

@ylwu Please check my previous message. Can you help me on it ?

From the screenshot, the detector interval is 40 minutes. The error message means you have no enough data to train model. The model need about 150 data points to train. Here one data point is the aggregated result of one 40 minutes interval. If you don’t have historical data, then the detector will wait 150 intervals to get enough training data. That will be 150 * 40 = 6000 minutes. To make the model pass training faster, you can ingest enough historical data or reduce detector interval, make sure every interval (40 minutes in this case) has data.