rewrited to OOP

[blzzua]

LLM made me do this code structure, after that I fixed those bugs to make the code workable
there are so many changes that it makes sense to create a specific branch

some code is not tested, for example code related to Kibana is not tested at all
also I changed scroll id lifetime to 10 minutes

[AnthraX1]

LLM made me do this code structure, after that I fixed those bugs to make the code workable there are so many changes that it makes sense to create a specific branch

some code is not tested, for example code related to Kibana is not tested at all also I changed scroll id lifetime to 10 minutes

I intended to set scroll time to 1d because sometimes the connection breaks and goes unnoticed. The session will be expired when you restart it.

[blzzua]

I understand using a very long scroll_id session for production use, but I tested on my production elasticsearch :\ and it seemed to me that storing a large number of scroll_ids could cause problems. so I lowered this parameter.

ps. oh: I found in docs https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results " To prevent against issues caused by having too many scrolls open, the user is not allowed to open scrolls past a certain limit. By default, the maximum number of open scrolls is 500. This limit can be updated with the search.max_open_scroll_context cluster setting.
"

I thought that using scroll_id should have a relatively short session and continue it regularly after getting batch
https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-context
"Each scroll request (with the scroll parameter) sets a new expiry time."

[AnthraX1]

no problem. I'll just make it an option and default to 10m

[AnthraX1]

It might take a few days to test and review the code change.