I was playing with async cron tasks from fang-examples.
I configured the task to be unique and to keep the history of executions.
After running the example on 2 parallel processes I noticed duplicated tasks in the DB and in the application logs.
insert_task_if_not_exist_query seems to not handle the case when multiple instances (processes) are running in parallel.
A possible solution to this is to do something like:
- Create a unique index like:
CREATE UNIQUE INDEX fang_task_uniqueness_idx ON fang_tasks (uniq_hash, state) WHERE (uniq_hash is not null and state in ('new', 'in_progress', 'retried'));
- update insert_task_uniq.sql:
INSERT INTO "fang_tasks" ("metadata", "task_type" , "uniq_hash", "scheduled_at") VALUES ($1, $2 , $3, $4) ON CONFLICT DO NOTHING RETURNING *;WARNING: the above SQL statements were not tested and might have SQL errors or not be optimal, so should not be treated as a working solution before testing.
This probably doesn't make sense to do if fang is not designed to work on multiple processes. In this case, it is worth mentioning somewhere in the docs that fact (sorry if it is mentioned already and I missed it).
I was playing with async cron tasks from fang-examples.
I configured the task to be unique and to keep the history of executions.
After running the example on 2 parallel processes I noticed duplicated tasks in the DB and in the application logs.
insert_task_if_not_exist_query seems to not handle the case when multiple instances (processes) are running in parallel.
A possible solution to this is to do something like:
WARNING: the above SQL statements were not tested and might have SQL errors or not be optimal, so should not be treated as a working solution before testing.
This probably doesn't make sense to do if fang is not designed to work on multiple processes. In this case, it is worth mentioning somewhere in the docs that fact (sorry if it is mentioned already and I missed it).