Verify if syncing process is susceptible to race conditions and propose solution

[Azdaroth]

In some cases there are problems with synced_id already being taken on uniqueness validation, even though no duplicates exist, might be caused by some race conditions when running concurrent syncing processes.

[Fredar]

Imo we should make the sync unique per scope.

Its easily achieved on app with sidekiq-uniqe-jobs, otherwise would have to store the sync status in redis and discard new syncs if needed.

[Mandaryn]

for me its still the problem of apps design to prevent concurrent syncs, i cannot see a clear solution to drop duplicate syncs on synced level without introducing major changes to the apps design

[Fredar]

Everyone ok then to use sidekiq-unique or other means to stop it on app side?

[Azdaroth]

Not sure if sidekiq-unique solves the problem, I mean it would solve some parts, but if 1 job and e.g. something synchronous from the app run at the same time and they both synchronize e.g. Rental, we could have the same issue. I think we should do some locking for synchronizing models. So if some service / job synchronizes Booking-Rental-Rate and other service synchronizes Rental-Tax then the second one should wait for the first one, if they synchronize for the same scope of course. Such thing may happen in bsa-website with RentalsSynchronizer running both from jobs and synchronously from admin.

[Mandaryn]

relying on things being synchronous is a bad way, somehow feel this in my bones, would rather make some code checking if synced data is integral, some references counting maybe?

[Azdaroth]

Not sure what you mean here, how / why would you check for integrity? Anyway, in admin I think it needs to be synchronous, somehow we need to show these rentals, refreshing isn't the best UX I guess, so there are some cases where synchronous syncs may happen.

[Mandaryn]

my case is bsa-notifications, once i sync bookings i should be sure all its associations up to this point in time are also in sync, to send mail with all proper data, synchronous sync doesn't really cut it, even if you sync all stuff before booking for instance you can still get booking the was updated after the rest of stuff, which would trigger mail with invalid datas to be sent

[Mandaryn]

so my use case would be to detect inconsistent data and defer sending mail until its properly in sync