Read/Write lock for PickledDB

[mnoukhov]

I worked off of https://github.com/elarivie/pyReaderWriterLock/ to make a wrapper for FileLock that allows for multiple readers but only one writer at a time

WIP, still need to test

[bouthilx]

🎊 Thanks a lot! Let me know when it is ready for review. :)

[Thomsch]

@mnoukhov @bouthilx Is there a status update on this contribution?

[mnoukhov]

Ah sorry. I never got around to writing tests for R/W concurrency. If you send me a pointer to one, I'd happily do it

[Thomsch]

No worries, thanks for the update. @bouthilx should be able to give you a pointer. I'll check with him next week.

[bouthilx]

Unittests with concurrency 😰 😅 I don't see any way you can do this without using python multithreading/multiprocessing. One simple test for the locking would be to have n threads/process with a DB having a single data point {'n': 0}, all readers do read_and_write(query={'n': worker_n}, data={'n': worker_n+1}) until the DB reaches 'n' == num_workers and you verify that you actually went through the sequence (0, 1, 2, 3, ...).

But that's only testing the atomicity of write/read, not number of concurrent readers. For testing number of concurrent readers, I would hardcode a scenario where there is one writer and N readers and mock open to force read some valid or invalid numbers of readers and confirm it is handled properly.