Implement LRU cache

[agritheory]

• Create LRU cache with S3 fallback
• Eviction policy should be based on a configured cache size in MB
• To facilitate hybrid cloud deployments
  • Save file locally if there's a connection error
  • Track connection errors between Frappe and S3 services with retry logic on scheduler

[agritheory]

@batonac Would love some feedback on this skeleton of a design

[batonac]

In my scenario frappe is less reliable than s3, so I haven't seen the need. 😅

Where I have experienced the need for a fallback is with uploading when direct upload via presigned urls is deployed, but that's a different subject.

It seems to me that offloading all files to s3 is all that's needed for hybrid cloud deployments, but perhaps I'm missing something...

[agritheory]

@batonac

offloading all files to s3

Yes, if S3 is reachable by the (on prem) ERPNext instance. This feature proposes that there's a local cache and failover/retry in case of connection errors.

[batonac]

Ah. So is one of the goals here to fix #107?

[batonac]

What about just mounting the share via s3fs-fuse, if available on the server?