feat: add download quota limit per provider#62
Merged
Conversation
Adds QuotaBytes and QuotaPeriod fields to Provider, allowing operators to cap the total bytes downloaded from each provider per rolling window (e.g. 100 GB/month). When a provider's quota is reached it is skipped in dispatch — both during selection (weight 0 in round-robin, skipped in FIFO) and as a hard guard in the main/backup provider loops. If all providers are over quota, ErrQuotaExceeded is returned. The hot-path overhead is a single atomic.Bool load (~1 ns) when quota is not exceeded. time.Now() is only called once the cached flag is set, i.e. when a provider has actually run out of quota. Quota usage resets automatically after QuotaPeriod elapses (0 = lifetime cap, no reset). Quota state (QuotaBytes, QuotaUsed, QuotaResetAt, QuotaExceeded) is exposed via ProviderStats / Client.Stats().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
QuotaBytes int64andQuotaPeriod time.Durationfields toProvider, allowing a per-provider download cap with optional auto-reset (e.g. 100 GB/month)ErrQuotaExceededreturned to callerProviderStatsviaClient.Stats():QuotaBytes,QuotaUsed,QuotaResetAt,QuotaExceededPerformance
Hot-path overhead when quota is not exceeded: one
atomic.Boolload (~1 ns).time.Now()is only called after the quota flag is set (i.e. when a provider has actually run out). See PR description for full analysis.Test plan
TestProviderGroup_isQuotaExceeded_Unlimited—QuotaBytes=0always returns falseTestProviderGroup_isQuotaExceeded_NotYetHit— fast path before thresholdTestProviderGroup_isQuotaExceeded_FlagSet_NoReset— lifetime cap never resetsTestProviderGroup_isQuotaExceeded_PeriodElapsed_Resets— quota resets after periodTestProviderGroup_isQuotaExceeded_PeriodNotYetElapsed— stays exceeded before period endsTestProviderStats_QuotaAccounting— byte accounting sets the flag correctlyTestClient_Stats_QuotaFields—Stats()exposes quota fields with correct valuesTestClient_QuotaExceeded_FallsThrough— request falls through to second provider when first is exceededTestClient_AllQuotaExceeded_ReturnsError—ErrQuotaExceededwhen all providers exceededmake check(lint + race test suite) passes