add traffic calculator #249
Conversation
| private final int thresholdMultiplier; | ||
| private final ICloudStorage cloudStorage; | ||
| private final String s3DeltaPrefix; // (e.g. "optout-v2/delta/") | ||
| private final String whitelistS3Path; // (e.g. "optout-breaker/traffic-filter-config.json") |
There was a problem hiding this comment.
Should we reuse the same S3 bucket just different folder?
There was a problem hiding this comment.
yes, optout-v2 is top-level folder within the bucket, optout-breaker would also be folder in same bucket
|
|
||
| private long getTotalWhitelistDuration() { | ||
| long totalDuration = 0; | ||
| for (List<Long> range : this.whitelistRanges) { |
There was a problem hiding this comment.
There probably are several whitelist ranges, some within the first 24 hours, others in the 24–48 hour range, or even older if we don’t clear them. Maybe we should check the whitelist duration during specific time ranges instead of calculating over the entire period every time?
|
|
||
| try { | ||
| // Get list of delta files from S3 (sorted newest to oldest) | ||
| List<String> deltaS3Paths = listDeltaFiles(); |
There was a problem hiding this comment.
Maybe we can check the time it spends to list all delta files names in 3 months versus list 7 days delta file names.
There was a problem hiding this comment.
I don't think we need to optimize this. We get 1000 objects per list API call, with CloudStorageS3 handling the pagination automatically. Should only take a second or two to list all the deltas, we can check on it after deploy/test?
There was a problem hiding this comment.
Cool. Yeah, maybe we need to monitor this later.
Rough Usage: