Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ $$ Throttling = \begin{cases}
\displaystyle 0 \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ otherwise \\
\end{cases}$$

Implementation note: internal `Utilization` telemetry is a ratio (`1.0 == 100%`), while `danger_zone_*` settings are configured in percentage points (`(0, 100]`).

## Architecture

The MemLimiter comprises two main parts:
Expand Down Expand Up @@ -126,14 +128,19 @@ You must also provide your own `stats.ServiceStatsSubscription` and `stats.Servi

There are several key settings in MemLimiter configuration (see [top-level config](config.go) and [controller config](controller/nextgc/config.go)):

- `go_memory_limit` (optional, top-level)
- `controller_nextgc.rss_limit`
- `controller_nextgc.danger_zone_gogc`
- `controller_nextgc.danger_zone_throttling`
- `controller_nextgc.min_gogc`
- `controller_nextgc.period`
- `controller_nextgc.component_proportional.window_size`
- `controller_nextgc.component_proportional.coefficient` ($C_{p}$)
| Setting name | Type | Allowed range | Default | Description |
| --- | --- | --- | --- | --- |
| `go_memory_limit` | bytes string (`"800M"`, `"1G"`, `"0"`) | `"0"` (disabled) or `(0, MaxInt64]` bytes | `0` (disabled) | Optional Go runtime soft memory limit applied via `debug.SetMemoryLimit` during service lifecycle. |
| `controller_nextgc.rss_limit` | bytes string | `(0, +inf)` bytes | none (required) | Hard process RSS budget used by the controller. |
| `controller_nextgc.danger_zone_gogc` | unsigned integer | `(0, 100]` | none (required) | Utilization threshold that enables GC tightening logic. Value `100` is emergency-only trigger (near-full-budget). |
| `controller_nextgc.danger_zone_throttling` | unsigned integer | `(0, 100]` | none (required) | Utilization threshold that enables request throttling. Value `100` is emergency-only trigger (near-full-budget). |
| `controller_nextgc.min_gogc` | integer | `0` (auto-default), or `[1, 100]` | `10` (when set to `0`) | Lower bound for computed `GOGC` in red zone. |
| `controller_nextgc.period` | duration string (`"100ms"`, `"1s"`) | `(0, +inf)` duration | none (required) | Controller loop period for control recomputation. |
| `controller_nextgc.component_proportional.coefficient` (`C_p`) | float | any non-zero value | none (required) | Proportional component strength (higher value means more aggressive reaction near limit). |
| `controller_nextgc.component_proportional.window_size` | unsigned integer | `[0, +inf)` | `0` | EMA smoothing window size for controller output (`0` disables smoothing). |

Recommendation: keep `danger_zone_throttling >= danger_zone_gogc` so GC intensification starts before request shedding.
Implementation detail: current NextGC controller clamps output to `99`, so maximum throttling emitted by this controller is `99%`.

Example:

Expand Down
2 changes: 1 addition & 1 deletion backpressure/throttler.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ func (t *throttler) AllowRequest() bool {
return true
}

// Flip a coin in the range [0; 100].
// Flip a coin in the range [0; 99].
// If the actual value is less than the threshold value, throttle the request.
// Otherwise, allow the request.
// math/rand/v2 top-level functions are safe for concurrent use and provide
Expand Down
12 changes: 6 additions & 6 deletions controller/nextgc/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ type ControllerConfig struct {
RSSLimit bytes.Bytes `json:"rss_limit"`
// DangerZoneGOGC - RSS utilization threshold that triggers controller to
// set more conservative parameters for GC.
// Possible values are in range (0; 100).
// Possible values are in range (0; 100].
DangerZoneGOGC uint32 `json:"danger_zone_gogc"`
// DangerZoneThrottling - RSS utilization threshold that triggers controller to
// throttle incoming requests.
// Possible values are in range (0; 100).
// Possible values are in range (0; 100].
// It's recommended to keep it greater than or equal to DangerZoneGOGC so that
// the service first intensifies GC and starts throttling only later.
DangerZoneThrottling uint32 `json:"danger_zone_throttling"`
Expand Down Expand Up @@ -82,16 +82,16 @@ func (c *ControllerConfig) validateRSSLimit() error {
}

func (c *ControllerConfig) validateDangerZoneGOGC() error {
if c.DangerZoneGOGC == 0 || c.DangerZoneGOGC >= 100 {
return errors.New("invalid DangerZoneGOGC value (must belong to (0; 100))")
if c.DangerZoneGOGC == 0 || c.DangerZoneGOGC > 100 {
return errors.New("invalid DangerZoneGOGC value (must belong to (0; 100])")
}

return nil
}

func (c *ControllerConfig) validateDangerZoneThrottling() error {
if c.DangerZoneThrottling == 0 || c.DangerZoneThrottling >= 100 {
return errors.New("invalid DangerZoneThrottling value (must belong to (0; 100))")
if c.DangerZoneThrottling == 0 || c.DangerZoneThrottling > 100 {
return errors.New("invalid DangerZoneThrottling value (must belong to (0; 100])")
}

return nil
Expand Down
21 changes: 15 additions & 6 deletions controller/nextgc/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,17 @@ func TestComponentConfig(t *testing.T) {
require.Error(t, c.Prepare())
})

t.Run("bad danger zone GOGC equal to 100", func(t *testing.T) {
t.Run("danger zone GOGC equal to 100 is allowed", func(t *testing.T) {
c := &ControllerConfig{
RSSLimit: bytes.Bytes{Value: 1},
DangerZoneGOGC: 100,
RSSLimit: bytes.Bytes{Value: 1},
DangerZoneGOGC: 100,
DangerZoneThrottling: 100,
Period: duration.Duration{Duration: 1},
ComponentProportional: &ComponentProportionalConfig{
Coefficient: 1,
},
}
require.Error(t, c.Prepare())
require.NoError(t, c.Prepare())
})

t.Run("bad danger zone throttling", func(t *testing.T) {
Expand All @@ -46,13 +51,17 @@ func TestComponentConfig(t *testing.T) {
require.Error(t, c.Prepare())
})

t.Run("bad danger zone throttling equal to 100", func(t *testing.T) {
t.Run("danger zone throttling equal to 100 is allowed", func(t *testing.T) {
c := &ControllerConfig{
RSSLimit: bytes.Bytes{Value: 1},
DangerZoneGOGC: 50,
DangerZoneThrottling: 100,
Period: duration.Duration{Duration: 1},
ComponentProportional: &ComponentProportionalConfig{
Coefficient: 1,
},
}
require.Error(t, c.Prepare())
require.NoError(t, c.Prepare())
})

t.Run("bad period", func(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion controller/nextgc/controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ type controllerImpl struct {
pValue float64 // proportional component's output
sumValue float64 // final output
goAllocLimit uint64 // memory budget [bytes]
utilization float64 // memory budget utilization [percents]
utilization float64 // memory budget utilization ratio (1.0 = 100%)
rss uint64 // physical memory actual consumption
consumptionReport *stats.ConsumptionReport // latest special memory consumers report
controlParameters *stats.ControlParameters // latest control parameters value
Expand Down
4 changes: 2 additions & 2 deletions stats/memlimiter.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ type MemoryBudgetStats struct {
RSSLimit uint64
// GoAllocLimit - allocation limit for Go Runtime (with the except of CGO) [bytes].
GoAllocLimit uint64
// Utilization - memory budget utilization [percents]
// (definition depends on a particular controller implementation).
// Utilization - memory budget utilization ratio
// (for example, 1.0 means 100%; definition depends on controller implementation).
Utilization float64
}

Expand Down