Skip to content

Add per-client IP rate limiting via Apache mod_qos#957

Open
Raven-182 wants to merge 1 commit intojuju:masterfrom
Raven-182:apache-rate-limiting
Open

Add per-client IP rate limiting via Apache mod_qos#957
Raven-182 wants to merge 1 commit intojuju:masterfrom
Raven-182:apache-rate-limiting

Conversation

@Raven-182
Copy link
Copy Markdown
Contributor

@Raven-182 Raven-182 commented Apr 13, 2026

When apache-rate-limit-enabled=true, Apache limits how many HTTP requests a client IP can make within a time window and returns HTTP 429 when limit is hit.

Builds on proxy protocol support (f516806). HAProxy forwards the real client IP, which mod_remoteip extracts, but mod_qos cannot use directly. A small Lua script bridges this gap by running after mod_remoteip has extracted the real IP and copying it into X-Real-Client-IP header that mod_qos can read.

The mod_qos module is installed and enabled automatically.

A few notes:

  • When services are running behind HAProxy, proxy protocol must be enabled for rate limiting to take effect
  • Limits are enforced per unit (not cluster-wide), so the effective limit scales with the number of backend units
  • Cluster peer IPs are exempted to avoid throttling health checks
  • LuaScope thread and LuaCodeCache forever directives are set for performance because the lua script is stateless and does not need to be re-read and re-compiled on every request

Testing

  • Integration tested on 3-unit neutron-api HA deployment with HTTPS and proxy protocol
  • Integration tested on single-unit neutron-api with HTTPS:
    • Rate limiting still requires proxy protocol for neutron-api (local HAProxy service masks client IP without it)

Note on mod_qos and mod_lua

Noble ships version 11.74 for mod_qos which cannot use the client IP from mod_remoteip directly, newer versions of mod_qos (11.75+) improve this, so the lua script workaround may not be needed in the future.

See: https://mod-qos.sourceforge.net/CHANGES.txt

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds per-client IP HTTP request rate limiting to the Apache vhost templates using mod_qos, with an optional Lua hook to copy the real client IP (extracted by mod_remoteip from PROXY protocol) into a header that mod_qos can read.

Changes:

  • Add _get_rate_limit_context() plus a Lua script payload to generate template context and exempt cluster peer IPs.
  • Enable/install required Apache modules/packages (qos, lua, libapache2-mod-qos) when rate limiting is enabled.
  • Extend Apache vhost templates and unit tests to cover the new rate-limiting behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
charmhelpers/contrib/openstack/context.py Adds rate-limit context generation, Lua script content/path, and module enablement logic.
charmhelpers/contrib/openstack/templates/openstack_https_frontend Adds Lua hook + mod_qos directives for HTTPS frontend vhosts.
charmhelpers/contrib/openstack/templates/wsgi-openstack-api.conf Adds Lua hook + mod_qos directives for WSGI vhost template.
tests/contrib/openstack/test_os_contexts.py Adds/adjusts mocks and new tests validating rate-limit context/module enablement and Lua script deployment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread charmhelpers/contrib/openstack/context.py Outdated
Comment thread charmhelpers/contrib/openstack/context.py Outdated
Comment thread charmhelpers/contrib/openstack/templates/wsgi-openstack-api.conf Outdated
Comment thread charmhelpers/contrib/openstack/templates/openstack_https_frontend Outdated
@Raven-182 Raven-182 force-pushed the apache-rate-limiting branch from b6bbed7 to c1221e9 Compare April 20, 2026 20:09
@Raven-182
Copy link
Copy Markdown
Contributor Author

Raven-182 commented Apr 20, 2026

@juju juju deleted a comment from Copilot AI Apr 21, 2026
@juju juju deleted a comment from Copilot AI Apr 21, 2026
Copy link
Copy Markdown
Collaborator

@freyes freyes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Raven-182 , I left the copilot suggestions that make sense, maybe the one that requires some extra explanation would be the lack of umask, for CIS hardened systems, the default umask is changed , I don't remember the exact value used, it's better we use an explicit permission to be set when rendering the file.

When apache-rate-limit-enabled=true, Apache limits how many HTTP requests
a client IP can make within a time window and returns HTTP 429 when exceeded.

Builds on proxy protocol support (f516806).
HAProxy forwards the real client IP, which mod_remoteip extracts, but
mod_qos cannot use directly. A small Lua script copies it into the
`X-Real-Client-IP` header for mod_qos.

Cluster peer IPs are automatically exempted to avoid throttling health checks.

Limits are enforced per unit (not cluster-wide), so the effective limit
scales with the number of backend units.

The mod_qos module is installed and enabled automatically. Both HTTPS
and WSGI templates are conditionally updated, so charms that do not opt
in are unaffected. In the WSGI template, rate-limiting directives are
only added to the public endpoint.

Signed-off-by: Raven Kaur <raven.kaur@canonical.com>
Assisted-by: Claude Code (claude-sonnet-4-6)
@Raven-182 Raven-182 force-pushed the apache-rate-limiting branch from c1221e9 to e75437c Compare April 21, 2026 14:07
@Raven-182 Raven-182 requested a review from freyes April 21, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants