Skip to content

Add benchmarks and documentation for IAM policy performance characteristics#1694

Open
Copilot wants to merge 4 commits intomasterfrom
copilot/optimize-iam-policies
Open

Add benchmarks and documentation for IAM policy performance characteristics#1694
Copilot wants to merge 4 commits intomasterfrom
copilot/optimize-iam-policies

Conversation

Copy link
Contributor

Copilot AI commented Jan 25, 2026

Users reported significant performance degradation when using explicit allow/deny effect fields (p_eft) versus separate policy types for IAM-like authorization systems.

Analysis

The ~2x performance difference is fundamental to the evaluation semantics:

  • AllowOverrideEffect (no p_eft): Short-circuits on first matching allow. Average N/2 policy evaluations.
  • AllowAndDenyEffect (with p_eft): Must evaluate all N policies since deny can appear anywhere.

This is correct behavior for AWS IAM-like semantics, not a performance bug.

Changes

Benchmarks (iam_optimization_b_test.go):

  • BenchmarkIAMWithoutEffectField / BenchmarkIAMWithEffectField
  • Large dataset variants (1K-5K policies)
  • Results: 2.03x slower with explicit effects (259µs → 528µs @ 1K policies)

Documentation (docs/IAM_POLICY_OPTIMIZATION.md):

  • Performance analysis with benchmark data
  • Decision matrix: when to use each approach
  • Alternative patterns (PriorityEffect for ordered evaluation)

Example

// Option 1: Fast, simple allow-only (short-circuits)
[policy_effect]
e = some(where (p.eft == allow))

// Option 2: IAM-like explicit deny (checks all policies)
[policy_effect]  
e = some(where (p.eft == allow)) && !some(where (p.eft == deny))

Use Option 1 for performance-critical allow-only systems. Use Option 2 when deny semantics are required and 2x overhead is acceptable.

Original prompt

This section details on the original issue you should resolve

<issue_title>[Question] Optimization for IAM-like policies</issue_title>
<issue_description>Hello,
I'm trying to design an AWS IAM like system where we have a set of resources which you can "allow" or "deny" for certain roles and users.

My Initial was this:

Initial Option: Model
[request_definition]
r = sub, dom, obj, act

[policy_definition]
p = sub, obj, act, eft

[role_definition]
g = _, _, _ # Roles permission
g2 = _, _ # Data group
g3 = _, _ # Action group

[policy_effect]
# Allow as soon as there one rule allowing
#e = some(where (p.eft == allow))
#Deny as soon as there is one rule denying
e = some(where (p.eft == allow)) && !some(where (p.eft == deny))

[matchers]
m = g3(r.act, p.act) && g(r.sub, p.sub, r.dom) && keyMatch5(r.obj, p.obj)
Initial Option: Policy
# sub/perm, resource, action
p, perm:instance:list, /instance, READ, allow
p, perm:instance:delete, /instance/{name}, DELETE, allow
p, perm:instance:describe, /instance/{name}, READ, allow
p, perm:instance:create, /instance/{name}, POST, allow

p, perm:service:list, /service, READ, allow

p, perm:proxy:read, /proxy/{name}/*, READ, allow
p, perm:proxy:write, /proxy/{name}/*, WRITE, allow
p, perm:proxy:crashes, /proxy/{name}/crashes, READ, allow
p, block:proxy:crashes, /proxy/{name}/crashes, READ, deny

# Adding wild cards in the group does seem to add a lot of time when users grow
#
#g, role:instance:admin, perm:instance:*, *
#g, role:service:admin, perm:service:*, *
#g, role:proxy:admin, perm:proxy:*, *
#
g, role:instance:admin, perm:instance:list, *
g, role:instance:admin, perm:instance:delete, *
g, role:instance:admin, perm:instance:describe, *
g, role:instance:admin, perm:instance:create, *

g, role:service:admin, perm:service:list, *

g, role:proxy:admin, perm:proxy:read, *
g, role:proxy:admin, perm:proxy:write, *

g, role:instance:viewer, perm:instance:list, *
g, role:instance:viewer, perm:instance:describe, *

g, role:service:viewer, perm:service:list, *

g, role:proxy:viewer, perm:proxy:read, *
g, role:proxy:viewer, block:proxy:crashes, *

g, user:admin, role:instance:admin, *
g, user:admin, role:service:admin, *
g, user:admin, role:proxy:admin, *

g, user:alice, role:instance:admin, orgA
g, user:alice, role:service:admin, orgA
g, user:alice, role:proxy:admin, orgA

g, user:alice, role:instance:viewer, orgB
g, user:alice, role:service:viewer, orgB
g, user:alice, role:proxy:viewer, orgB

g, user:bob, role:instance:viewer, orgB
g, user:bob, role:service:viewer, orgB
g, user:bob, role:proxy:viewer, orgB

g, user:nancy, role:proxy:admin, orgA

g3, POST, WRITE
g3, PUT, WRITE
g3, PATCH, WRITE
g3, DELETE, WRITE

g3, OPTIONS, READ
g3, HEAD, READ
g3, GET, READ

g3, READ, ANY
g3, WRITE, ANY

I find it's not ideal because I would need to duplicate every p to have one for deny and one for allow.

So My second option is this:

Tentative Option; Model
[request_definition]
r = sub, dom, obj, act

[policy_definition]
p = sub, obj, eft, dom

[role_definition]
g = _, _, _ # Roles permission
g2 = _, _, _ # Data group
g3 = _, _ # Action group

[policy_effect]
# Allow as soon as there one rule allowing
#e = some(where (p.eft == allow))
#Deny as soon as there is one rule denying
e = some(where (p.eft == allow)) && !some(where (p.eft == deny))

[matchers]
m = globMatch(r.dom, p.dom) && g(r.sub, p.sub, r.dom) && g2(r.obj, p.obj, r.act)
Tentative Option; Policy
#
p, role:instance:admin, perm:instance:list, allow, *
p, role:instance:admin, perm:instance:delete, allow, *
p, role:instance:admin, perm:instance:describe, allow, *
p, role:instance:admin, perm:instance:create, allow, *
p, role:service:admin, perm:service:list, allow, *
p, role:proxy:admin, perm:proxy:read, allow, *
p, role:proxy:admin, perm:proxy:write, allow, *
p, role:instance:viewer, perm:instance:list, allow, *
p, role:instance:viewer, perm:instance:describe, allow, *
p, role:service:viewer, perm:service:list, allow, *
p, role:proxy:viewer, perm:proxy:read, allow, *
p, role:proxy:viewer, perm:proxy:crashes, deny, *
# sub/perm, resource, action
g2, /instance, perm:instance:list, GET
g2, /instance/{name}, perm:instance:delete, DELETE
g2, /instance/{name}, perm:instance:describe, GET
g2, /instance/{name}, perm:instance:create, POST
g2, /service, perm:service:list, GET
g2, /proxy/{name}/*, perm:proxy:read, GET
g2, /proxy/{name}/*, perm:proxy:write, POST
g2, /proxy/{name}/crashes, perm:proxy:crashes, GET


g, user:admin, role:instance:admin, *
g, user:admin, role:service:admin, *
g, user:admin, role:proxy:admin, *

g, user:alice, role:instance:admin, orgA
g, user:alice, role:service:admin, orgA
g, user:alice, role:proxy:admin, orgA

g, user:alice, role:instance:viewer, orgB
g, user:alice, role:service:viewer, orgB
g, user:alice...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes casbin/casbin#1688

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI and others added 3 commits January 25, 2026 03:52
Co-authored-by: hsluoyz <3787410+hsluoyz@users.noreply.github.com>
Co-authored-by: hsluoyz <3787410+hsluoyz@users.noreply.github.com>
Co-authored-by: hsluoyz <3787410+hsluoyz@users.noreply.github.com>
Copilot AI changed the title [WIP] Optimize IAM-like policy system design Add benchmarks and documentation for IAM policy performance characteristics Jan 25, 2026
Copilot AI requested a review from hsluoyz January 25, 2026 03:58
@hsluoyz hsluoyz marked this pull request as ready for review January 25, 2026 04:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants