RFC: OpenPolicyAgent (OPA) Integration for Pod Authorization

[dciangot]

RFC: OpenPolicyAgent (OPA) Integration for Pod Authorization

Summary

This RFC proposes integrating OpenPolicyAgent (OPA) into interLink to provide fine-grained authorization control over which pods are allowed to execute on remote resources. This would enable policy-driven decision making for pod scheduling and execution based on various attributes like user identity, namespace, resource requirements, labels, annotations, and more.

Background

Currently, interLink provides authentication through OIDC/OAuth2 integration but lacks fine-grained authorization controls beyond basic authentication. Pod execution decisions are primarily based on:

1. Authentication: OIDC/OAuth2 token validation
2. Basic resource allocation: Node capacity and taints/tolerations
3. Kubernetes RBAC: Standard K8s permissions for pod creation

However, there's no mechanism to implement custom policies for:

• Workload classification and routing
• Security policy compliance
• Multi-tenancy isolation

Proposed Solution

Architecture Overview

The OPA integration would be implemented as a policy evaluation layer within the interLink API server, specifically in the pod creation flow:

Virtual Kubelet --> interLink API Server --> Plugin (SLURM/etc)
                           |
                           v
                    OPA Engine + Policies

Integration Points

Pod Creation Authorization (Primary)

• Location: pkg/interlink/api/create.go - CreateHandler function
• Decision Point: Before forwarding pod creation to sidecar
• Input Context: Full pod specification, user identity, namespace metadata
• Policy Decision: Allow/Deny pod execution with optional modifications

Implementation Design

Configuration Structure

# InterLinkConfig.yaml
authorization:
  enabled: true
  type: "opa"
  opa:
    endpoint: "http://localhost:8181"
    policy_path: "/v1/data/interlink/authz"
    timeout: "5s"
    enable_decision_logs: true
    fallback_policy: "deny"  # deny|allow

Policy Input Schema

{
  "input": {
    "pod": {
      "metadata": {},
      "spec": {}
    },
    "user": {
      "username": "string",
      "groups": ["string"],
      "extra": {}
    },
    "namespace": {
      "name": "string",
      "labels": {},
      "annotations": {}
    },
    "node": {
      "name": "interlink-node",
      "labels": {},
      "available_resources": {}
    },
    "operation": "create|delete|status"
  }
}

Policy Decision Schema

{
  "result": {
    "allowed": true,
    "reason": "Policy evaluation passed",
    "modifications": {
      "pod_annotations": {
        "billing.project": "research-team-a"
      },
      "resource_limits": {
        "cpu": "4",
        "memory": "8Gi"
      }
    }
  }
}

Example Policies

1. Resource Quota by User Group

package interlink.authz

default allowed = false

allowed {
    input.user.groups[_] == "gpu-users"
    requires_gpu
    gpu_quota_available
}

requires_gpu {
    input.pod.spec.containers[_].resources.limits["nvidia.com/gpu"] > 0
}

gpu_quota_available {
    # Check against external quota system
    count_user_running_gpu_pods < 10
}

2. Multi-tenant Isolation

package interlink.authz

default allowed = false

allowed {
    input.namespace.labels["tenant"] == input.user.tenant
    input.pod.spec.nodeSelector["node-pool"] == input.user.tenant
}

Implementation Steps

Phase 1: Core OPA Integration

1. Add OPA client library to interLink dependencies
2. Implement authorization middleware in pkg/interlink/api/create.go
3. Add configuration parsing for OPA settings
4. Create policy input context builder
5. Implement policy decision enforcement

Phase 2: Advanced Features

1. Policy decision logging and auditing
2. Policy reload without restart
3. Caching for policy decisions
4. Integration with external data sources (quotas, billing)
5. Metrics and monitoring for policy decisions

Phase 3: Enhanced Integration Points

1. Authorization for status and log access
2. Dynamic resource allocation based on policies
3. Policy-driven job scheduling hints
4. Integration with Kubernetes ValidatingAdmissionWebhooks

Migration Strategy

1. Backward Compatibility: Authorization disabled by default
2. Gradual Rollout: Start with audit-only mode for policy testing
3. Fallback Mechanisms: Configure fallback behavior when OPA is unavailable
4. Documentation: Comprehensive policy examples and best practices

Security Considerations

1. Policy Security: Secure policy storage and management
2. Input Validation: Sanitize and validate all policy inputs
3. Least Privilege: Default deny with explicit allow policies
4. Audit Logging: Comprehensive audit trail of all authorization decisions
5. Performance: Ensure authorization doesn't significantly impact latency

Future Extensions

1. Dynamic Policies: Policies based on real-time cluster state
2. Cross-cluster Policies: Authorization across multiple interLink deployments
3. Policy Testing: Built-in policy simulation and testing tools

Questions for Discussion

1. Should we support multiple policy engines or focus exclusively on OPA?
2. What's the preferred approach for policy distribution and management?
3. Should authorization be optional or required for production deployments?
4. How should we handle policy evaluation failures and performance concerns?
5. What external data sources should be supported for policy evaluation?
6. Should we provide a default set of common policies out-of-the-box?

Next Steps

1. Gather community feedback on this RFC
2. Create detailed technical design document
3. Implement proof-of-concept with basic OPA integration
4. Develop policy examples and best practices guide
5. Plan phased implementation and migration strategy

---

Note: This RFC is open for community discussion. Please share your thoughts on the proposed approach, implementation details, and any concerns or suggestions you may have.