feat(context): enforce sensitivity policy — drop/redact restricted items from prompts by dgenio · Pull Request #98 · dgenio/contextweaver

dgenio · 2026-03-04T05:28:59Z

Summary

Closes #16.

The Sensitivity enum and ContextPolicy.sensitivity_floor existed but had zero enforcement in the context pipeline. Items tagged restricted or confidential flowed into prompts unchecked — a data leakage risk.

This PR wires sensitivity enforcement into the context compilation pipeline so that items at or above the configured floor are dropped (default) or redacted before reaching the prompt.

What changed

Implementation (`feat` commit)

File	Change
`src/contextweaver/types.py`	Added `sensitivity: Sensitivity = Sensitivity.public` to `ContextItem`; updated `to_dict()`/`from_dict()`
`src/contextweaver/config.py`	Added `sensitivity_action: str = "drop"` to `ContextPolicy`; updated docstring
`src/contextweaver/context/sensitivity.py`	New — `_SENSITIVITY_ORDER`, `MaskRedactionHook`, `apply_sensitivity_filter()`, built-in hook registry
`src/contextweaver/context/manager.py`	Wired `apply_sensitivity_filter()` into `_build()` (step 2b, between dependency closure and firewall); records drops in `BuildStats`
`src/contextweaver/context/__init__.py`	Exports `apply_sensitivity_filter`, `MaskRedactionHook`
`src/contextweaver/__init__.py`	Exports `MaskRedactionHook`
`tests/test_sensitivity.py`	New — 19 tests
`CHANGELOG.md`	Updated `[Unreleased]`

Documentation (`docs` commit)

File	Change
`docs/architecture.md`	Added sensitivity_filter as pipeline step 3 (now 8 steps)
`docs/concepts.md`	Added "Sensitivity Enforcement" section
`AGENTS.md`	Updated context/ pipeline description
`CLAUDE.md`	Added sensitivity.py to module responsibility map
`.github/copilot-instructions.md`	Added sensitivity_filter to pipeline, MaskRedactionHook to key types

How it works

Drop mode (default): items with sensitivity >= floor are removed from candidates before scoring or rendering. BuildStats.dropped_reasons["sensitivity"] records the count.
Redact mode (sensitivity_action="redact"): MaskRedactionHook replaces item text with [REDACTED: {sensitivity}] while preserving all metadata. The item stays in the pipeline.
Configurable via ContextPolicy(sensitivity_floor=Sensitivity.confidential, sensitivity_action="drop"|"redact", redaction_hooks=["mask"]).
Default sensitivity=Sensitivity.public on ContextItem is fully backward-compatible.

Testing performed

ruff format src/ tests/ examples/   → 77 files already formatted
ruff check src/ tests/ examples/    → All checks passed
mypy src/                           → Success: no issues found in 40 source files
pytest tests/ -q --ignore=tests/test_cli.py → 419 passed
pytest tests/test_cli.py -q         → 14 passed

Total: 433 passed, 0 failed.

Test coverage (19 new tests)

Drop mode: restricted/confidential dropped, public/internal pass through
Redact mode: text replaced, metadata preserved
MaskRedactionHook: direct hook tests
Edge cases: empty list, all-dropped, unknown hook name raises ValueError
Serde: to_dict()/from_dict() roundtrip with sensitivity field
Integration: ContextManager.build_sync() excludes/redacts sensitive content

Risks / edge cases

Backward compat: ContextItem.sensitivity defaults to Sensitivity.public — existing code unaffected.
Dependency closure: A sensitive parent pulled in by closure is correctly dropped/redacted (security-correct behavior).
from_dict() missing field: Gracefully defaults to Sensitivity.public for serialized data without the field.

Add sensitivity filtering to the context compilation pipeline. Items whose sensitivity level meets or exceeds ContextPolicy.sensitivity_floor are now dropped (default) or redacted before reaching the prompt. - Add ContextItem.sensitivity field (default: Sensitivity.public) - Add ContextPolicy.sensitivity_action field ('drop' or 'redact') - Add context/sensitivity.py with apply_sensitivity_filter() and MaskRedactionHook (replaces text with '[REDACTED: {sensitivity}]') - Wire sensitivity filter into manager._build() between dependency closure and firewall (step 2b) - Record sensitivity drops in BuildStats.dropped_reasons['sensitivity'] - Export MaskRedactionHook from context/ and top-level __init__.py - Add 19 tests in tests/test_sensitivity.py covering drop mode, redact mode, hook behavior, edge cases, serde roundtrip, and integration Closes #16

…vity enforcement - AGENTS.md: update context/ pipeline description to include sensitivity filter - CLAUDE.md: note sensitivity.py in module responsibility map - .github/copilot-instructions.md: add sensitivity_filter step to pipeline, add MaskRedactionHook to key types - docs/architecture.md: add sensitivity_filter as pipeline step 3 (now 8 steps) - docs/concepts.md: add Sensitivity Enforcement section

Copilot

Pull request overview

This PR closes #16 by enforcing the existing sensitivity policy in the Context Engine so sensitive ContextItems are dropped (default) or redacted before they can reach the rendered prompt, reducing data-leakage risk.

Changes:

Added ContextItem.sensitivity (serde included) and ContextPolicy.sensitivity_action to support drop vs redact behavior.
Introduced contextweaver.context.sensitivity with apply_sensitivity_filter() and a built-in MaskRedactionHook.
Wired sensitivity filtering into ContextManager._build() and added a dedicated test suite + documentation updates.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`src/contextweaver/types.py`	Adds sensitivity to `ContextItem` plus `to_dict()`/`from_dict()` support.
`src/contextweaver/config.py`	Adds `sensitivity_action` and updates policy docs.
`src/contextweaver/context/sensitivity.py`	Implements sensitivity ordering, hook resolution, and drop/redact filtering.
`src/contextweaver/context/manager.py`	Inserts sensitivity filter into the pipeline and updates `BuildStats`.
`src/contextweaver/context/__init__.py`	Re-exports sensitivity helpers.
`src/contextweaver/__init__.py`	Re-exports `MaskRedactionHook` at package level.
`tests/test_sensitivity.py`	Adds unit + integration coverage for drop/redact/serde behavior.
`docs/concepts.md`	Documents sensitivity enforcement behavior and stats.
`docs/architecture.md`	Updates pipeline stages to include sensitivity filtering.
`CHANGELOG.md`	Notes new enforcement and public API changes.
`AGENTS.md` / `CLAUDE.md` / `.github/copilot-instructions.md`	Updates internal docs/instructions to reflect the new step/type.

You can also share your feedback on Copilot code review. Take the survey.

src/contextweaver/context/manager.py

src/contextweaver/context/sensitivity.py

docs/concepts.md

docs/architecture.md

…tats select_and_pack() computes total_candidates from the post-filter list, so adding sensitivity_drops to dropped_count without also adjusting total_candidates broke the invariant: dropped_count + included_count <= total_candidates Now both total_candidates and dropped_count are incremented by sensitivity_drops, keeping BuildStats internally consistent.

Use get(..., 0) + N instead of direct assignment, matching the pattern in selection.py. Defensive against future multi-pass scenarios.

Raise ValueError for unrecognised sensitivity_action values (e.g. typos) instead of silently falling through to drop mode. Consistent with the existing _resolve_hooks() validation pattern in the same module.

…filter The guard checked floor_level > restricted, but restricted is the maximum Sensitivity level, making the branch unreachable dead code.

…hooks

dgenio added 2 commits March 4, 2026 05:25

dgenio requested a review from Copilot March 4, 2026 05:29

Copilot started reviewing on behalf of dgenio March 4, 2026 05:30 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

dgenio added 13 commits March 4, 2026 06:03

fix(context): use accumulation pattern for dropped_reasons[sensitivity]

d669016

Use get(..., 0) + N instead of direct assignment, matching the pattern in selection.py. Defensive against future multi-pass scenarios.

fix(context): validate sensitivity_action to reject unknown values

9d282df

Raise ValueError for unrecognised sensitivity_action values (e.g. typos) instead of silently falling through to drop mode. Consistent with the existing _resolve_hooks() validation pattern in the same module.

refactor(context): remove unreachable fast-path guard in sensitivity …

1c31509

…filter The guard checked floor_level > restricted, but restricted is the maximum Sensitivity level, making the branch unreachable dead code.

docs: fix BuildStats description only drops are recorded, not redactions

369dc5f

docs: fix pipeline stage count seven eight

d60be54

merge: resolve CHANGELOG.md conflict with main

cd60369

chore: remove test_output.txt and add to .gitignore

f0dea92

docs(manager): renumber pipeline steps to include sensitivity filter

8766508

docs(manager): renumber inline step comments 1-8 for consistency

e582989

feat(sensitivity): add register_redaction_hook() for user-extensible …

97e2618

…hooks

style(sensitivity): move _VALID_ACTIONS to module scope

44c9347

fix(manager): shorten docstring line to satisfy E501

81fcf83

dgenio merged commit 12b23ac into main Mar 4, 2026
3 checks passed

dgenio deleted the feat/sensitivity-enforcement branch March 4, 2026 06:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(context): enforce sensitivity policy — drop/redact restricted items from prompts#98

feat(context): enforce sensitivity policy — drop/redact restricted items from prompts#98
dgenio merged 15 commits intomainfrom
feat/sensitivity-enforcement

dgenio commented Mar 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dgenio commented Mar 4, 2026

Summary

What changed

Implementation (feat commit)

Documentation (docs commit)

How it works

Testing performed

Test coverage (19 new tests)

Risks / edge cases

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Implementation (`feat` commit)

Documentation (`docs` commit)