Docs/GitHub token setup

.agent-os/instructions/core/aurea-codex-workflow.md

@@ -0,0 +1,24 @@
+# SYSTEM UPDATE FOR AUREA — TRUTH-FIRST CODEX WORKFLOW (NON-BYPASSABLE)
+
+**Scope:** Do not edit files directly. Output only: PLAN → DIFFS → COMMANDS → TESTS → VERIFICATION → ROLLBACK.
+
+**Contracts to enforce:**
+- NON SIMULATION CONTRACT (no persona/emotion/trust simulation). 
+- Truth-first engineering (separate DATA/REASONING/SPECULATION; surface contradictions; use ASSUMPTION when needed). 
+- Rights & Safety: refuse unsafe or rights-violating actions; propose safe alternatives. 
+
+**Output format (always):**
+1. PLAN (≤8 steps, risks/unknowns)
+2. DIFFS (fenced ```diff blocks with correct paths)
+3. COMMANDS (Codex-executable shell lines incl. git)
+4. TESTS (unit/integration proving acceptance)
+5. VERIFICATION (local/CI checklist)
+6. ROLLBACK (safe revert)
+
+**Determinism & Safety:**
+- Idempotent commands; no destructive ops without gating/dry-run.
+- Secrets via env vars only; `.env.example` updates if needed.
+- Explicit failure messages; halt on contradictions.
+
+**Acceptance criteria:** Each task must define clear, testable bullets that would fail pre-change and pass post-change.
+

.agent-os/instructions/core/codex-runbook.md

@@ -0,0 +1,31 @@
+# CODEX RUNBOOK — APPLY PATCHES WITH TRUTH KERNEL SAFETY
+
+**Objective:** Apply Aurea’s DIFFS exactly; validate with audit + tests; commit and push.
+
+## Steps (strict order)
+1) **Repo prep**
+   - `git status --porcelain`
+   - `git pull --rebase`
+   - `git checkout -b feat/<slug> || git switch feat/<slug>`
+2) **Apply patches**
+   - For each ```diff block from Aurea, apply using:
+     - `git apply --index --reject`  # fail on rejects
+   - Ensure any new directories exist before apply.
+3) **Install & build**
+   - `npm ci`
+   - `npm run build || true`  # only if a build script exists
+4) **Static checks & tests**
+   - `npm test`
+   - `node backend/scripts/audit-truth.cjs`
+5) **Dry-run and (if required) safe execute**
+   - If `sf` exists: `sf dry`  # never run `sf apply --apply` unless explicitly requested
+6) **Commit & push**
+   - `git add -A`
+   - `git commit -m "feat: <concise summary of change>"`
+   - `git push -u origin HEAD`
+7) **PR**
+   - Create PR titled `feat: <summary> (TruthKernel)`
+   - PR body includes acceptance-criteria checklist and last audit/test outputs.
+
+**Failure policy:** On ANY failure (apply/test/audit), stop and post the failing command, exit code, and last 120 lines of logs. No partial commits.
+

.agent-os/instructions/core/github-token-setup.md

@@ -0,0 +1,80 @@
+# GitHub Token Setup (Ubuntu) — for PR helper & batch merges
+
+This guide shows how to create a GitHub Personal Access Token (PAT) and export it as `GITHUB_TOKEN` without leaking it into shell history.
+
+## 0) Create a PAT
+1. Go to GitHub → **Settings** → **Developer settings**.
+2. Choose one:
+   - **Fine-grained token** → grant access to the **mrhpython/Soulfield** repo with read/write on **Pull requests** and **Contents**.
+   - **Classic token** → select **repo** scope.
+3. Copy the token once (looks like `ghp_…` or `github_pat_…`). Keep it private.
+
+---
+
+## 1) Export the token (safe, no history)
+Run this in your terminal (it won’t echo the token and won’t store it in history):
+```bash
+unset GITHUB_TOKEN 2>/dev/null
+read -s -p "Paste GitHub token (hidden): " GITHUB_TOKEN; echo
+export GITHUB_TOKEN
+```
+> `read -s` hides input, and exporting afterward avoids recording the token inline in shell history.
+
+**Verify (does not print the token):**
+```bash
+if [ -n "$GITHUB_TOKEN" ]; then echo "GITHUB_TOKEN set (length: ${#GITHUB_TOKEN})"; else echo "GITHUB_TOKEN missing"; fi
+```
+
+**Optional (persist across terminals, private file outside the repo):**
+```bash
+mkdir -p ~/.config/soulfield
+chmod 700 ~/.config/soulfield
+printf 'export GITHUB_TOKEN=%s\n' "$GITHUB_TOKEN" > ~/.config/soulfield/env
+chmod 600 ~/.config/soulfield/env
+echo 'source ~/.config/soulfield/env' >> ~/.bashrc
+```
+> Do **not** commit any token to the repo. The file above lives in your home dir, not tracked by git.
+
+---
+
+## 2) Quick dry-run (no network)
+```bash
+DRY=1 bash tools/merge-three-prs.sh
+```
+Expected: prints the three PR steps and exits 0 even without a token.
+
+---
+
+## 3) Real merges with helper (token required)
+**One-shot (all three PRs):**
+```bash
+export GITHUB_TOKEN=$GITHUB_TOKEN   # ensure it is still set
+bash tools/merge-three-prs.sh
+```
+
+**Stepwise:**
+```bash
+node tools/gh-pr-open-and-merge.cjs --repo mrhpython/Soulfield --base main --head fix/audit-hyphenated-identifiers --title "fix(audit): ignore code blocks; tighten pronoun detection" --body "Refines simulation detection to skip fenced/inline code and avoid hyphen/underscore-bound identifiers. Adds unit tests." --labels "truth-kernel,audit"
+
+node tools/gh-pr-open-and-merge.cjs --repo mrhpython/Soulfield --base main --head test/node-runner --title "test: switch to Node built-in runner; convert audit tests" --body "Replaces Mocha semantics with node:test. Updates package.json test script. Ensures zero-deps test execution in CI." --labels "tests,truth-kernel"
+
+git switch docs/truthlens-readme-status && git fetch origin && git rebase origin/main
+npm ci && npm test && node backend/scripts/audit-truth.cjs
+
+node tools/gh-pr-open-and-merge.cjs --repo mrhpython/Soulfield --base main --head docs/truthlens-readme-status --title "docs: TruthLens runtime + audit; compliance quick checks; Codex workflow" --body "README adds TruthLens + CI gate overview; STATUS adds compliance quick checks; TruthLens policy page gains canonical pointers." --labels "docs,truth-kernel"
+```
+
+---
+
+## 4) Cleanup
+When finished (especially on shared machines):
+```bash
+unset GITHUB_TOKEN
+```
+
+---
+
+## Troubleshooting
+- **“GITHUB_TOKEN not set”** → re-run the export step or open a new terminal after adding `source ~/.config/soulfield/env` to `~/.bashrc`.
+- **“CI failed”** from helper → open the PR page, read failed job logs, push a fix to the branch, re-run the helper.
+

.agent-os/instructions/core/pr-merge-runbook.md

@@ -0,0 +1,90 @@
+# PR Merge Runbook — Truth Kernel Branches
+
+Purpose: deterministically open and merge our three current branches using either the GitHub UI or the repo’s helper script. This file is the canonical reference for future threads.
+
+## Branches
+1. **PR 1** (audit fix) → `fix/audit-hyphenated-identifiers` → `main`
+2. **PR 2** (node test runner) → `test/node-runner` → `main`
+3. **PR 3** (docs) → `docs/truthlens-readme-status` → `main` (after rebasing on merged PR1+PR2)
+
+---
+
+## Option A — GitHub UI (simplest)
+1. Open each “Compare & pull request” link for the three branches.
+2. Titles/Bodies:
+   - PR 1 — **Title:** `fix(audit): ignore code blocks; tighten pronoun detection`  
+     **Body:** Refines simulation detection to skip fenced/inline code and avoid hyphen/underscore-bound identifiers. Adds unit tests.
+   - PR 2 — **Title:** `test: switch to Node built-in runner; convert audit tests`  
+     **Body:** Replaces Mocha semantics with node:test. Updates package.json test script. Ensures zero-deps test execution in CI.
+   - PR 3 — **Title:** `docs: TruthLens runtime + audit; compliance quick checks; Codex workflow`  
+     **Body:** README adds TruthLens + CI gate overview; STATUS adds compliance quick checks; TruthLens policy page gains canonical pointers.
+3. Labels:
+   - PR 1: `truth-kernel`, `audit`
+   - PR 2: `tests`, `truth-kernel`
+   - PR 3: `docs`, `truth-kernel`
+4. Merge order:
+   - Merge PR 1 → Merge PR 2 → Rebase `docs/truthlens-readme-status` on `main`, then open & merge PR 3.
+
+---
+
+## Option B — Helper script (no gh CLI)
+**Prereq:** create a GitHub Personal Access Token (repo scope) and export it. See:
+`.agent-os/instructions/core/github-token-setup.md`
+
+Quick export (hidden input):
+```bash
+unset GITHUB_TOKEN 2>/dev/null
+read -s -p "Paste GitHub token (hidden): " GITHUB_TOKEN; echo
+export GITHUB_TOKEN
+```
+
+Run these commands from the repo root (Ubuntu).
+
+### 1) PR 1 — audit fix
+```bash
+node tools/gh-pr-open-and-merge.cjs \
+  --repo mrhpython/Soulfield \
+  --base main \
+  --head fix/audit-hyphenated-identifiers \
+  --title "fix(audit): ignore code blocks; tighten pronoun detection" \
+  --body "Refines simulation detection to skip fenced/inline code and avoid hyphen/underscore-bound identifiers. Adds unit tests." \
+  --labels "truth-kernel,audit"
+```
+### 2) PR 2 — node test runner
+```bash
+node tools/gh-pr-open-and-merge.cjs \
+  --repo mrhpython/Soulfield \
+  --base main \
+  --head test/node-runner \
+  --title "test: switch to Node built-in runner; convert audit tests" \
+  --body "Replaces Mocha semantics with node:test. Updates package.json test script. Ensures zero-deps test execution in CI." \
+  --labels "tests,truth-kernel"
+```
+### 3) Rebase docs branch after PR 1 & 2 merge
+```bash
+git switch docs/truthlens-readme-status
+git fetch origin
+git rebase origin/main
+npm ci && npm test && node backend/scripts/audit-truth.cjs
+```
+### 4) PR 3 — docs
+```bash
+node tools/gh-pr-open-and-merge.cjs \
+  --repo mrhpython/Soulfield \
+  --base main \
+  --head docs/truthlens-readme-status \
+  --title "docs: TruthLens runtime + audit; compliance quick checks; Codex workflow" \
+  --body "README adds TruthLens + CI gate overview; STATUS adds compliance quick checks; TruthLens policy page gains canonical pointers." \
+  --labels "docs,truth-kernel"
+```
+
+---
+
+## Notes
+- TruthLens is the governing policy; all outputs are wrapped at runtime:contentReference[oaicite:5]{index=5} and documented as OS intent:contentReference[oaicite:6]{index=6}.
+- Non-simulation contract applies to all agent outputs:contentReference[oaicite:7]{index=7}.
+- CI gate runs the Truth audit + tests on every push/PR.
+
+## Troubleshooting
+- If the helper reports “CI failed”, click into the PR checks to view logs. Fix locally, push to the branch, rerun the helper.
+- If `git rebase` reports conflicts, resolve locally, `git add <files> && git rebase --continue`, then rerun the PR 3 helper command.

.agent-os/specs/1757008963219-jina-ai-for-retrieval-rerank-over-workspace-docs-https-do/spec.md

@@ -0,0 +1,64 @@
+# spec.md
+
+## Intent
+Build a UK-focused semantic search API that integrates Jina AI's embedding and reranking services with TruthLens verification, enabling RAG-powered document search with fact-checking capabilities for workspace knowledge bases.
+
+## Acceptance Criteria
+1. Successfully index 100+ workspace documents via Jina Reader API
+2. Search response time <300ms for 95% of queries
+3. Rerank top-10 results with >85% relevance accuracy
+4. TruthLens confidence scores correlate with rerank scores (r>0.7)
+5. API handles 100 concurrent requests without degradation
+6. All responses include source citations and confidence metrics
+
+## Thin-Slice MVP
+A REST API endpoint that accepts a search query, embeds it using Jina, searches a vector store, reranks results, and returns TruthLens-verified responses with citations.
+
+```
+POST /api/v1/search
+{
+  "query": "How does TruthLens verify facts?",
+  "limit": 5,
+  "verify": true
+}
+→ Returns ranked, verified results with confidence scores
+```
+
+## Tasks (≤12)
+
+### Setup [2h]
+1. Configure Jina API credentials and rate limits in `.env`
+2. Initialize PostgreSQL with pgvector extension for embeddings
+
+### Core Implementation [8h]
+3. Create `JinaAdapter` class with embed/rerank methods
+4. Build document chunker (1000 tokens, 200 overlap)
+5. Implement batch document indexer with progress tracking
+6. Create vector search function with similarity threshold
+7. Build reranking pipeline for top-20 candidates
+8. Implement TruthLens verification against indexed sources
+
+### API & Integration [4h]
+9. Create FastAPI endpoint with request/response models
+10. Add caching layer for frequent embeddings (Redis TTL=1h)
+11. Implement async processing for parallel operations
+
+### Testing & Deploy [2h]
+12. Write integration tests with mock documents and deploy to Railway/Render
+
+## Week 1 Metrics
+- **Indexing Volume**: 500+ documents processed
+- **Query Latency**: P95 <300ms, P50 <150ms
+- **Relevance Score**: >85% for test query set
+- **API Uptime**: >99% availability
+- **Cost Efficiency**: <£50 total API spend
+- **User Feedback**: 10+ beta testers provide relevance ratings
+
+## Run (approved)
+echo "[spec] sanity checks"
+ls -alh /home/michael/soulfield/workspace/specs
+echo "[policy] show TruthLens rules"
+head -n 20 /home/michael/soulfield/workspace/knowledge/TruthLens.md
+echo "[index] first 10 lines"
+head -n 10 /home/michael/soulfield/workspace/data/index.json
+

.agent-os/standards/CODEX RUNBOOK APPLY PATCHES.txt

@@ -0,0 +1,37 @@
+CODEX RUNBOOK — APPLY PATCHES WITH TRUTH KERNEL SAFETY
+
+Objective: Apply Aurea’s DIFFS exactly; validate with audit + tests; commit and push.
+
+Steps (strict order):
+1) Repo prep
+   - git status --porcelain
+   - git pull --rebase
+   - git checkout -b feat/<slug> || git switch feat/<slug>
+
+2) Apply patches
+   - For each ```diff block from Aurea, apply using `git apply --index --reject` (fail on rejects).
+   - If new files: ensure directories exist before apply.
+
+3) Install & build
+   - npm ci
+   - npm run build || true  # only if build script exists
+
+4) Static checks & tests
+   - npm test
+   - node backend/scripts/audit-truth.cjs
+
+5) Dry-run and (if required) safe execute
+   - if `sf` exists: `sf dry`  # do not run `sf apply --apply` unless explicitly requested
+
+6) Commit & push
+   - git add -A
+   - git commit -m "feat: <concise summary of change>"
+   - git push -u origin HEAD
+
+7) PR (if platform supports it)
+   - Create PR titled "feat: <summary> (TruthKernel)"
+   - PR body: include checklist of acceptance criteria and audit/test outputs.
+
+Failure policy:
+- On ANY failure (apply/test/audit), stop and post the failing command, exit code, and last 120 lines of logs. No partial commits.
+

.github/pull_request_template.md

@@ -1,20 +1,12 @@
-Title: <short summary>
+## Summary
+Describe the change in one or two sentences.
 
-Summary
-- What changed and why (1–3 lines).
+## Checklist (Truth Kernel)
+- [ ] Local: `npm ci`
+- [ ] Local: `node backend/scripts/audit-truth.cjs` (no violations)
+- [ ] Local: `npm test` (Node’s built-in runner)
+- [ ] Docs updated if behavior changes (README / STATUS / Knowledge)
 
-Checklist
-- [ ] Scope is minimal and focused
-- [ ] CI green locally (if applicable)
-- [ ] Tests added/updated or not applicable
-- [ ] Docs updated or not applicable
-
-Validation
-- Steps to verify manually:
-  1. 
-  2. 
-
-Links
-- Related issue: #
-- Context/spec: 
+## Notes
+Link to related specs, runs, or artifacts if relevant.