From 63f9bfc33f9dd82987dface895c5d865478aec54 Mon Sep 17 00:00:00 2001 From: Affaan Mustafa Date: Wed, 13 May 2026 00:38:18 -0400 Subject: [PATCH] docs: gate ECC progress sync readiness Make the ECC 2.0 GitHub/Linear/handoff/roadmap progress-sync model part of the local observability readiness gate instead of leaving it as roadmap prose only. - add `docs/architecture/progress-sync-contract.md` for GitHub, Linear, handoff, roadmap, and work-items sync - add a `Tracker Sync` check to `scripts/observability-readiness.js` - update observability tests with passing and missing-contract coverage - update observability and GA roadmap docs so the local readiness gate is now 18/18 and records #1848 supply-chain hardening evidence Validation: - node tests/scripts/observability-readiness.test.js (9 passed, 0 failed) - npm run observability:ready -- --format json (18/18, ready true) - npx markdownlint-cli 'docs/architecture/progress-sync-contract.md' 'docs/architecture/observability-readiness.md' 'docs/ECC-2.0-GA-ROADMAP.md' - git diff --check - node tests/docs/ecc2-release-surface.test.js (18 passed) - node tests/run-all.js (2378 passed, 0 failed) - GitHub CI for #1849 green across Ubuntu, Windows, and macOS No release, tag, npm publish, plugin tag, marketplace submission, or announcement was performed. --- docs/ECC-2.0-GA-ROADMAP.md | 30 ++++++--- docs/architecture/observability-readiness.md | 10 ++- docs/architecture/progress-sync-contract.md | 67 +++++++++++++++++++ scripts/observability-readiness.js | 37 ++++++++++ tests/scripts/observability-readiness.test.js | 28 ++++++++ 5 files changed, 159 insertions(+), 13 deletions(-) create mode 100644 docs/architecture/progress-sync-contract.md diff --git a/docs/ECC-2.0-GA-ROADMAP.md b/docs/ECC-2.0-GA-ROADMAP.md index ae05626a..84a77b63 100644 --- a/docs/ECC-2.0-GA-ROADMAP.md +++ b/docs/ECC-2.0-GA-ROADMAP.md @@ -30,10 +30,15 @@ As of 2026-05-13: Linear project status updates remain the active tracking surfaces until the workspace is upgraded or issue capacity is freed. - `npm run harness:audit -- --format json` reports 70/70 on current `main`. -- `npm run observability:ready` reports 16/16 readiness on current `main`. +- `npm run observability:ready` reports 18/18 readiness on current `main`, + including the GitHub/Linear/handoff/roadmap progress-sync contract. - PR #1846 merged as `797f283036904128bb1b348ae62019eb9f08cf39` and made npm registry signature verification a durable workflow-security gate: workflows that run `npm audit` now need `npm audit signatures`. +- PR #1848 merged as `cbecf5689d8d1bd5915e7031697a1d56aac538f2` and added + `docs/security/supply-chain-incident-response.md`, plus a workflow-security + validator rule blocking `pull_request_target` workflows from restoring or + saving shared dependency caches. - `docs/architecture/harness-adapter-compliance.md` maps Claude Code, Codex, OpenCode, Cursor, Gemini, Zed-adjacent, dmux, Orca, Superset, Ghast, and terminal-only support to install paths, verification commands, and risk @@ -56,6 +61,8 @@ As of 2026-05-13: release-readiness evidence refresh: 70/70 harness audit, adapter compliance PASS, 16/16 observability readiness, 2376/2376 root Node tests, markdownlint, release-surface and npm publish-surface tests, and 462/462 `ecc2` Rust tests. +- After #1848, `node tests/run-all.js` reports 2377/2377 and the current + observability gate reports 18/18. - A detached clean worktree at `bfacf37715b39655cbc2c48f12f2a35c67cb0253` verified Claude plugin tag dry-run without `--force`, local marketplace discovery, temp-home local @@ -218,10 +225,10 @@ is not complete unless the evidence column exists and has been freshly verified. | Prompt requirement | Required artifact or gate | Current evidence | Status | | --- | --- | --- | --- | -| Keep public PRs below 20 | Repo-family PR recheck | 0 open PRs across the tracked public repos on 2026-05-13 after merging #1846 | Complete for this checkpoint | +| Keep public PRs below 20 | Repo-family PR recheck | 0 open PRs across the tracked public repos on 2026-05-13 after merging #1848 | Complete for this checkpoint | | Keep public issues below 20 | Repo-family issue recheck | 0 open issues across the tracked public repos on 2026-05-13 | Complete for this checkpoint | | Manage repository discussions | Repo-family discussion recheck | Latest trunk discussion GraphQL sweep returned closed discussions only; satellite repos remain disabled or empty | Complete for this checkpoint | -| Manage PR discussions | PR review/comment closure plus merge/close state | #1846 merged after current-head CI; no open PRs remain | Complete for this checkpoint | +| Manage PR discussions | PR review/comment closure plus merge/close state | #1848 merged after current-head CI; no open PRs remain | Complete for this checkpoint | | Salvage useful stale work | `docs/stale-pr-salvage-ledger.md` | Ledger records salvaged, superseded, skipped, and manual-review tails; #1815-#1818 added cost tracking, skill scout, frontend design guidance, code-reviewer false-positive guardrails, and the May 12 gap pass | Complete except translation/manual review tail | | ECC 2.0 preview pack ready | Release docs, quickstart, publication readiness, release notes | `docs/releases/2.0.0-rc.1/` and readiness docs are in-tree; May 13 evidence refresh records harness, adapter, observability, Node, lint, release-surface, npm publish-surface, and Rust checks | Needs final clean-checkout release approval | | Hermes specialized skills included safely | Hermes setup/import docs and sanitized skill surface | Hermes setup and import playbook are public; secrets stay local | Needs final release review | @@ -230,20 +237,21 @@ is not complete unless the evidence column exists and has been freshly verified. | Articles, tweets, and announcements | X thread, LinkedIn copy, GitHub release copy, push checklist | Draft launch collateral exists under rc.1 release docs | Needs URL-backed refresh | | AgentShield enterprise iteration | Policy gates, SARIF, packs, provenance, corpus, HTML reports, exception lifecycle audit, baseline drift Action/CLI surfaces, enterprise research roadmap | PRs #53, #55-#64 landed with test evidence; native PDF export deferred in favor of self-contained HTML plus print-to-PDF until explicit enterprise demand appears; `docs/architecture/agentshield-enterprise-research-roadmap.md` selects baseline drift as the first control-plane slice | Baseline-drift Action and CLI write surfaces landed; evidence-pack routing remains | | ECC Tools next-level app | Billing audit, PR checks, deep analyzer, sync backlog, evaluator/RAG corpus | PRs #26-#40 landed with test evidence | Needs capacity-backed Linear rollout | -| GitGuardian/Dependabot/CodeRabbit-style checks | Non-blocking taxonomy and deterministic follow-up checks | ECC-Tools risk taxonomy check plus follow-up signals landed, including Skill Quality, Deep Analyzer Evidence, Analyzer Corpus Evidence, RAG/Evaluator Evidence, and PR Review/Salvage Evidence | Partially complete | +| GitGuardian/Dependabot/CodeRabbit-style checks | Non-blocking taxonomy, deterministic follow-up checks, and local supply-chain gates | ECC-Tools risk taxonomy check plus follow-up signals landed, including Skill Quality, Deep Analyzer Evidence, Analyzer Corpus Evidence, RAG/Evaluator Evidence, and PR Review/Salvage Evidence; #1846 added npm registry signature gates; #1848 added the supply-chain incident-response playbook and `pull_request_target` cache-poisoning validator guard | Partially complete | | Harness-agnostic learning system | Audit, adapter matrix, observability, traces, promotion loop | Audit/adapters/observability gates plus `docs/architecture/evaluator-rag-prototype.md`, `examples/evaluator-rag-prototype/`, and ECC-Tools PR #40 define read-only stale-salvage, billing-readiness, CI-failure-diagnosis, harness-config-quality, AgentShield policy-exception, skill-quality evidence, deep-analyzer evidence, and RAG/evaluator comparison scenarios with trace, report, playbook, verifier, and predictive-check artifacts | Local corpus complete; hosted integration remains future | | Linear roadmap is detailed | Linear project status plus repo mirror | Repo mirror exists; issue creation was retried on 2026-05-12 and remains blocked by the workspace free issue limit | Needs recurring status updates after each merge batch | -| Flow separation and progress tracking | Flow lanes with owner artifacts and update cadence | This roadmap defines lanes below | Active | -| Realtime Linear sync | Project updates while issue limit is blocked; issues later | ECC-Tools #39 implements opt-in Linear API sync for deferred follow-up backlog items | Needs workspace capacity/config rollout | -| Observability for self-use | Local readiness gate, traces, status snapshots, HUD/status contract, risk ledger | `npm run observability:ready` reports 16/16 | Complete for local gate | +| Flow separation and progress tracking | Flow lanes with owner artifacts and update cadence | This roadmap defines lanes below and `docs/architecture/progress-sync-contract.md` makes GitHub/Linear/handoff/roadmap sync part of the readiness gate | Active | +| Realtime Linear sync | Project updates while issue limit is blocked; issues later | ECC-Tools #39 implements opt-in Linear API sync for deferred follow-up backlog items; `docs/architecture/progress-sync-contract.md` defines the local file-backed realtime boundary while issue capacity is blocked | Needs workspace capacity/config rollout | +| Observability for self-use | Local readiness gate, traces, status snapshots, HUD/status contract, risk ledger, progress-sync contract | `npm run observability:ready` reports 18/18 | Complete for local gate | | Proper release and notifications | Release tag, npm publish state, plugin state, social posts | Publication readiness gate exists with May 12 dry-run and May 13 readiness evidence | Not complete; approval/live URLs required | ## Execution Lanes And Tracking Contract Until Linear issue capacity is cleared, this document is the durable execution -ledger and Linear receives project status updates only. When capacity is -available, each lane below should become a small set of Linear issues linked -back to the repo evidence and merge commits. +ledger and Linear receives project status updates only. The sync contract lives +at `docs/architecture/progress-sync-contract.md`. When capacity is available, +each lane below should become a small set of Linear issues linked back to the +repo evidence and merge commits. | Lane | Source of truth | Next tracked artifact | Update cadence | | --- | --- | --- | --- | @@ -253,7 +261,7 @@ back to the repo evidence and merge commits. | Evaluation and RAG | Reference-set validation, harness audit, traces, ECC-Tools corpus | Read-only evaluator/RAG prototype plus stale-salvage, billing-readiness, CI-failure-diagnosis, harness-config-quality, AgentShield policy-exception, skill-quality evidence, deep-analyzer evidence, and RAG/evaluator comparison fixtures | Hosted retrieval/check-run automation plan | | AgentShield enterprise | AgentShield PR evidence and roadmap notes | Baseline-drift evidence-pack and backlog sync follow-up | Next implementation batch | | ECC Tools app | ECC-Tools PR evidence, billing audit, risk taxonomy, evaluator/RAG corpus | Capacity-backed Linear rollout | Next implementation batch | -| Linear progress | Linear project status updates and this mirror | Status update with queue/evidence/missing gates | Every significant merge batch | +| Linear progress | Linear project status updates, `docs/architecture/progress-sync-contract.md`, and this mirror | Status update with queue/evidence/missing gates | Every significant merge batch | The project status update should always include: diff --git a/docs/architecture/observability-readiness.md b/docs/architecture/observability-readiness.md index c0f0ce31..08bfdd77 100644 --- a/docs/architecture/observability-readiness.md +++ b/docs/architecture/observability-readiness.md @@ -32,6 +32,9 @@ operator needs. `tool-usage.jsonl` events that ECC2 can sync. - Risk ledger: `ecc2/src/observability/mod.rs` scores tool calls and stores a paginated ledger for review. +- Progress sync: `docs/architecture/progress-sync-contract.md` defines how + GitHub, Linear, local handoffs, the repo roadmap, and `scripts/work-items.js` + stay aligned during merge batches and release-gate reviews. ## Reference Pressure @@ -64,9 +67,12 @@ later, but only after the local event model is useful enough to trust. operator dashboard. 5. Run `node scripts/session-inspect.js --list-adapters` to confirm which session surfaces are available. -6. Use ECC2 tool logs for risky operations, conflict analysis, and handoff +6. Run `node scripts/work-items.js sync-github --repo ` before + relying on local work-item status for a tracked repository. +7. Use ECC2 tool logs for risky operations, conflict analysis, and handoff review before increasing autonomy. The end-state is practical: before asking ECC to run larger multi-agent loops, the operator can prove the system has live status, durable session traces, -baseline scorecards, and a local risk ledger. +baseline scorecards, a local risk ledger, and a progress-sync contract that +keeps GitHub, Linear, handoffs, and roadmap evidence from drifting apart. diff --git a/docs/architecture/progress-sync-contract.md b/docs/architecture/progress-sync-contract.md new file mode 100644 index 00000000..8830e615 --- /dev/null +++ b/docs/architecture/progress-sync-contract.md @@ -0,0 +1,67 @@ +# Progress Sync Contract + +ECC 2.0 tracks execution state across GitHub, Linear, local handoffs, and the +repo roadmap. This contract defines the minimum evidence required before a +status update can claim a lane is current. + +## Sources Of Truth + +| Surface | Role | Current rule | +| --- | --- | --- | +| GitHub PRs/issues/discussions | Public queue and review state | Recheck live counts before every significant merge batch and before release approval. | +| Linear project | Executive roadmap and stakeholder status update | Post project status updates while issue capacity blocks issue creation. Create/reuse issues only when workspace capacity is available. | +| Local handoff | Durable operator continuity | Update the active handoff after every merge batch, queue drain, skipped release gate, or blocked external action. | +| Repo roadmap | Auditable planning mirror | Keep `docs/ECC-2.0-GA-ROADMAP.md` aligned to merged PR evidence and unresolved gates. | +| `scripts/work-items.js` | Local tracker bridge | Sync GitHub PRs/issues into the SQLite work-items store for status snapshots and blocked follow-up. | + +## Flow Lanes + +The repo mirror uses these flow lanes so ECC work does not collapse into one +undifferentiated backlog: + +- Queue hygiene and stale-work salvage +- Release, naming, plugin publication, and announcements +- Harness adapter compliance +- Local observability, HUD/status, and session control +- Evaluator/RAG and self-improving harness loops +- AgentShield enterprise security platform +- ECC Tools billing, PR-risk checks, deep analysis, and Linear sync +- Legacy artifact audit and translator/manual-review tails + +Each flow lane needs one owner artifact, one current evidence source, and one +next action. A lane is not current if any of those three fields are missing. + +## Significant Merge Batch Update + +After a significant merge batch, update Linear and the handoff with: + +1. Current public queue counts for tracked GitHub repos. +2. Merged PR numbers, commit IDs, and validation evidence. +3. Changed release gates, if any. +4. Deferred or skipped work and the explicit reason. +5. The next one or two implementation slices. + +When Linear issue capacity is unavailable, use a project status update instead +of creating placeholder issues. When issue capacity is available, create or +reuse exact-title issues and link them to the repo evidence. + +## Realtime Boundary + +The local realtime path is file-backed by default: + +- `node scripts/work-items.js sync-github --repo ` imports current + GitHub PR and issue state into the SQLite work-items store. +- `node scripts/status.js --json` and `node scripts/work-items.js list --json` + expose local state for a HUD, handoff, or later Linear sync. +- Linear remains the external status surface; the repo does not require hosted + telemetry to be release-ready. + +Hosted telemetry such as PostHog can be added later, but it must consume the +same event model rather than becoming a second source of truth. + +## Release Gate + +Do not publish, tag, announce, submit marketplace packages, or claim plugin +availability from this contract alone. Release readiness still requires the +publication-readiness evidence documents, fresh queue checks, package checks, +plugin checks, and explicit maintainer approval. diff --git a/scripts/observability-readiness.js b/scripts/observability-readiness.js index 97d4bd15..e90321b3 100644 --- a/scripts/observability-readiness.js +++ b/scripts/observability-readiness.js @@ -124,6 +124,9 @@ function buildChecks(rootDir) { const sessionManagerRust = readText(rootDir, 'ecc2/src/session/manager.rs'); const readinessDoc = readText(rootDir, 'docs/architecture/observability-readiness.md'); const hudStatusContract = readText(rootDir, 'docs/architecture/hud-status-session-control.md'); + const progressSyncContract = readText(rootDir, 'docs/architecture/progress-sync-contract.md'); + const gaRoadmap = readText(rootDir, 'docs/ECC-2.0-GA-ROADMAP.md'); + const workItems = readText(rootDir, 'scripts/work-items.js'); const hudStatusFixture = safeParseJson(readText(rootDir, 'examples/hud-status-contract.json')) || {}; const quickstart = readText(rootDir, 'docs/releases/2.0.0-rc.1/quickstart.md'); const releaseNotes = readText(rootDir, 'docs/releases/2.0.0-rc.1/release-notes.md'); @@ -238,6 +241,40 @@ function buildChecks(rootDir) { && releaseNotes.includes('observability-readiness.md'), fix: 'Add the observability readiness doc and link it from rc.1 release docs.' }, + { + id: 'progress-sync-contract', + category: 'Tracker Sync', + points: 2, + path: 'docs/architecture/progress-sync-contract.md', + description: 'Linear, GitHub, handoff, and roadmap progress sync has an evidence-backed contract', + pass: fileExists(rootDir, 'docs/architecture/progress-sync-contract.md') + && includesAll(progressSyncContract, [ + 'Linear', + 'GitHub', + 'handoff', + 'work-items', + 'issue capacity', + 'status update', + 'queue counts', + 'release gate', + 'flow lanes', + 'evidence' + ]) + && includesAll(gaRoadmap, [ + 'Execution Lanes And Tracking Contract', + 'docs/architecture/progress-sync-contract.md', + 'Linear progress', + 'Every significant merge batch' + ]) + && includesAll(workItems, [ + 'sync-github', + 'github-pr', + 'github-issue', + 'sourceClosedAt', + 'ecc-work-items-sync-github' + ]), + fix: 'Add the progress sync contract, link it from the GA roadmap, and preserve work-items GitHub sync.' + }, { id: 'package-exposes-readiness-gate', category: 'Packaging', diff --git a/tests/scripts/observability-readiness.test.js b/tests/scripts/observability-readiness.test.js index d2e1b6bf..c4e4c07e 100644 --- a/tests/scripts/observability-readiness.test.js +++ b/tests/scripts/observability-readiness.test.js @@ -57,11 +57,22 @@ function seedMinimalRepo(rootDir, overrides = {}) { 'scripts/session-inspect.js': '--list-adapters --write inspectSessionTarget', 'scripts/lib/session-adapters/registry.js': 'module.exports = {};', 'scripts/harness-audit.js': 'Deterministic harness audit --format overall_score', + 'scripts/work-items.js': 'sync-github github-pr github-issue sourceClosedAt ecc-work-items-sync-github', 'scripts/hooks/session-activity-tracker.js': 'tool-usage.jsonl session_id tool_name', 'ecc2/src/observability/mod.rs': 'ToolCallEvent RiskAssessment ToolLogger', 'ecc2/src/session/store.rs': 'insert_tool_log query_tool_logs', 'ecc2/src/session/manager.rs': 'sync_tool_activity_metrics tool-usage.jsonl', 'docs/architecture/observability-readiness.md': 'node scripts/observability-readiness.js --format json', + 'docs/architecture/progress-sync-contract.md': [ + 'Linear GitHub handoff work-items issue capacity status update', + 'queue counts release gate flow lanes evidence' + ].join('\n'), + 'docs/ECC-2.0-GA-ROADMAP.md': [ + 'Execution Lanes And Tracking Contract', + 'docs/architecture/progress-sync-contract.md', + 'Linear progress', + 'Every significant merge batch' + ].join('\n'), 'docs/architecture/hud-status-session-control.md': [ 'context toolCalls activeAgents todos checks cost risk queueState', 'create resume status stop diff pr mergeQueue conflictQueue', @@ -230,6 +241,23 @@ function runTests() { } })) passed++; else failed++; + if (test('missing progress sync contract fails without disturbing core tool checks', () => { + const projectRoot = createTempDir('observability-readiness-sync-fail-'); + + try { + seedMinimalRepo(projectRoot, { + 'docs/architecture/progress-sync-contract.md': null + }); + const report = buildReport(projectRoot); + + assert.strictEqual(report.ready, false); + assert.ok(report.checks.some(check => check.id === 'progress-sync-contract' && !check.pass)); + assert.ok(report.checks.some(check => check.id === 'loop-status-live-signal' && check.pass)); + } finally { + cleanup(projectRoot); + } + })) passed++; else failed++; + console.log('\nResults:'); console.log(` Passed: ${passed}`); console.log(` Failed: ${failed}`);