docs: add harness adapter compliance matrix (#1784)

This commit is contained in:
Affaan Mustafa 2026-05-12 02:24:04 -04:00 committed by GitHub
parent 60bd26fadf
commit 969acd9078
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 196 additions and 1 deletions

View File

@ -20,6 +20,10 @@ As of 2026-05-12:
`agentshield`, `JARVIS`, `ECC-Tools`, and `ECC-website`. `agentshield`, `JARVIS`, `ECC-Tools`, and `ECC-website`.
- `npm run harness:audit -- --format json` reports 70/70 on current `main`. - `npm run harness:audit -- --format json` reports 70/70 on current `main`.
- `npm run observability:ready` reports 14/14 readiness on current `main`. - `npm run observability:ready` reports 14/14 readiness on current `main`.
- `docs/architecture/harness-adapter-compliance.md` maps Claude Code, Codex,
OpenCode, Cursor, Gemini, Zed-adjacent, dmux, Orca, Superset, Ghast, and
terminal-only support to install paths, verification commands, and risk
notes.
- AgentShield PR #53 reduced two context-rule false positives and closed the - AgentShield PR #53 reduced two context-rule false positives and closed the
remaining AgentShield issues. remaining AgentShield issues.
- ECC PR #1778 recovered the useful stale #1413 network/homelab architect-agent - ECC PR #1778 recovered the useful stale #1413 network/homelab architect-agent
@ -175,7 +179,8 @@ Acceptance:
## Next Engineering Slices ## Next Engineering Slices
1. Add the harness adapter compliance matrix and public scorecard onramp. 1. Move the harness adapter compliance matrix from Markdown to a data-backed
validator.
2. Add the release/name/plugin publication checklist with evidence fields. 2. Add the release/name/plugin publication checklist with evidence fields.
3. Start AgentShield enterprise policy schema and SARIF implementation in the 3. Start AgentShield enterprise policy schema and SARIF implementation in the
AgentShield repo. AgentShield repo.

View File

@ -13,6 +13,9 @@ The goal is to keep the durable parts of agentic work in one repo:
Claude Code, Codex, OpenCode, Cursor, Gemini, and future harnesses should adapt those assets at the edge instead of requiring a new workflow model for every tool. Claude Code, Codex, OpenCode, Cursor, Gemini, and future harnesses should adapt those assets at the edge instead of requiring a new workflow model for every tool.
For the operator-facing support matrix and scorecard workflow, see
[Harness Adapter Compliance Matrix](harness-adapter-compliance.md).
## Portability Model ## Portability Model
| Surface | Shared Source | Harness Adapter | Current Status | | Surface | Shared Source | Harness Adapter | Current Status |

View File

@ -0,0 +1,97 @@
# Harness Adapter Compliance Matrix
This matrix is the public onramp for teams that want to use ECC across more
than one coding harness. It turns the cross-harness architecture into a
practical scorecard: what works today, what is instruction-only, what needs an
adapter, and what evidence an operator should collect before trusting a setup.
ECC's durable units stay in shared sources:
- `skills/*/SKILL.md`
- `rules/`
- `commands/`
- `hooks/hooks.json`
- `scripts/hooks/`
- MCP reference configs
- session and observability contracts
Harness-specific files should only adapt loading, event shape, command names,
or platform limits.
## Compliance States
| State | Meaning |
| --- | --- |
| Native | ECC can install or verify the surface directly for this harness. |
| Adapter-backed | ECC has a thin adapter, plugin, or package surface, but parity differs by harness. |
| Instruction-backed | ECC can provide the guidance and files, but the harness does not expose the runtime hook/session surface ECC needs for enforcement. |
| Reference-only | The tool is useful as a design pressure or external runtime, but ECC does not yet ship a direct installer or adapter for it. |
## Matrix
| Harness or runtime | State | Supported assets | Unsupported or different surfaces | Install or onramp | Verification command | Risk notes |
| --- | --- | --- | --- | --- | --- | --- |
| Claude Code | Native | Claude plugin assets, skills, commands, hooks, MCP config, local rules, statusline-oriented workflows | Claude-native hooks do not imply parity in other harnesses | `./install.sh --profile minimal --target claude` or Claude plugin install | `npm run harness:audit -- --format json` and `node scripts/session-inspect.js --list-adapters` | Avoid loading every skill by default; keep hooks opt-in and inspectable. |
| Codex | Instruction-backed | `AGENTS.md`, Codex plugin metadata, skills, MCP reference config, command patterns | Native hook enforcement and Claude slash-command semantics are not equivalent | `./install.sh --profile minimal --target codex` plus repo-local `AGENTS.md` review | `npm run harness:audit -- --format json` | Treat hooks as policy text unless a native Codex hook surface exists. |
| OpenCode | Adapter-backed | OpenCode package/plugin metadata, shared skills, MCP config, event adapter patterns | Event names, plugin packaging, and command dispatch differ from Claude Code | OpenCode package or plugin surface from this repo | `node tests/scripts/build-opencode.test.js` and `npm run harness:audit -- --format json` | Keep hook logic in shared scripts and adapt only event shape at the edge. |
| Cursor | Adapter-backed | Cursor rules, project-local skills, hook adapter, shared scripts | Cursor hook events and rule loading differ from Claude Code | `./install.sh --profile minimal --target cursor` | `node tests/lib/install-targets.test.js` and `npm run harness:audit -- --format json` | Cursor adapters must preserve existing project rules and avoid silent overwrite. |
| Gemini | Instruction-backed | Gemini project-local instructions, shared skills, rules, compatibility docs | No full ECC hook parity; ecosystem ports must document drift from upstream ECC | `./install.sh --profile minimal --target gemini` | `node tests/lib/install-targets.test.js` | Treat Gemini ports as ecosystem adapters until validated end to end inside Gemini CLI. |
| Zed-adjacent workflows | Instruction-backed | Shared skills, `AGENTS.md` style project instructions, verification loops | Zed agent surfaces vary; no first-party ECC installer is shipped today | Manual copy from shared ECC sources until adapter requirements settle | `npm run harness:audit -- --format json` | Do not claim native Zed support before a real adapter and verification path exist. |
| dmux | Adapter-backed | Session snapshots, tmux/worktree orchestration status, handoff exports | dmux is an orchestration runtime, not an install target for skills/rules | `node scripts/session-inspect.js --list-adapters` and dmux session target inspection | `node tests/lib/session-adapters.test.js` | Treat dmux events as session/runtime signals, not as a replacement for repo validation. |
| Orca | Reference-only | Worktree lifecycle, review state, notification, and provider-identity design pressure | No ECC installer or direct adapter today | Use as a comparison target for worktree/session state requirements | `npm run observability:ready` | Do not import product-specific assumptions; convert lessons into ECC event fields. |
| Superset | Reference-only | Workspace presets, parallel-agent review loops, worktree isolation design pressure | No ECC installer or direct adapter today | Use as a comparison target for workspace preset taxonomy | `npm run observability:ready` | Keep ECC portable; do not require a desktop workspace to get basic value. |
| Ghast | Reference-only | Terminal-native pane grouping, cwd grouping, search, notifications | No ECC installer or direct adapter today | Use as a comparison target for terminal-first session grouping | `node scripts/session-inspect.js --list-adapters` | Preserve terminal ergonomics before adding visual UI assumptions. |
| Terminal-only | Native | Skills, rules, commands, scripts, harness audit, observability readiness, handoffs | No external UI, no automatic session control unless scripts are run explicitly | Clone repo, run commands directly, use minimal profile for project installs | `npm run harness:audit -- --format json` and `npm run observability:ready` | This is the fallback contract; every higher-level adapter should degrade to it. |
## Scorecard Onramp
Use this sequence before asking ECC to make a team or repo setup more
autonomous:
```bash
npm run harness:audit -- --format json
npm run observability:ready
node scripts/session-inspect.js --list-adapters
node scripts/loop-status.js --json --write-dir .ecc/loop-status
```
Read the result as a setup scorecard, not a product badge:
- `harness:audit` scores tool coverage, context efficiency, quality gates,
memory persistence, eval coverage, security guardrails, and cost efficiency.
- `observability:ready` proves the repo still exposes the local status,
session, tool-activity, risk-ledger, and release-onramp signals.
- `session-inspect --list-adapters` shows which session surfaces are actually
inspectable in the current environment.
- `loop-status --json` creates a machine-readable handoff/status payload for
longer autonomous runs.
## Evidence Fields For Future Data Matrix
When this matrix moves from Markdown to a data-backed validator, each adapter
record should expose:
- `id`
- `state`
- `supported_assets`
- `unsupported_surfaces`
- `install_or_onramp`
- `verification_commands`
- `risk_notes`
- `last_verified_at`
- `owner`
- `source_docs`
The validator should fail if a public adapter claim has no install path,
verification command, or risk note.
## Operating Rules
- Prefer small, additive adapters over harness-specific forks of the same
workflow.
- Do not call a harness native until the adapter has an install path and a
verification command.
- Keep Codex, Gemini, and Zed surfaces honest when enforcement is
instruction-backed rather than runtime-backed.
- Treat reference-only tools as design pressure until ECC has a direct adapter.
- Keep the terminal-only path healthy; it is the portability floor.

View File

@ -0,0 +1,90 @@
'use strict';
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const repoRoot = path.resolve(__dirname, '..', '..');
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
fn();
console.log(`${name}`);
passed++;
} catch (error) {
console.log(`${name}`);
console.log(` Error: ${error.message}`);
failed++;
}
}
function read(relativePath) {
return fs.readFileSync(path.join(repoRoot, relativePath), 'utf8');
}
console.log('\n=== Testing harness adapter compliance docs ===\n');
test('adapter compliance matrix covers the required harness surfaces', () => {
const source = read('docs/architecture/harness-adapter-compliance.md');
for (const harness of [
'Claude Code',
'Codex',
'OpenCode',
'Cursor',
'Gemini',
'Zed-adjacent',
'dmux',
'Orca',
'Superset',
'Ghast',
'Terminal-only'
]) {
assert.ok(source.includes(harness), `Expected matrix to include ${harness}`);
}
});
test('adapter compliance matrix includes the required evidence columns', () => {
const source = read('docs/architecture/harness-adapter-compliance.md');
for (const heading of [
'Supported assets',
'Unsupported or different surfaces',
'Install or onramp',
'Verification command',
'Risk notes'
]) {
assert.ok(source.includes(heading), `Expected matrix to include ${heading}`);
}
});
test('scorecard onramp names the local verification commands', () => {
const source = read('docs/architecture/harness-adapter-compliance.md');
for (const command of [
'npm run harness:audit -- --format json',
'npm run observability:ready',
'node scripts/session-inspect.js --list-adapters',
'node scripts/loop-status.js --json --write-dir .ecc/loop-status'
]) {
assert.ok(source.includes(command), `Expected onramp to include ${command}`);
}
});
test('cross-harness architecture links to the adapter compliance matrix', () => {
const source = read('docs/architecture/cross-harness.md');
assert.ok(source.includes('harness-adapter-compliance.md'));
});
test('GA roadmap records the matrix as current evidence and points to data-backed validation next', () => {
const source = read('docs/ECC-2.0-GA-ROADMAP.md');
assert.ok(source.includes('docs/architecture/harness-adapter-compliance.md'));
assert.ok(source.includes('data-backed'));
});
if (failed > 0) {
console.log(`\nFailed: ${failed}`);
process.exit(1);
}
console.log(`\nPassed: ${passed}`);