docs: add harness adapter compliance matrix (#1784)

This commit is contained in:
Affaan Mustafa 2026-05-12 02:24:04 -04:00 committed by GitHub
parent 60bd26fadf
commit 969acd9078
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 196 additions and 1 deletions

View File

@ -20,6 +20,10 @@ As of 2026-05-12:
`agentshield`, `JARVIS`, `ECC-Tools`, and `ECC-website`.
- `npm run harness:audit -- --format json` reports 70/70 on current `main`.
- `npm run observability:ready` reports 14/14 readiness on current `main`.
- `docs/architecture/harness-adapter-compliance.md` maps Claude Code, Codex,
OpenCode, Cursor, Gemini, Zed-adjacent, dmux, Orca, Superset, Ghast, and
terminal-only support to install paths, verification commands, and risk
notes.
- AgentShield PR #53 reduced two context-rule false positives and closed the
remaining AgentShield issues.
- ECC PR #1778 recovered the useful stale #1413 network/homelab architect-agent
@ -175,7 +179,8 @@ Acceptance:
## Next Engineering Slices
1. Add the harness adapter compliance matrix and public scorecard onramp.
1. Move the harness adapter compliance matrix from Markdown to a data-backed
validator.
2. Add the release/name/plugin publication checklist with evidence fields.
3. Start AgentShield enterprise policy schema and SARIF implementation in the
AgentShield repo.

View File

@ -13,6 +13,9 @@ The goal is to keep the durable parts of agentic work in one repo:
Claude Code, Codex, OpenCode, Cursor, Gemini, and future harnesses should adapt those assets at the edge instead of requiring a new workflow model for every tool.
For the operator-facing support matrix and scorecard workflow, see
[Harness Adapter Compliance Matrix](harness-adapter-compliance.md).
## Portability Model
| Surface | Shared Source | Harness Adapter | Current Status |

View File

@ -0,0 +1,97 @@
# Harness Adapter Compliance Matrix
This matrix is the public onramp for teams that want to use ECC across more
than one coding harness. It turns the cross-harness architecture into a
practical scorecard: what works today, what is instruction-only, what needs an
adapter, and what evidence an operator should collect before trusting a setup.
ECC's durable units stay in shared sources:
- `skills/*/SKILL.md`
- `rules/`
- `commands/`
- `hooks/hooks.json`
- `scripts/hooks/`
- MCP reference configs
- session and observability contracts
Harness-specific files should only adapt loading, event shape, command names,
or platform limits.
## Compliance States
| State | Meaning |
| --- | --- |
| Native | ECC can install or verify the surface directly for this harness. |
| Adapter-backed | ECC has a thin adapter, plugin, or package surface, but parity differs by harness. |
| Instruction-backed | ECC can provide the guidance and files, but the harness does not expose the runtime hook/session surface ECC needs for enforcement. |
| Reference-only | The tool is useful as a design pressure or external runtime, but ECC does not yet ship a direct installer or adapter for it. |
## Matrix
| Harness or runtime | State | Supported assets | Unsupported or different surfaces | Install or onramp | Verification command | Risk notes |
| --- | --- | --- | --- | --- | --- | --- |
| Claude Code | Native | Claude plugin assets, skills, commands, hooks, MCP config, local rules, statusline-oriented workflows | Claude-native hooks do not imply parity in other harnesses | `./install.sh --profile minimal --target claude` or Claude plugin install | `npm run harness:audit -- --format json` and `node scripts/session-inspect.js --list-adapters` | Avoid loading every skill by default; keep hooks opt-in and inspectable. |
| Codex | Instruction-backed | `AGENTS.md`, Codex plugin metadata, skills, MCP reference config, command patterns | Native hook enforcement and Claude slash-command semantics are not equivalent | `./install.sh --profile minimal --target codex` plus repo-local `AGENTS.md` review | `npm run harness:audit -- --format json` | Treat hooks as policy text unless a native Codex hook surface exists. |
| OpenCode | Adapter-backed | OpenCode package/plugin metadata, shared skills, MCP config, event adapter patterns | Event names, plugin packaging, and command dispatch differ from Claude Code | OpenCode package or plugin surface from this repo | `node tests/scripts/build-opencode.test.js` and `npm run harness:audit -- --format json` | Keep hook logic in shared scripts and adapt only event shape at the edge. |
| Cursor | Adapter-backed | Cursor rules, project-local skills, hook adapter, shared scripts | Cursor hook events and rule loading differ from Claude Code | `./install.sh --profile minimal --target cursor` | `node tests/lib/install-targets.test.js` and `npm run harness:audit -- --format json` | Cursor adapters must preserve existing project rules and avoid silent overwrite. |
| Gemini | Instruction-backed | Gemini project-local instructions, shared skills, rules, compatibility docs | No full ECC hook parity; ecosystem ports must document drift from upstream ECC | `./install.sh --profile minimal --target gemini` | `node tests/lib/install-targets.test.js` | Treat Gemini ports as ecosystem adapters until validated end to end inside Gemini CLI. |
| Zed-adjacent workflows | Instruction-backed | Shared skills, `AGENTS.md` style project instructions, verification loops | Zed agent surfaces vary; no first-party ECC installer is shipped today | Manual copy from shared ECC sources until adapter requirements settle | `npm run harness:audit -- --format json` | Do not claim native Zed support before a real adapter and verification path exist. |
| dmux | Adapter-backed | Session snapshots, tmux/worktree orchestration status, handoff exports | dmux is an orchestration runtime, not an install target for skills/rules | `node scripts/session-inspect.js --list-adapters` and dmux session target inspection | `node tests/lib/session-adapters.test.js` | Treat dmux events as session/runtime signals, not as a replacement for repo validation. |
| Orca | Reference-only | Worktree lifecycle, review state, notification, and provider-identity design pressure | No ECC installer or direct adapter today | Use as a comparison target for worktree/session state requirements | `npm run observability:ready` | Do not import product-specific assumptions; convert lessons into ECC event fields. |
| Superset | Reference-only | Workspace presets, parallel-agent review loops, worktree isolation design pressure | No ECC installer or direct adapter today | Use as a comparison target for workspace preset taxonomy | `npm run observability:ready` | Keep ECC portable; do not require a desktop workspace to get basic value. |
| Ghast | Reference-only | Terminal-native pane grouping, cwd grouping, search, notifications | No ECC installer or direct adapter today | Use as a comparison target for terminal-first session grouping | `node scripts/session-inspect.js --list-adapters` | Preserve terminal ergonomics before adding visual UI assumptions. |
| Terminal-only | Native | Skills, rules, commands, scripts, harness audit, observability readiness, handoffs | No external UI, no automatic session control unless scripts are run explicitly | Clone repo, run commands directly, use minimal profile for project installs | `npm run harness:audit -- --format json` and `npm run observability:ready` | This is the fallback contract; every higher-level adapter should degrade to it. |
## Scorecard Onramp
Use this sequence before asking ECC to make a team or repo setup more
autonomous:
```bash
npm run harness:audit -- --format json
npm run observability:ready
node scripts/session-inspect.js --list-adapters
node scripts/loop-status.js --json --write-dir .ecc/loop-status
```
Read the result as a setup scorecard, not a product badge:
- `harness:audit` scores tool coverage, context efficiency, quality gates,
memory persistence, eval coverage, security guardrails, and cost efficiency.
- `observability:ready` proves the repo still exposes the local status,
session, tool-activity, risk-ledger, and release-onramp signals.
- `session-inspect --list-adapters` shows which session surfaces are actually
inspectable in the current environment.
- `loop-status --json` creates a machine-readable handoff/status payload for
longer autonomous runs.
## Evidence Fields For Future Data Matrix
When this matrix moves from Markdown to a data-backed validator, each adapter
record should expose:
- `id`
- `state`
- `supported_assets`
- `unsupported_surfaces`
- `install_or_onramp`
- `verification_commands`
- `risk_notes`
- `last_verified_at`
- `owner`
- `source_docs`
The validator should fail if a public adapter claim has no install path,
verification command, or risk note.
## Operating Rules
- Prefer small, additive adapters over harness-specific forks of the same
workflow.
- Do not call a harness native until the adapter has an install path and a
verification command.
- Keep Codex, Gemini, and Zed surfaces honest when enforcement is
instruction-backed rather than runtime-backed.
- Treat reference-only tools as design pressure until ECC has a direct adapter.
- Keep the terminal-only path healthy; it is the portability floor.

View File

@ -0,0 +1,90 @@
'use strict';
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const repoRoot = path.resolve(__dirname, '..', '..');
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
fn();
console.log(`${name}`);
passed++;
} catch (error) {
console.log(`${name}`);
console.log(` Error: ${error.message}`);
failed++;
}
}
function read(relativePath) {
return fs.readFileSync(path.join(repoRoot, relativePath), 'utf8');
}
console.log('\n=== Testing harness adapter compliance docs ===\n');
test('adapter compliance matrix covers the required harness surfaces', () => {
const source = read('docs/architecture/harness-adapter-compliance.md');
for (const harness of [
'Claude Code',
'Codex',
'OpenCode',
'Cursor',
'Gemini',
'Zed-adjacent',
'dmux',
'Orca',
'Superset',
'Ghast',
'Terminal-only'
]) {
assert.ok(source.includes(harness), `Expected matrix to include ${harness}`);
}
});
test('adapter compliance matrix includes the required evidence columns', () => {
const source = read('docs/architecture/harness-adapter-compliance.md');
for (const heading of [
'Supported assets',
'Unsupported or different surfaces',
'Install or onramp',
'Verification command',
'Risk notes'
]) {
assert.ok(source.includes(heading), `Expected matrix to include ${heading}`);
}
});
test('scorecard onramp names the local verification commands', () => {
const source = read('docs/architecture/harness-adapter-compliance.md');
for (const command of [
'npm run harness:audit -- --format json',
'npm run observability:ready',
'node scripts/session-inspect.js --list-adapters',
'node scripts/loop-status.js --json --write-dir .ecc/loop-status'
]) {
assert.ok(source.includes(command), `Expected onramp to include ${command}`);
}
});
test('cross-harness architecture links to the adapter compliance matrix', () => {
const source = read('docs/architecture/cross-harness.md');
assert.ok(source.includes('harness-adapter-compliance.md'));
});
test('GA roadmap records the matrix as current evidence and points to data-backed validation next', () => {
const source = read('docs/ECC-2.0-GA-ROADMAP.md');
assert.ok(source.includes('docs/architecture/harness-adapter-compliance.md'));
assert.ok(source.includes('data-backed'));
});
if (failed > 0) {
console.log(`\nFailed: ${failed}`);
process.exit(1);
}
console.log(`\nPassed: ${passed}`);