mirror of
https://github.com/affaan-m/everything-claude-code.git
synced 2026-05-13 18:00:35 +08:00
ci: gate observability on release safety evidence
Add release-safety evidence coverage to observability readiness and refresh rc.1 publication gate docs.
This commit is contained in:
Affaan Mustafa
GitHub
parent
d4728a0d80
commit
42f04edc03
@ -35,6 +35,10 @@ operator needs.
|
||||
- Progress sync: `docs/architecture/progress-sync-contract.md` defines how
|
||||
GitHub, Linear, local handoffs, the repo roadmap, and `scripts/work-items.js`
|
||||
stay aligned during merge batches and release-gate reviews.
|
||||
- Release safety: `docs/releases/2.0.0-rc.1/publication-readiness.md`,
|
||||
post-hardening evidence, supply-chain incident response, workflow-security
|
||||
validation, npm pack checks, and release-surface tests must be present before
|
||||
any public tag, package publish, plugin submission, or announcement action.
|
||||
|
||||
## Reference Pressure
|
||||
|
||||
@ -71,6 +75,9 @@ later, but only after the local event model is useful enough to trust.
|
||||
relying on local work-item status for a tracked repository.
|
||||
7. Use ECC2 tool logs for risky operations, conflict analysis, and handoff
|
||||
review before increasing autonomy.
|
||||
8. Re-run the release-safety evidence checks before any public release action:
|
||||
publication readiness, supply-chain incident response, workflow-security
|
||||
validation, package surface, and release-surface tests.
|
||||
|
||||
The end-state is practical: before asking ECC to run larger multi-agent loops,
|
||||
the operator can prove the system has live status, durable session traces,
|
||||
|
||||
@ -43,14 +43,15 @@ The prior post-#42 local checkout handoff recorded both ECC-Tools repos at
|
||||
| --- | --- | --- |
|
||||
| Harness audit | `npm run harness:audit -- --format json` | `overall_score: 70`, `max_score: 70`, no top actions |
|
||||
| Adapter scorecard | `npm run harness:adapters -- --check` | `Harness Adapter Compliance: PASS`; 11 adapters |
|
||||
| Observability readiness | `npm run observability:ready -- --format json` | `overall_score: 18`, `max_score: 18`, `ready: true`, no top actions |
|
||||
| Observability readiness | `npm run observability:ready -- --format json` | `overall_score: 21`, `max_score: 21`, `ready: true`, no top actions; includes Release Safety 3/3 |
|
||||
| Workflow security validator | `node scripts/ci/validate-workflow-security.js` | Validated 7 workflow files |
|
||||
| Workflow validator tests | `node tests/ci/validate-workflow-security.test.js` | Passed 14/14 |
|
||||
| Release surface | `node tests/docs/ecc2-release-surface.test.js` | Passed 18/18 |
|
||||
| Package surface | `node tests/scripts/npm-publish-surface.test.js` | Passed 2/2 |
|
||||
| Root suite | `node tests/run-all.js` | Passed 2380/2380, 0 failed |
|
||||
| Root suite | `node tests/run-all.js` | Passed 2381/2381, 0 failed |
|
||||
| Markdown lint | `npx markdownlint-cli '**/*.md' --ignore node_modules --ignore docs/drafts` | Passed |
|
||||
| Rust surface | `cd ecc2 && cargo test` | Passed 462/462; warnings only for unused functions/fields |
|
||||
| GitGuardian Security Checks | GitHub check on post-hardening security PRs | Passed before merge |
|
||||
|
||||
## Supply-Chain Evidence
|
||||
|
||||
@ -60,6 +61,7 @@ The prior post-#42 local checkout handoff recorded both ECC-Tools repos at
|
||||
| Local npm signature audit | `npm audit signatures` | 241 verified registry signatures and 30 verified attestations |
|
||||
| Rust advisory audit | `cd ecc2 && cargo audit -q` | Passed silently |
|
||||
| TanStack / Mini Shai-Hulud IOC check | Grep for affected package namespaces, payload filenames, and known commit marker | No runtime or lockfile dependency on affected packages; no worm IOC matches |
|
||||
| GitGuardian Security Checks | GitHub check on post-hardening security PRs | Passed before merge |
|
||||
|
||||
## External Advisory Mapping
|
||||
|
||||
|
||||
@ -52,8 +52,10 @@ Record the exact commit SHA and command output before any publication action:
|
||||
| Clean release branch | `git status --short --branch` | On intended release commit; no unrelated files | Pending final clean-checkout release pass; May 13 evidence branch still had unrelated untracked `docs/drafts/` |
|
||||
| Harness audit | `npm run harness:audit -- --format json` | 70/70 passing | `publication-evidence-2026-05-13.md`: 70/70 |
|
||||
| Adapter scorecard | `npm run harness:adapters -- --check` | PASS | `publication-evidence-2026-05-13.md`: PASS, 11 adapters |
|
||||
| Observability readiness | `npm run observability:ready` | 18/18 passing | `publication-evidence-2026-05-13-post-hardening.md`: 18/18, ready true |
|
||||
| Root suite | `node tests/run-all.js` | 0 failures | `publication-evidence-2026-05-13-post-hardening.md`: 2380 passed, 0 failed |
|
||||
| Observability readiness | `npm run observability:ready` | 21/21 passing | `publication-evidence-2026-05-13-post-hardening.md`: 21/21, ready true after release-safety gate refresh |
|
||||
| Release safety gate | `npm run observability:ready -- --format json` | Release Safety category passing with publication readiness, supply-chain, workflow security, package surface, and release-surface evidence | `publication-evidence-2026-05-13-post-hardening.md`: Release Safety 3/3 |
|
||||
| Supply-chain verification | `npm audit --json`; `npm audit signatures`; `cd ecc2 && cargo audit -q`; Dependabot alerts; GitGuardian Security Checks | 0 vulnerabilities/alerts, registry signatures verified, GitGuardian clean | `publication-evidence-2026-05-13-post-hardening.md`: npm, cargo, Dependabot, TanStack/Mini Shai-Hulud, and GitGuardian evidence |
|
||||
| Root suite | `node tests/run-all.js` | 0 failures | `publication-evidence-2026-05-13-post-hardening.md`: 2381 passed, 0 failed |
|
||||
| Markdown lint | `npx markdownlint-cli '**/*.md' --ignore node_modules` | 0 failures | `publication-evidence-2026-05-13.md`: passed after zh-CN CLAUDE list-marker normalization |
|
||||
| Package surface | `node tests/scripts/npm-publish-surface.test.js` | 0 failures; no Python bytecode in npm tarball | `2/2` passed in May 12 evidence pass |
|
||||
| Release surface | `node tests/docs/ecc2-release-surface.test.js` | 0 failures | `publication-evidence-2026-05-13.md`: 18/18 passed |
|
||||
|
||||
@ -8,7 +8,9 @@ they do not prove that the workflow executed the intended code path.
|
||||
## Current External Trigger
|
||||
|
||||
As of 2026-05-13, the active incident class is the May 2026 TanStack npm
|
||||
supply-chain compromise:
|
||||
supply-chain compromise. ECC also keeps Mini Shai-Hulud-style npm worm IOCs in
|
||||
the same release-safety sweep because both incident classes target package
|
||||
install/publish paths and developer credentials:
|
||||
|
||||
- TanStack reported 84 malicious versions across 42 `@tanstack/*` packages,
|
||||
published on 2026-05-11 between 19:20 and 19:26 UTC.
|
||||
|
||||
@ -127,6 +127,13 @@ function buildChecks(rootDir) {
|
||||
const progressSyncContract = readText(rootDir, 'docs/architecture/progress-sync-contract.md');
|
||||
const gaRoadmap = readText(rootDir, 'docs/ECC-2.0-GA-ROADMAP.md');
|
||||
const workItems = readText(rootDir, 'scripts/work-items.js');
|
||||
const publicationReadiness = readText(rootDir, 'docs/releases/2.0.0-rc.1/publication-readiness.md');
|
||||
const postHardeningEvidence = readText(rootDir, 'docs/releases/2.0.0-rc.1/publication-evidence-2026-05-13-post-hardening.md');
|
||||
const supplyChainIncidentResponse = readText(rootDir, 'docs/security/supply-chain-incident-response.md');
|
||||
const workflowSecurityValidator = readText(rootDir, 'scripts/ci/validate-workflow-security.js');
|
||||
const workflowSecurityValidatorTests = readText(rootDir, 'tests/ci/validate-workflow-security.test.js');
|
||||
const publishSurfaceTest = readText(rootDir, 'tests/scripts/npm-publish-surface.test.js');
|
||||
const releaseSurfaceTest = readText(rootDir, 'tests/docs/ecc2-release-surface.test.js');
|
||||
const hudStatusFixture = safeParseJson(readText(rootDir, 'examples/hud-status-contract.json')) || {};
|
||||
const quickstart = readText(rootDir, 'docs/releases/2.0.0-rc.1/quickstart.md');
|
||||
const releaseNotes = readText(rootDir, 'docs/releases/2.0.0-rc.1/release-notes.md');
|
||||
@ -275,6 +282,56 @@ function buildChecks(rootDir) {
|
||||
]),
|
||||
fix: 'Add the progress sync contract, link it from the GA roadmap, and preserve work-items GitHub sync.'
|
||||
},
|
||||
{
|
||||
id: 'release-safety-evidence',
|
||||
category: 'Release Safety',
|
||||
points: 3,
|
||||
path: 'docs/releases/2.0.0-rc.1/publication-readiness.md',
|
||||
description: 'Release readiness includes package, workflow, and supply-chain evidence before publication',
|
||||
pass: fileExists(rootDir, 'docs/releases/2.0.0-rc.1/publication-readiness.md')
|
||||
&& fileExists(rootDir, 'docs/releases/2.0.0-rc.1/publication-evidence-2026-05-13-post-hardening.md')
|
||||
&& fileExists(rootDir, 'docs/security/supply-chain-incident-response.md')
|
||||
&& fileExists(rootDir, 'scripts/ci/validate-workflow-security.js')
|
||||
&& fileExists(rootDir, 'tests/ci/validate-workflow-security.test.js')
|
||||
&& fileExists(rootDir, 'tests/scripts/npm-publish-surface.test.js')
|
||||
&& fileExists(rootDir, 'tests/docs/ecc2-release-surface.test.js')
|
||||
&& includesAll(publicationReadiness, [
|
||||
'Publication Gates',
|
||||
'Required Command Evidence',
|
||||
'Do Not Publish If',
|
||||
'npm dist-tag',
|
||||
'GitGuardian',
|
||||
'Dependabot alerts',
|
||||
'npm audit signatures'
|
||||
])
|
||||
&& includesAll(postHardeningEvidence, [
|
||||
'npm audit --json',
|
||||
'npm audit signatures',
|
||||
'cargo audit',
|
||||
'Dependabot alert API',
|
||||
'TanStack',
|
||||
'Mini Shai-Hulud',
|
||||
'GitGuardian Security Checks'
|
||||
])
|
||||
&& includesAll(supplyChainIncidentResponse, [
|
||||
'TanStack',
|
||||
'Mini Shai-Hulud',
|
||||
'npm audit signatures',
|
||||
'trusted publishing',
|
||||
'pull_request_target',
|
||||
'id-token: write'
|
||||
])
|
||||
&& includesAll(workflowSecurityValidator, [
|
||||
'persist-credentials: false',
|
||||
'npm audit signatures',
|
||||
'pull_request_target',
|
||||
'id-token: write'
|
||||
])
|
||||
&& includesAll(workflowSecurityValidatorTests, ['npm audit signatures', 'persist-credentials: false'])
|
||||
&& includesAll(publishSurfaceTest, ['npm pack', 'Python bytecode'])
|
||||
&& includesAll(releaseSurfaceTest, ['publication-readiness.md']),
|
||||
fix: 'Refresh publication readiness, post-hardening evidence, supply-chain response docs, workflow-security validator coverage, and package/release surface tests.'
|
||||
},
|
||||
{
|
||||
id: 'package-exposes-readiness-gate',
|
||||
category: 'Packaging',
|
||||
|
||||
@ -92,7 +92,43 @@ function seedMinimalRepo(rootDir, overrides = {}) {
|
||||
sync: {}
|
||||
}, null, 2),
|
||||
'docs/releases/2.0.0-rc.1/quickstart.md': 'observability-readiness.md',
|
||||
'docs/releases/2.0.0-rc.1/release-notes.md': 'observability-readiness.md'
|
||||
'docs/releases/2.0.0-rc.1/release-notes.md': 'observability-readiness.md',
|
||||
'docs/releases/2.0.0-rc.1/publication-readiness.md': [
|
||||
'Publication Gates',
|
||||
'Required Command Evidence',
|
||||
'Do Not Publish If',
|
||||
'npm dist-tag',
|
||||
'GitGuardian',
|
||||
'Dependabot alerts',
|
||||
'npm audit signatures'
|
||||
].join('\n'),
|
||||
'docs/releases/2.0.0-rc.1/publication-evidence-2026-05-13-post-hardening.md': [
|
||||
'npm audit --json',
|
||||
'npm audit signatures',
|
||||
'cargo audit',
|
||||
'Dependabot alert API',
|
||||
'TanStack',
|
||||
'Mini Shai-Hulud',
|
||||
'GitGuardian Security Checks'
|
||||
].join('\n'),
|
||||
'docs/security/supply-chain-incident-response.md': [
|
||||
'TanStack',
|
||||
'Mini Shai-Hulud',
|
||||
'npm audit signatures',
|
||||
'trusted publishing',
|
||||
'pull_request_target',
|
||||
'id-token: write'
|
||||
].join('\n'),
|
||||
'scripts/ci/validate-workflow-security.js': [
|
||||
'persist-credentials: false',
|
||||
'npm audit signatures',
|
||||
'pull_request_target',
|
||||
'id-token: write',
|
||||
'shared cache'
|
||||
].join('\n'),
|
||||
'tests/ci/validate-workflow-security.test.js': 'npm audit signatures persist-credentials: false',
|
||||
'tests/scripts/npm-publish-surface.test.js': 'npm pack --dry-run Python bytecode',
|
||||
'tests/docs/ecc2-release-surface.test.js': 'publication-readiness.md',
|
||||
};
|
||||
|
||||
for (const [relativePath, content] of Object.entries({ ...files, ...overrides })) {
|
||||
@ -258,6 +294,23 @@ function runTests() {
|
||||
}
|
||||
})) passed++; else failed++;
|
||||
|
||||
if (test('missing release safety evidence fails without disturbing live status checks', () => {
|
||||
const projectRoot = createTempDir('observability-readiness-release-safety-fail-');
|
||||
|
||||
try {
|
||||
seedMinimalRepo(projectRoot, {
|
||||
'docs/releases/2.0.0-rc.1/publication-evidence-2026-05-13-post-hardening.md': 'npm audit --json only'
|
||||
});
|
||||
const report = buildReport(projectRoot);
|
||||
|
||||
assert.strictEqual(report.ready, false);
|
||||
assert.ok(report.checks.some(check => check.id === 'release-safety-evidence' && !check.pass));
|
||||
assert.ok(report.checks.some(check => check.id === 'loop-status-live-signal' && check.pass));
|
||||
} finally {
|
||||
cleanup(projectRoot);
|
||||
}
|
||||
})) passed++; else failed++;
|
||||
|
||||
console.log('\nResults:');
|
||||
console.log(` Passed: ${passed}`);
|
||||
console.log(` Failed: ${failed}`);
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user