mirror of
https://github.com/affaan-m/everything-claude-code.git
synced 2026-06-19 19:30:29 +08:00
db7f2a6fd5
The official Agent Skills spec (agentskills.io/specification) whitelists exactly 6 top-level frontmatter keys (name/description/license/compatibility/metadata/ allowed-tools). A top-level `origin` key fails the official validator (anthropics/skills quick_validate.py ALLOWED_PROPERTIES; skills-ref validate). This moves `origin: X` -> `metadata.origin: X` across the canonical skills/ tree, preserving each value verbatim. Frontmatter-only, minimal diff. - 251 SKILL.md updated (242 new metadata block, 9 appended to existing metadata) - origin values preserved verbatim (verified 251/251) - YAML validated on all changed files - scoped to canonical skills/ only (docs/<lang> translations + tool mirrors .cursor/.kiro/.agents left untouched; presumably regenerated from canonical) Addresses #2233
60 lines
2.3 KiB
Markdown
60 lines
2.3 KiB
Markdown
---
|
|
name: skill-comply
|
|
description: Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines
|
|
metadata:
|
|
origin: ECC
|
|
tools: Read, Bash
|
|
---
|
|
|
|
# skill-comply: Automated Compliance Measurement
|
|
|
|
Measures whether coding agents actually follow skills, rules, or agent definitions by:
|
|
1. Auto-generating expected behavioral sequences (specs) from any .md file
|
|
2. Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing)
|
|
3. Running `claude -p` and capturing tool call traces via stream-json
|
|
4. Classifying tool calls against spec steps using LLM (not regex)
|
|
5. Checking temporal ordering deterministically
|
|
6. Generating self-contained reports with spec, prompts, and timelines
|
|
|
|
## Supported Targets
|
|
|
|
- **Skills** (`skills/*/SKILL.md`): Workflow skills like search-first, TDD guides
|
|
- **Rules** (`rules/common/*.md`): Mandatory rules like testing.md, security.md, git-workflow.md
|
|
- **Agent definitions** (`agents/*.md`): Whether an agent gets invoked when expected (internal workflow verification not yet supported)
|
|
|
|
## When to Activate
|
|
|
|
- User runs `/skill-comply <path>`
|
|
- User asks "is this rule actually being followed?"
|
|
- After adding new rules/skills, to verify agent compliance
|
|
- Periodically as part of quality maintenance
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Full run
|
|
uv run python -m scripts.run ~/.claude/rules/common/testing.md
|
|
|
|
# Dry run (no cost, spec + scenarios only)
|
|
uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md
|
|
|
|
# Custom models
|
|
uv run python -m scripts.run --gen-model haiku --model sonnet <path>
|
|
```
|
|
|
|
## Key Concept: Prompt Independence
|
|
|
|
Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it.
|
|
|
|
## Report Contents
|
|
|
|
Reports are self-contained and include:
|
|
1. Expected behavioral sequence (auto-generated spec)
|
|
2. Scenario prompts (what was asked at each strictness level)
|
|
3. Compliance scores per scenario
|
|
4. Tool call timelines with LLM classification labels
|
|
|
|
### Advanced (optional)
|
|
|
|
For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.
|