mirror of
https://github.com/ultraworkers/claw-code.git
synced 2026-04-25 05:38:10 +08:00
docs+test: cycle #29 — document + lock text-mode vs JSON-mode exit divergence
Cycle #29 dogfood found a real pinpoint: cross-mode exit code divergence. ## The Pinpoint Dogfooding the CLI revealed that unknown subcommand errors return different exit codes depending on output mode: $ python3 -m src.main nonexistent-cmd # exit 2 $ python3 -m src.main nonexistent-cmd --output-format json # exit 1 ERROR_HANDLING.md documented the exit-code contract (1=parse, 2=timeout) but did NOT explicitly state the contract applies only to JSON mode. Text mode follows argparse defaults (exit 2 for any parse error), which violates the documented contract when interpreted generally. A claw using text mode with 'claw nonexistent' would see exit 2 and misclassify as timeout per the docs. Real protocol contract gap, not implementation bug. ## Classification This is a DOCUMENTATION gap, not a behavior bug: - Text mode follows argparse convention (reasonable for humans) - JSON mode normalizes to documented contract (reasonable for claws) - The divergence is intentional; only the docs were silent about it Fix = document the divergence explicitly + lock it with tests. NOT fix = change text mode exit code to 1 (would break argparse conventions and confuse human users). ## Documentation Changes ERROR_HANDLING.md: 1. Added IMPORTANT callout in Quick Reference section: 'The exit code contract applies ONLY when --output-format json is explicitly set. Text mode follows argparse conventions.' 2. New 'Text mode vs JSON mode exit codes' table showing exact divergence: - Unknown subcommand: text=2, json=1 - Missing required arg: text=2, json=1 - Session not found: text=1, json=1 (app-level, identical) - Success: text=0, json=0 (identical) - Timeout: text=2, json=2 (identical, #161) 3. Practical rule: 'always pass --output-format json' ## Tests Added (5) TestTextVsJsonModeDivergence in test_cross_channel_consistency.py: 1. test_unknown_command_text_mode_exits_2 — text mode argparse default 2. test_unknown_command_json_mode_exits_1 — JSON mode contract normalized 3. test_missing_required_arg_text_mode_exits_2 — same for missing args 4. test_missing_required_arg_json_mode_exits_1 — same normalization 5. test_success_path_identical_in_both_modes — success exit identical These tests LOCK the expected divergence so: - Documentation stays aligned with implementation - Future changes (either direction) are caught as intentional - Claws trust the docs ## Test Status - 217 → 222 tests passing (+5) - Zero regressions ## Discipline This cycle follows the cycle #28 template exactly: - Dogfood probe revealed real friction (test said exit=2, docs said exit=1) - Minimal fix shape (documentation clarification, not code change) - Regression guard via tests - Evidence-backed, not speculative Relationship to #181: - #181 fixed env.exit_code != process exit (WITHIN JSON mode) - #29 clarifies exit code contract scope (ONLY JSON mode) - Both establish: exit codes are deterministic, but only when --output-format json --- Classification (per cycle #24 calibration): - Red-state bug? ✗ (behavior was reasonable, docs were incomplete) - Real friction? ✓ (docs/code divergence revealed by dogfood) - Evidence-backed? ✓ (test suite probed both modes, found the gap) Source: Jobdori cycle #29 proactive dogfood — in response to Clawhip nudge for pinpoint hunting. Found that text-mode errors return exit 2 but ERROR_HANDLING.md implied exit 1 was the parse-error contract universally.
This commit is contained in:
parent
af306d489e
commit
de368a2615
@ -10,12 +10,26 @@ After cycles #178–#179 (parser-front-door hole closure), claw-code's error int
|
||||
|
||||
Every clawable command returns JSON on stdout when `--output-format json` is requested.
|
||||
|
||||
**IMPORTANT:** The exit code contract below applies **only when `--output-format json` is explicitly set**. Text mode follows argparse conventions and may return different exit codes (e.g., `2` for argparse parse errors). Claws consuming claw-code as a subprocess MUST always pass `--output-format json` to get the documented contract.
|
||||
|
||||
| Exit Code | Meaning | Response Format | Example |
|
||||
|---|---|---|---|
|
||||
| **0** | Success | `{success fields}` | `{"session_id": "...", "loaded": true}` |
|
||||
| **1** | Error / Not Found | `{error: {kind, message, ...}}` | `{"error": {"kind": "session_not_found", ...}}` |
|
||||
| **2** | Timeout | `{final_stop_reason: "timeout", final_cancel_observed: ...}` | `{"final_stop_reason": "timeout", ...}` |
|
||||
|
||||
### Text mode vs JSON mode exit codes
|
||||
|
||||
| Scenario | Text mode exit | JSON mode exit | Why |
|
||||
|---|---|---|---|
|
||||
| Unknown subcommand | 2 (argparse default) | 1 (parse error envelope) | argparse defaults to 2; JSON mode normalizes to contract |
|
||||
| Missing required arg | 2 (argparse default) | 1 (parse error envelope) | Same reason |
|
||||
| Session not found | 1 | 1 | Application-level error, same in both |
|
||||
| Command executed OK | 0 | 0 | Success path, identical |
|
||||
| Turn-loop timeout | 2 | 2 | Identical (#161 implementation) |
|
||||
|
||||
**Practical rule for claws:** always pass `--output-format json`. This eliminates text-mode surprises and gives you the documented exit-code contract for every error path.
|
||||
|
||||
---
|
||||
|
||||
## One-Handler Pattern
|
||||
|
||||
@ -186,3 +186,57 @@ class TestCrossChannelConsistency:
|
||||
'Boolean fields must correlate with error block:\n' +
|
||||
'\n'.join(failures)
|
||||
)
|
||||
|
||||
|
||||
class TestTextVsJsonModeDivergence:
|
||||
"""Cycle #29: Document known text-mode vs JSON-mode exit code divergence.
|
||||
|
||||
ERROR_HANDLING.md specifies the exit code contract applies ONLY when
|
||||
--output-format json is set. Text mode follows argparse defaults (e.g.,
|
||||
exit 2 for parse errors) while JSON mode normalizes to the contract
|
||||
(exit 1 for parse errors).
|
||||
|
||||
This test class LOCKS the expected divergence so:
|
||||
1. Documentation stays aligned with implementation
|
||||
2. Future changes to text mode behavior are caught as intentional
|
||||
3. Claws consuming subprocess output can trust the docs
|
||||
"""
|
||||
|
||||
def test_unknown_command_text_mode_exits_2(self) -> None:
|
||||
"""Text mode: argparse default exit 2 for unknown subcommand."""
|
||||
result = _run(['nonexistent-cmd'])
|
||||
assert result.returncode == 2, (
|
||||
f'text mode should exit 2 (argparse default), got {result.returncode}'
|
||||
)
|
||||
|
||||
def test_unknown_command_json_mode_exits_1(self) -> None:
|
||||
"""JSON mode: normalized exit 1 for parse error (#178)."""
|
||||
result = _run(['nonexistent-cmd', '--output-format', 'json'])
|
||||
assert result.returncode == 1, (
|
||||
f'JSON mode should exit 1 (protocol contract), got {result.returncode}'
|
||||
)
|
||||
envelope = json.loads(result.stdout)
|
||||
assert envelope['error']['kind'] == 'parse'
|
||||
|
||||
def test_missing_required_arg_text_mode_exits_2(self) -> None:
|
||||
"""Text mode: argparse default exit 2 for missing required arg."""
|
||||
result = _run(['exec-command']) # missing name + prompt
|
||||
assert result.returncode == 2, (
|
||||
f'text mode should exit 2, got {result.returncode}'
|
||||
)
|
||||
|
||||
def test_missing_required_arg_json_mode_exits_1(self) -> None:
|
||||
"""JSON mode: normalized exit 1 for parse error."""
|
||||
result = _run(['exec-command', '--output-format', 'json'])
|
||||
assert result.returncode == 1, (
|
||||
f'JSON mode should exit 1, got {result.returncode}'
|
||||
)
|
||||
|
||||
def test_success_path_identical_in_both_modes(self) -> None:
|
||||
"""Success exit codes are identical in both modes."""
|
||||
text_result = _run(['list-sessions'])
|
||||
json_result = _run(['list-sessions', '--output-format', 'json'])
|
||||
assert text_result.returncode == json_result.returncode == 0, (
|
||||
f'success exit should be 0 in both modes: '
|
||||
f'text={text_result.returncode}, json={json_result.returncode}'
|
||||
)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user