From de368a2615c25eaad260b7bb1601c77d371640c2 Mon Sep 17 00:00:00 2001 From: YeonGyu-Kim Date: Wed, 22 Apr 2026 22:03:08 +0900 Subject: [PATCH] =?UTF-8?q?docs+test:=20cycle=20#29=20=E2=80=94=20document?= =?UTF-8?q?=20+=20lock=20text-mode=20vs=20JSON-mode=20exit=20divergence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cycle #29 dogfood found a real pinpoint: cross-mode exit code divergence. ## The Pinpoint Dogfooding the CLI revealed that unknown subcommand errors return different exit codes depending on output mode: $ python3 -m src.main nonexistent-cmd # exit 2 $ python3 -m src.main nonexistent-cmd --output-format json # exit 1 ERROR_HANDLING.md documented the exit-code contract (1=parse, 2=timeout) but did NOT explicitly state the contract applies only to JSON mode. Text mode follows argparse defaults (exit 2 for any parse error), which violates the documented contract when interpreted generally. A claw using text mode with 'claw nonexistent' would see exit 2 and misclassify as timeout per the docs. Real protocol contract gap, not implementation bug. ## Classification This is a DOCUMENTATION gap, not a behavior bug: - Text mode follows argparse convention (reasonable for humans) - JSON mode normalizes to documented contract (reasonable for claws) - The divergence is intentional; only the docs were silent about it Fix = document the divergence explicitly + lock it with tests. NOT fix = change text mode exit code to 1 (would break argparse conventions and confuse human users). ## Documentation Changes ERROR_HANDLING.md: 1. Added IMPORTANT callout in Quick Reference section: 'The exit code contract applies ONLY when --output-format json is explicitly set. Text mode follows argparse conventions.' 2. New 'Text mode vs JSON mode exit codes' table showing exact divergence: - Unknown subcommand: text=2, json=1 - Missing required arg: text=2, json=1 - Session not found: text=1, json=1 (app-level, identical) - Success: text=0, json=0 (identical) - Timeout: text=2, json=2 (identical, #161) 3. Practical rule: 'always pass --output-format json' ## Tests Added (5) TestTextVsJsonModeDivergence in test_cross_channel_consistency.py: 1. test_unknown_command_text_mode_exits_2 — text mode argparse default 2. test_unknown_command_json_mode_exits_1 — JSON mode contract normalized 3. test_missing_required_arg_text_mode_exits_2 — same for missing args 4. test_missing_required_arg_json_mode_exits_1 — same normalization 5. test_success_path_identical_in_both_modes — success exit identical These tests LOCK the expected divergence so: - Documentation stays aligned with implementation - Future changes (either direction) are caught as intentional - Claws trust the docs ## Test Status - 217 → 222 tests passing (+5) - Zero regressions ## Discipline This cycle follows the cycle #28 template exactly: - Dogfood probe revealed real friction (test said exit=2, docs said exit=1) - Minimal fix shape (documentation clarification, not code change) - Regression guard via tests - Evidence-backed, not speculative Relationship to #181: - #181 fixed env.exit_code != process exit (WITHIN JSON mode) - #29 clarifies exit code contract scope (ONLY JSON mode) - Both establish: exit codes are deterministic, but only when --output-format json --- Classification (per cycle #24 calibration): - Red-state bug? ✗ (behavior was reasonable, docs were incomplete) - Real friction? ✓ (docs/code divergence revealed by dogfood) - Evidence-backed? ✓ (test suite probed both modes, found the gap) Source: Jobdori cycle #29 proactive dogfood — in response to Clawhip nudge for pinpoint hunting. Found that text-mode errors return exit 2 but ERROR_HANDLING.md implied exit 1 was the parse-error contract universally. --- ERROR_HANDLING.md | 14 +++++++ tests/test_cross_channel_consistency.py | 54 +++++++++++++++++++++++++ 2 files changed, 68 insertions(+) diff --git a/ERROR_HANDLING.md b/ERROR_HANDLING.md index 6f284c1..9b5794a 100644 --- a/ERROR_HANDLING.md +++ b/ERROR_HANDLING.md @@ -10,12 +10,26 @@ After cycles #178–#179 (parser-front-door hole closure), claw-code's error int Every clawable command returns JSON on stdout when `--output-format json` is requested. +**IMPORTANT:** The exit code contract below applies **only when `--output-format json` is explicitly set**. Text mode follows argparse conventions and may return different exit codes (e.g., `2` for argparse parse errors). Claws consuming claw-code as a subprocess MUST always pass `--output-format json` to get the documented contract. + | Exit Code | Meaning | Response Format | Example | |---|---|---|---| | **0** | Success | `{success fields}` | `{"session_id": "...", "loaded": true}` | | **1** | Error / Not Found | `{error: {kind, message, ...}}` | `{"error": {"kind": "session_not_found", ...}}` | | **2** | Timeout | `{final_stop_reason: "timeout", final_cancel_observed: ...}` | `{"final_stop_reason": "timeout", ...}` | +### Text mode vs JSON mode exit codes + +| Scenario | Text mode exit | JSON mode exit | Why | +|---|---|---|---| +| Unknown subcommand | 2 (argparse default) | 1 (parse error envelope) | argparse defaults to 2; JSON mode normalizes to contract | +| Missing required arg | 2 (argparse default) | 1 (parse error envelope) | Same reason | +| Session not found | 1 | 1 | Application-level error, same in both | +| Command executed OK | 0 | 0 | Success path, identical | +| Turn-loop timeout | 2 | 2 | Identical (#161 implementation) | + +**Practical rule for claws:** always pass `--output-format json`. This eliminates text-mode surprises and gives you the documented exit-code contract for every error path. + --- ## One-Handler Pattern diff --git a/tests/test_cross_channel_consistency.py b/tests/test_cross_channel_consistency.py index fad6c3d..3426bb3 100644 --- a/tests/test_cross_channel_consistency.py +++ b/tests/test_cross_channel_consistency.py @@ -186,3 +186,57 @@ class TestCrossChannelConsistency: 'Boolean fields must correlate with error block:\n' + '\n'.join(failures) ) + + +class TestTextVsJsonModeDivergence: + """Cycle #29: Document known text-mode vs JSON-mode exit code divergence. + + ERROR_HANDLING.md specifies the exit code contract applies ONLY when + --output-format json is set. Text mode follows argparse defaults (e.g., + exit 2 for parse errors) while JSON mode normalizes to the contract + (exit 1 for parse errors). + + This test class LOCKS the expected divergence so: + 1. Documentation stays aligned with implementation + 2. Future changes to text mode behavior are caught as intentional + 3. Claws consuming subprocess output can trust the docs + """ + + def test_unknown_command_text_mode_exits_2(self) -> None: + """Text mode: argparse default exit 2 for unknown subcommand.""" + result = _run(['nonexistent-cmd']) + assert result.returncode == 2, ( + f'text mode should exit 2 (argparse default), got {result.returncode}' + ) + + def test_unknown_command_json_mode_exits_1(self) -> None: + """JSON mode: normalized exit 1 for parse error (#178).""" + result = _run(['nonexistent-cmd', '--output-format', 'json']) + assert result.returncode == 1, ( + f'JSON mode should exit 1 (protocol contract), got {result.returncode}' + ) + envelope = json.loads(result.stdout) + assert envelope['error']['kind'] == 'parse' + + def test_missing_required_arg_text_mode_exits_2(self) -> None: + """Text mode: argparse default exit 2 for missing required arg.""" + result = _run(['exec-command']) # missing name + prompt + assert result.returncode == 2, ( + f'text mode should exit 2, got {result.returncode}' + ) + + def test_missing_required_arg_json_mode_exits_1(self) -> None: + """JSON mode: normalized exit 1 for parse error.""" + result = _run(['exec-command', '--output-format', 'json']) + assert result.returncode == 1, ( + f'JSON mode should exit 1, got {result.returncode}' + ) + + def test_success_path_identical_in_both_modes(self) -> None: + """Success exit codes are identical in both modes.""" + text_result = _run(['list-sessions']) + json_result = _run(['list-sessions', '--output-format', 'json']) + assert text_result.returncode == json_result.returncode == 0, ( + f'success exit should be 0 in both modes: ' + f'text={text_result.returncode}, json={json_result.returncode}' + )