docs: record argv-safe dogfood probe gap (#3186)

This commit is contained in:
Bellman 2026-05-28 15:34:04 +09:00 committed by GitHub
parent b7ea04661a
commit 0e6d48d9dc
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -7794,3 +7794,9 @@ Original filing (2026-04-18): the session emitted `SessionStart hook (completed)
**Acceptance.** `timeout 5s cargo run -q --bin claw -- --output-format json doctor --help` exits 0 and parses as JSON with `.kind=="help"`, `.command=="doctor"`, `.local_only==true`, `.requires_provider_request==false`, and `output_fields` containing `checks`. `timeout 5s cargo run -q --bin claw -- doctor --help` exits 0 with plaintext beginning `Doctor` and no JSON parsing requirement. Neither command starts a provider request or session resume. **Acceptance.** `timeout 5s cargo run -q --bin claw -- --output-format json doctor --help` exits 0 and parses as JSON with `.kind=="help"`, `.command=="doctor"`, `.local_only==true`, `.requires_provider_request==false`, and `output_fields` containing `checks`. `timeout 5s cargo run -q --bin claw -- doctor --help` exits 0 with plaintext beginning `Doctor` and no JSON parsing requirement. Neither command starts a provider request or session resume.
**Verification.** Regression tests: `doctor_help_json_is_local_structured_and_bounded_702` and `doctor_help_text_stays_plaintext_and_local_702` in `rust/crates/rusty-claude-cli/tests/output_format_contract.rs`; focused command repros recorded in `/tmp/claw_doctor_help_json.out` and `/tmp/claw_doctor_help_json.err` during the doctor-help fix branch. **Verification.** Regression tests: `doctor_help_json_is_local_structured_and_bounded_702` and `doctor_help_text_stays_plaintext_and_local_702` in `rust/crates/rusty-claude-cli/tests/output_format_contract.rs`; focused command repros recorded in `/tmp/claw_doctor_help_json.out` and `/tmp/claw_doctor_help_json.err` during the doctor-help fix branch.
813. **Dogfood probe shell-string loops can fabricate CLI argv failures for JSON help surfaces** — dogfooded 2026-05-28 after #3185 merged. A verification loop used a single shell string variable (`cmd="--output-format json doctor --help"`; then `cargo run -q --bin claw -- $cmd | python3 -c 'json.load(...)'`). The resulting channel transcript showed `unknown option: --output-format json doctor --help` and Python JSON parse stack noise, even though explicit argv invocations on fresh `main` all returned valid JSON: `cargo run -q --bin claw -- --output-format json doctor --help`, `cargo run -q --bin claw -- doctor --help --output-format json`, and `cargo run -q --bin claw -- help doctor --output-format json`. This is a dogfood-harness test-brittleness / event-log opacity gap, not a product parser regression. The log made a probe-construction mistake look like a claw-code failure.
**Required fix shape.** Add a tiny argv-safe dogfood helper (script or documented recipe) that runs CLI probes as explicit argv arrays rather than interpolated shell strings, captures stdout/stderr separately, and labels probe-construction failures distinctly from product failures. For ad-hoc shell loops, prefer arrays/functions (`run_probe --output-format json doctor --help`) over `$cmd` strings; never pipe unknown stdout directly into a JSON parser without first recording rc/stdout/stderr.
**Acceptance.** Future dogfood reports for argv-sensitive CLI surfaces include the exact argv vector and can distinguish `probe_error` from `product_error`; reproducing the three doctor-help forms through the helper yields three parseable JSON objects from byte 0 without Python parser stack noise. [SCOPE: claw-code dogfood harness]