claw-code/rust/crates/rusty-claude-cli/tests/output_format_contract.rs
YeonGyu-Kim 84466bbb6c fix: #247 classify prompt-related parse errors + unify JSON hint plumbing
Cycle #34 dogfood follow-through on Jobdori cycle #33 pinpoint (#247 filed
at fbcbe9d). Closes the two typed-error contract drifts surfaced in that
pinpoint against the Rust `claw` binary.

## What was wrong

1. `classify_error_kind()` (main.rs:~251) used substring matching but did
   NOT match two common prompt-related parse errors:
     - "prompt subcommand requires a prompt string"
     - "empty prompt: provide a subcommand..."
   Both fell through to `"unknown"`. §4.44 typed-error contract specifies
   `parse | usage | unknown` as distinct classes, so claws dispatching on
   `error.kind == "cli_parse"` missed those paths entirely.

2. JSON mode dropped the `Run `claw --help` for usage.` hint. Text mode
   appends it at stderr-print time (main.rs:~234) AFTER split_error_hint()
   has already serialized the envelope, so JSON consumers never saw it.
   Text-mode humans got an actionable pointer; machine consumers did not.

## Fix

Two small, targeted edits:

1. `classify_error_kind()`: add explicit branches for "prompt subcommand
   requires" and "empty prompt:" (the latter anchored with `starts_with`
   so it never hijacks unrelated error messages containing the word).
   Both route to `cli_parse`.

2. JSON error render path in `main()`: after calling split_error_hint(),
   if the message carried no embedded hint AND kind is `cli_parse` AND
   the short-reason does not already embed a `claw --help` pointer,
   synthesize the same `Run `claw --help` for usage.` trailer that
   text-mode stderr appends. The embedded-pointer check prevents
   duplication on the `empty prompt: ... (run `claw --help`)` message
   which already carries inline guidance.

## Verification

Direct repro on the compiled binary:

    $ claw --output-format json prompt
    {"error":"prompt subcommand requires a prompt string",
     "hint":"Run `claw --help` for usage.",
     "kind":"cli_parse","type":"error"}

    $ claw --output-format json ""
    {"error":"empty prompt: provide a subcommand (run `claw --help`) or a non-empty prompt string",
     "hint":null,"kind":"cli_parse","type":"error"}

    $ claw --output-format json doctor --foo   # regression guard
    {"error":"unrecognized argument `--foo` for subcommand `doctor`",
     "hint":"Run `claw --help` for usage.",
     "kind":"cli_parse","type":"error"}

Text mode unchanged in shape; `[error-kind: ...]` prefix now reads
`cli_parse` for the two previously-misclassified paths.

## Regression coverage

- Unit test `classify_error_kind_covers_prompt_parse_errors_247`: locks
  both patterns route to `cli_parse` AND that generic "prompt"-containing
  messages still fall through to `unknown`.
- Integration tests in `tests/output_format_contract.rs`:
  * prompt_subcommand_without_arg_emits_cli_parse_envelope_with_hint_247
  * empty_positional_arg_emits_cli_parse_envelope_247
  * whitespace_only_positional_arg_emits_cli_parse_envelope_247
  * unrecognized_argument_still_classifies_as_cli_parse_247_regression_guard
- Full rusty-claude-cli test suite: 218 tests pass (180 bin unit + 15
  output_format_contract + 12 resume_slash + 7 compact + 3 mock + 1 cli).

## Family / related

Joins §4.44 typed-envelope contract gap family closure: #130, #179, #181,
and now **#247**. All four quartet items now have real fixes landed on
the canonical binary surface rather than only the Python harness.

ROADMAP.md: #247 marked CLOSED with before/after evidence preserved.
2026-04-22 22:43:14 +09:00

582 lines
20 KiB
Rust

use std::fs;
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{SystemTime, UNIX_EPOCH};
use runtime::Session;
use serde_json::Value;
static TEMP_COUNTER: AtomicU64 = AtomicU64::new(0);
#[test]
fn help_emits_json_when_requested() {
let root = unique_temp_dir("help-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let parsed = assert_json_command(&root, &["--output-format", "json", "help"]);
assert_eq!(parsed["kind"], "help");
assert!(parsed["message"]
.as_str()
.expect("help text")
.contains("Usage:"));
}
#[test]
fn version_emits_json_when_requested() {
let root = unique_temp_dir("version-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let parsed = assert_json_command(&root, &["--output-format", "json", "version"]);
assert_eq!(parsed["kind"], "version");
assert_eq!(parsed["version"], env!("CARGO_PKG_VERSION"));
}
#[test]
fn status_and_sandbox_emit_json_when_requested() {
let root = unique_temp_dir("status-sandbox-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let status = assert_json_command(&root, &["--output-format", "json", "status"]);
assert_eq!(status["kind"], "status");
assert!(status["workspace"]["cwd"].as_str().is_some());
let sandbox = assert_json_command(&root, &["--output-format", "json", "sandbox"]);
assert_eq!(sandbox["kind"], "sandbox");
assert!(sandbox["filesystem_mode"].as_str().is_some());
}
#[test]
fn acp_guidance_emits_json_when_requested() {
let root = unique_temp_dir("acp-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let acp = assert_json_command(&root, &["--output-format", "json", "acp"]);
assert_eq!(acp["kind"], "acp");
assert_eq!(acp["status"], "discoverability_only");
assert_eq!(acp["supported"], false);
assert_eq!(acp["serve_alias_only"], true);
assert_eq!(acp["discoverability_tracking"], "ROADMAP #64a");
assert_eq!(acp["tracking"], "ROADMAP #76");
assert!(acp["message"]
.as_str()
.expect("acp message")
.contains("discoverability alias"));
}
#[test]
fn inventory_commands_emit_structured_json_when_requested() {
let root = unique_temp_dir("inventory-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let isolated_home = root.join("home");
let isolated_config = root.join("config-home");
let isolated_codex = root.join("codex-home");
fs::create_dir_all(&isolated_home).expect("isolated home should exist");
let agents = assert_json_command_with_env(
&root,
&["--output-format", "json", "agents"],
&[
("HOME", isolated_home.to_str().expect("utf8 home")),
(
"CLAW_CONFIG_HOME",
isolated_config.to_str().expect("utf8 config home"),
),
(
"CODEX_HOME",
isolated_codex.to_str().expect("utf8 codex home"),
),
],
);
assert_eq!(agents["kind"], "agents");
assert_eq!(agents["action"], "list");
assert_eq!(agents["count"], 0);
assert_eq!(agents["summary"]["active"], 0);
assert!(agents["agents"]
.as_array()
.expect("agents array")
.is_empty());
let mcp = assert_json_command(&root, &["--output-format", "json", "mcp"]);
assert_eq!(mcp["kind"], "mcp");
assert_eq!(mcp["action"], "list");
let skills = assert_json_command(&root, &["--output-format", "json", "skills"]);
assert_eq!(skills["kind"], "skills");
assert_eq!(skills["action"], "list");
}
#[test]
fn agents_command_emits_structured_agent_entries_when_requested() {
let root = unique_temp_dir("agents-json-populated");
let workspace = root.join("workspace");
let project_agents = workspace.join(".codex").join("agents");
let home = root.join("home");
let user_agents = home.join(".codex").join("agents");
let isolated_config = root.join("config-home");
let isolated_codex = root.join("codex-home");
fs::create_dir_all(&workspace).expect("workspace should exist");
write_agent(
&project_agents,
"planner",
"Project planner",
"gpt-5.4",
"medium",
);
write_agent(
&project_agents,
"verifier",
"Verification agent",
"gpt-5.4-mini",
"high",
);
write_agent(
&user_agents,
"planner",
"User planner",
"gpt-5.4-mini",
"high",
);
let parsed = assert_json_command_with_env(
&workspace,
&["--output-format", "json", "agents"],
&[
("HOME", home.to_str().expect("utf8 home")),
(
"CLAW_CONFIG_HOME",
isolated_config.to_str().expect("utf8 config home"),
),
(
"CODEX_HOME",
isolated_codex.to_str().expect("utf8 codex home"),
),
],
);
assert_eq!(parsed["kind"], "agents");
assert_eq!(parsed["action"], "list");
assert_eq!(parsed["count"], 3);
assert_eq!(parsed["summary"]["active"], 2);
assert_eq!(parsed["summary"]["shadowed"], 1);
assert_eq!(parsed["agents"][0]["name"], "planner");
assert_eq!(parsed["agents"][0]["source"]["id"], "project_claw");
assert_eq!(parsed["agents"][0]["active"], true);
assert_eq!(parsed["agents"][1]["name"], "verifier");
assert_eq!(parsed["agents"][2]["name"], "planner");
assert_eq!(parsed["agents"][2]["active"], false);
assert_eq!(parsed["agents"][2]["shadowed_by"]["id"], "project_claw");
}
#[test]
fn bootstrap_and_system_prompt_emit_json_when_requested() {
let root = unique_temp_dir("bootstrap-system-prompt-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let plan = assert_json_command(&root, &["--output-format", "json", "bootstrap-plan"]);
assert_eq!(plan["kind"], "bootstrap-plan");
assert!(plan["phases"].as_array().expect("phases").len() > 1);
let prompt = assert_json_command(&root, &["--output-format", "json", "system-prompt"]);
assert_eq!(prompt["kind"], "system-prompt");
assert!(prompt["message"]
.as_str()
.expect("prompt text")
.contains("interactive agent"));
}
#[test]
fn dump_manifests_and_init_emit_json_when_requested() {
let root = unique_temp_dir("manifest-init-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let upstream = write_upstream_fixture(&root);
let manifests = assert_json_command(
&root,
&[
"--output-format",
"json",
"dump-manifests",
"--manifests-dir",
upstream.to_str().expect("utf8 upstream"),
],
);
assert_eq!(manifests["kind"], "dump-manifests");
assert_eq!(manifests["commands"], 1);
assert_eq!(manifests["tools"], 1);
let workspace = root.join("workspace");
fs::create_dir_all(&workspace).expect("workspace should exist");
let init = assert_json_command(&workspace, &["--output-format", "json", "init"]);
assert_eq!(init["kind"], "init");
assert!(workspace.join("CLAUDE.md").exists());
}
#[test]
fn doctor_and_resume_status_emit_json_when_requested() {
let root = unique_temp_dir("doctor-resume-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let doctor = assert_json_command(&root, &["--output-format", "json", "doctor"]);
assert_eq!(doctor["kind"], "doctor");
assert!(doctor["message"].is_string());
let summary = doctor["summary"].as_object().expect("doctor summary");
assert!(summary["ok"].as_u64().is_some());
assert!(summary["warnings"].as_u64().is_some());
assert!(summary["failures"].as_u64().is_some());
let checks = doctor["checks"].as_array().expect("doctor checks");
assert_eq!(checks.len(), 6);
let check_names = checks
.iter()
.map(|check| {
assert!(check["status"].as_str().is_some());
assert!(check["summary"].as_str().is_some());
assert!(check["details"].is_array());
check["name"].as_str().expect("doctor check name")
})
.collect::<Vec<_>>();
assert_eq!(
check_names,
vec![
"auth",
"config",
"install source",
"workspace",
"sandbox",
"system"
]
);
let install_source = checks
.iter()
.find(|check| check["name"] == "install source")
.expect("install source check");
assert_eq!(
install_source["official_repo"],
"https://github.com/ultraworkers/claw-code"
);
assert_eq!(
install_source["deprecated_install"],
"cargo install claw-code"
);
let workspace = checks
.iter()
.find(|check| check["name"] == "workspace")
.expect("workspace check");
assert!(workspace["cwd"].as_str().is_some());
assert!(workspace["in_git_repo"].is_boolean());
let sandbox = checks
.iter()
.find(|check| check["name"] == "sandbox")
.expect("sandbox check");
assert!(sandbox["filesystem_mode"].as_str().is_some());
assert!(sandbox["enabled"].is_boolean());
assert!(sandbox["fallback_reason"].is_null() || sandbox["fallback_reason"].is_string());
let session_path = write_session_fixture(&root, "resume-json", Some("hello"));
let resumed = assert_json_command(
&root,
&[
"--output-format",
"json",
"--resume",
session_path.to_str().expect("utf8 session path"),
"/status",
],
);
assert_eq!(resumed["kind"], "status");
// model is null in resume mode (not known without --model flag)
assert!(resumed["model"].is_null());
assert_eq!(resumed["usage"]["messages"], 1);
assert!(resumed["workspace"]["cwd"].as_str().is_some());
assert!(resumed["sandbox"]["filesystem_mode"].as_str().is_some());
}
#[test]
fn resumed_inventory_commands_emit_structured_json_when_requested() {
let root = unique_temp_dir("resume-inventory-json");
let config_home = root.join("config-home");
let home = root.join("home");
fs::create_dir_all(&config_home).expect("config home should exist");
fs::create_dir_all(&home).expect("home should exist");
let session_path = write_session_fixture(&root, "resume-inventory-json", Some("inventory"));
let mcp = assert_json_command_with_env(
&root,
&[
"--output-format",
"json",
"--resume",
session_path.to_str().expect("utf8 session path"),
"/mcp",
],
&[
(
"CLAW_CONFIG_HOME",
config_home.to_str().expect("utf8 config home"),
),
("HOME", home.to_str().expect("utf8 home")),
],
);
assert_eq!(mcp["kind"], "mcp");
assert_eq!(mcp["action"], "list");
assert!(mcp["servers"].is_array());
let skills = assert_json_command_with_env(
&root,
&[
"--output-format",
"json",
"--resume",
session_path.to_str().expect("utf8 session path"),
"/skills",
],
&[
(
"CLAW_CONFIG_HOME",
config_home.to_str().expect("utf8 config home"),
),
("HOME", home.to_str().expect("utf8 home")),
],
);
assert_eq!(skills["kind"], "skills");
assert_eq!(skills["action"], "list");
assert!(skills["summary"]["total"].is_number());
assert!(skills["skills"].is_array());
}
#[test]
fn resumed_version_and_init_emit_structured_json_when_requested() {
let root = unique_temp_dir("resume-version-init-json");
fs::create_dir_all(&root).expect("temp dir should exist");
let session_path = write_session_fixture(&root, "resume-version-init-json", None);
let version = assert_json_command(
&root,
&[
"--output-format",
"json",
"--resume",
session_path.to_str().expect("utf8 session path"),
"/version",
],
);
assert_eq!(version["kind"], "version");
assert_eq!(version["version"], env!("CARGO_PKG_VERSION"));
let init = assert_json_command(
&root,
&[
"--output-format",
"json",
"--resume",
session_path.to_str().expect("utf8 session path"),
"/init",
],
);
assert_eq!(init["kind"], "init");
assert!(root.join("CLAUDE.md").exists());
}
fn assert_json_command(current_dir: &Path, args: &[&str]) -> Value {
assert_json_command_with_env(current_dir, args, &[])
}
/// #247 regression helper: run claw expecting a non-zero exit and return
/// the JSON error envelope parsed from stderr. Asserts exit != 0 and that
/// the envelope includes `type: "error"` at the very least.
fn assert_json_error_envelope(current_dir: &Path, args: &[&str]) -> Value {
let output = run_claw(current_dir, args, &[]);
assert!(
!output.status.success(),
"command unexpectedly succeeded; stdout:\n{}\nstderr:\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
// The JSON envelope is written to stderr for error cases (see main.rs).
let envelope: Value = serde_json::from_slice(&output.stderr).unwrap_or_else(|err| {
panic!(
"stderr should be a JSON error envelope but failed to parse: {err}\nstderr bytes:\n{}",
String::from_utf8_lossy(&output.stderr)
)
});
assert_eq!(
envelope["type"], "error",
"envelope should carry type=error"
);
envelope
}
#[test]
fn prompt_subcommand_without_arg_emits_cli_parse_envelope_with_hint_247() {
// #247: `claw prompt` with no argument must classify as `cli_parse`
// (not `unknown`) and the JSON envelope must carry the same actionable
// `Run claw --help for usage.` hint that text-mode stderr appends.
let root = unique_temp_dir("247-prompt-no-arg");
fs::create_dir_all(&root).expect("temp dir should exist");
let envelope = assert_json_error_envelope(&root, &["--output-format", "json", "prompt"]);
assert_eq!(
envelope["kind"], "cli_parse",
"prompt subcommand without arg should classify as cli_parse, envelope: {envelope}"
);
assert_eq!(
envelope["error"], "prompt subcommand requires a prompt string",
"short reason should match the raw error, envelope: {envelope}"
);
assert_eq!(
envelope["hint"],
"Run `claw --help` for usage.",
"JSON envelope must carry the same help-runbook hint as text mode, envelope: {envelope}"
);
}
#[test]
fn empty_positional_arg_emits_cli_parse_envelope_247() {
// #247: `claw ""` must classify as `cli_parse`, not `unknown`. The
// message itself embeds a ``run `claw --help`` pointer so the explicit
// hint field is allowed to remain null to avoid duplication — what
// matters for the typed-error contract is that `kind == cli_parse`.
let root = unique_temp_dir("247-empty-arg");
fs::create_dir_all(&root).expect("temp dir should exist");
let envelope = assert_json_error_envelope(&root, &["--output-format", "json", ""]);
assert_eq!(
envelope["kind"], "cli_parse",
"empty-prompt error should classify as cli_parse, envelope: {envelope}"
);
let short = envelope["error"]
.as_str()
.expect("error field should be a string");
assert!(
short.starts_with("empty prompt:"),
"short reason should preserve the original empty-prompt message, got: {short}"
);
}
#[test]
fn whitespace_only_positional_arg_emits_cli_parse_envelope_247() {
// #247: same rule for `claw " "` — any whitespace-only prompt must
// flow through the empty-prompt path and classify as `cli_parse`.
let root = unique_temp_dir("247-whitespace-arg");
fs::create_dir_all(&root).expect("temp dir should exist");
let envelope = assert_json_error_envelope(&root, &["--output-format", "json", " "]);
assert_eq!(
envelope["kind"], "cli_parse",
"whitespace-only prompt should classify as cli_parse, envelope: {envelope}"
);
}
#[test]
fn unrecognized_argument_still_classifies_as_cli_parse_247_regression_guard() {
// #247 regression guard: the new empty-prompt / prompt-subcommand
// patterns must NOT hijack the existing #77 unrecognized-argument
// classification. `claw doctor --foo` must still surface as cli_parse
// with the runbook hint present.
let root = unique_temp_dir("247-unrecognized-arg");
fs::create_dir_all(&root).expect("temp dir should exist");
let envelope =
assert_json_error_envelope(&root, &["--output-format", "json", "doctor", "--foo"]);
assert_eq!(
envelope["kind"], "cli_parse",
"unrecognized-argument must remain cli_parse, envelope: {envelope}"
);
assert_eq!(
envelope["hint"],
"Run `claw --help` for usage.",
"unrecognized-argument hint should stay intact, envelope: {envelope}"
);
}
fn assert_json_command_with_env(current_dir: &Path, args: &[&str], envs: &[(&str, &str)]) -> Value {
let output = run_claw(current_dir, args, envs);
assert!(
output.status.success(),
"stdout:\n{}\n\nstderr:\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
serde_json::from_slice(&output.stdout).expect("stdout should be valid json")
}
fn run_claw(current_dir: &Path, args: &[&str], envs: &[(&str, &str)]) -> Output {
let mut command = Command::new(env!("CARGO_BIN_EXE_claw"));
command.current_dir(current_dir).args(args);
for (key, value) in envs {
command.env(key, value);
}
command.output().expect("claw should launch")
}
fn write_upstream_fixture(root: &Path) -> PathBuf {
let upstream = root.join("claw-code");
let src = upstream.join("src");
let entrypoints = src.join("entrypoints");
fs::create_dir_all(&entrypoints).expect("upstream entrypoints dir should exist");
fs::write(
src.join("commands.ts"),
"import FooCommand from './commands/foo'\n",
)
.expect("commands fixture should write");
fs::write(
src.join("tools.ts"),
"import ReadTool from './tools/read'\n",
)
.expect("tools fixture should write");
fs::write(
entrypoints.join("cli.tsx"),
"if (args[0] === '--version') {}\nstartupProfiler()\n",
)
.expect("cli fixture should write");
upstream
}
fn write_session_fixture(root: &Path, session_id: &str, user_text: Option<&str>) -> PathBuf {
let session_path = root.join("session.jsonl");
let mut session = Session::new()
.with_workspace_root(root.to_path_buf())
.with_persistence_path(session_path.clone());
session.session_id = session_id.to_string();
if let Some(text) = user_text {
session
.push_user_text(text)
.expect("session fixture message should persist");
} else {
session
.save_to_path(&session_path)
.expect("session fixture should persist");
}
session_path
}
fn write_agent(root: &Path, name: &str, description: &str, model: &str, reasoning: &str) {
fs::create_dir_all(root).expect("agent root should exist");
fs::write(
root.join(format!("{name}.toml")),
format!(
"name = \"{name}\"\ndescription = \"{description}\"\nmodel = \"{model}\"\nmodel_reasoning_effort = \"{reasoning}\"\n"
),
)
.expect("agent fixture should write");
}
fn unique_temp_dir(label: &str) -> PathBuf {
let millis = SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("clock should be after epoch")
.as_millis();
let counter = TEMP_COUNTER.fetch_add(1, Ordering::Relaxed);
std::env::temp_dir().join(format!(
"claw-output-format-{label}-{}-{millis}-{counter}",
std::process::id()
))
}