docs: add Ollama streaming NDJSON issue guide and workaround (#1197)
* docs: add Ollama streaming NDJSON issue troubleshooting guide - Document problem: JSON Parse error when using Ollama with stream: true - Explain root cause: NDJSON vs single JSON object mismatch - Provide 3 solutions: disable streaming, avoid tool agents, wait for SDK fix - Include NDJSON parsing code example for SDK maintainers - Add curl testing command for verification - Link to issue #1124 and Ollama API docs Fixes #1124 * docs: add Ollama provider configuration with streaming workaround - Add Ollama Provider section to configurations.md - Document stream: false requirement for Ollama - Explain NDJSON vs single JSON mismatch - Provide supported models table (qwen3-coder, ministral-3, lfm2.5-thinking) - Add troubleshooting steps and curl test command - Link to troubleshooting guide feat: add NDJSON parser utility for Ollama streaming responses - Create src/shared/ollama-ndjson-parser.ts - Implement parseOllamaStreamResponse() for merging NDJSON lines - Implement isNDJSONResponse() for format detection - Add TypeScript interfaces for Ollama message structures - Include JSDoc with usage examples - Handle edge cases: malformed lines, stats aggregation This utility can be contributed to Claude Code SDK for proper NDJSON support. Related to #1124 * fix: use logger instead of console, remove trailing whitespace - Replace console.warn with log() from shared/logger - Remove trailing whitespace from troubleshooting guide - Ensure TypeScript compatibility
This commit is contained in:
parent
acc19fcd41
commit
895f366a11
@ -85,6 +85,66 @@ When both `oh-my-opencode.jsonc` and `oh-my-opencode.json` files exist, `.jsonc`
|
|||||||
|
|
||||||
**Recommended**: For Google Gemini authentication, install the [`opencode-antigravity-auth`](https://github.com/NoeFabris/opencode-antigravity-auth) plugin (`@latest`). It provides multi-account load balancing, variant-based thinking levels, dual quota system (Antigravity + Gemini CLI), and active maintenance. See [Installation > Google Gemini](docs/guide/installation.md#google-gemini-antigravity-oauth).
|
**Recommended**: For Google Gemini authentication, install the [`opencode-antigravity-auth`](https://github.com/NoeFabris/opencode-antigravity-auth) plugin (`@latest`). It provides multi-account load balancing, variant-based thinking levels, dual quota system (Antigravity + Gemini CLI), and active maintenance. See [Installation > Google Gemini](docs/guide/installation.md#google-gemini-antigravity-oauth).
|
||||||
|
|
||||||
|
## Ollama Provider
|
||||||
|
|
||||||
|
**IMPORTANT**: When using Ollama as a provider, you **must** disable streaming to avoid JSON parsing errors.
|
||||||
|
|
||||||
|
### Required Configuration
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"agents": {
|
||||||
|
"explore": {
|
||||||
|
"model": "ollama/qwen3-coder",
|
||||||
|
"stream": false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Why `stream: false` is Required
|
||||||
|
|
||||||
|
Ollama returns NDJSON (newline-delimited JSON) when streaming is enabled, but Claude Code SDK expects a single JSON object. This causes `JSON Parse error: Unexpected EOF` when agents attempt tool calls.
|
||||||
|
|
||||||
|
**Example of the problem**:
|
||||||
|
```json
|
||||||
|
// Ollama streaming response (NDJSON - multiple lines)
|
||||||
|
{"message":{"tool_calls":[...]}, "done":false}
|
||||||
|
{"message":{"content":""}, "done":true}
|
||||||
|
|
||||||
|
// Claude Code SDK expects (single JSON object)
|
||||||
|
{"message":{"tool_calls":[...], "content":""}, "done":true}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Supported Models
|
||||||
|
|
||||||
|
Common Ollama models that work with oh-my-opencode:
|
||||||
|
|
||||||
|
| Model | Best For | Configuration |
|
||||||
|
|-------|----------|---------------|
|
||||||
|
| `ollama/qwen3-coder` | Code generation, build fixes | `{"model": "ollama/qwen3-coder", "stream": false}` |
|
||||||
|
| `ollama/ministral-3:14b` | Exploration, codebase search | `{"model": "ollama/ministral-3:14b", "stream": false}` |
|
||||||
|
| `ollama/lfm2.5-thinking` | Documentation, writing | `{"model": "ollama/lfm2.5-thinking", "stream": false}` |
|
||||||
|
|
||||||
|
### Troubleshooting
|
||||||
|
|
||||||
|
If you encounter `JSON Parse error: Unexpected EOF`:
|
||||||
|
|
||||||
|
1. **Verify `stream: false` is set** in your agent configuration
|
||||||
|
2. **Check Ollama is running**: `curl http://localhost:11434/api/tags`
|
||||||
|
3. **Test with curl**:
|
||||||
|
```bash
|
||||||
|
curl -s http://localhost:11434/api/chat \
|
||||||
|
-d '{"model": "qwen3-coder", "messages": [{"role": "user", "content": "Hello"}], "stream": false}'
|
||||||
|
```
|
||||||
|
4. **See detailed troubleshooting**: [docs/troubleshooting/ollama-streaming-issue.md](troubleshooting/ollama-streaming-issue.md)
|
||||||
|
|
||||||
|
### Future SDK Fix
|
||||||
|
|
||||||
|
The proper long-term fix requires Claude Code SDK to parse NDJSON responses correctly. Until then, use `stream: false` as a workaround.
|
||||||
|
|
||||||
|
**Tracking**: https://github.com/code-yeongyu/oh-my-opencode/issues/1124
|
||||||
|
|
||||||
## Agents
|
## Agents
|
||||||
|
|
||||||
Override built-in agent settings:
|
Override built-in agent settings:
|
||||||
|
|||||||
126
docs/troubleshooting/ollama-streaming-issue.md
Normal file
126
docs/troubleshooting/ollama-streaming-issue.md
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
# Ollama Streaming Issue - JSON Parse Error
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
When using Ollama as a provider with oh-my-opencode agents, you may encounter:
|
||||||
|
|
||||||
|
```
|
||||||
|
JSON Parse error: Unexpected EOF
|
||||||
|
```
|
||||||
|
|
||||||
|
This occurs when agents attempt tool calls (e.g., `explore` agent using `mcp_grep_search`).
|
||||||
|
|
||||||
|
## Root Cause
|
||||||
|
|
||||||
|
Ollama returns **NDJSON** (newline-delimited JSON) when `stream: true` is used in API requests:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"message":{"tool_calls":[{"function":{"name":"read","arguments":{"filePath":"README.md"}}}]}, "done":false}
|
||||||
|
{"message":{"content":""}, "done":true}
|
||||||
|
```
|
||||||
|
|
||||||
|
Claude Code SDK expects a single JSON object, not multiple NDJSON lines, causing the parse error.
|
||||||
|
|
||||||
|
### Why This Happens
|
||||||
|
|
||||||
|
- **Ollama API**: Returns streaming responses as NDJSON by design
|
||||||
|
- **Claude Code SDK**: Doesn't properly handle NDJSON responses for tool calls
|
||||||
|
- **oh-my-opencode**: Passes through the SDK's behavior (can't fix at this layer)
|
||||||
|
|
||||||
|
## Solutions
|
||||||
|
|
||||||
|
### Option 1: Disable Streaming (Recommended - Immediate Fix)
|
||||||
|
|
||||||
|
Configure your Ollama provider to use `stream: false`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"provider": "ollama",
|
||||||
|
"model": "qwen3-coder",
|
||||||
|
"stream": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pros:**
|
||||||
|
- Works immediately
|
||||||
|
- No code changes needed
|
||||||
|
- Simple configuration
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Slightly slower response time (no streaming)
|
||||||
|
- Less interactive feedback
|
||||||
|
|
||||||
|
### Option 2: Use Non-Tool Agents Only
|
||||||
|
|
||||||
|
If you need streaming, avoid agents that use tools:
|
||||||
|
|
||||||
|
- ✅ **Safe**: Simple text generation, non-tool tasks
|
||||||
|
- ❌ **Problematic**: Any agent with tool calls (explore, librarian, etc.)
|
||||||
|
|
||||||
|
### Option 3: Wait for SDK Fix (Long-term)
|
||||||
|
|
||||||
|
The proper fix requires Claude Code SDK to:
|
||||||
|
|
||||||
|
1. Detect NDJSON responses
|
||||||
|
2. Parse each line separately
|
||||||
|
3. Merge `tool_calls` from multiple lines
|
||||||
|
4. Return a single merged response
|
||||||
|
|
||||||
|
**Tracking**: https://github.com/code-yeongyu/oh-my-opencode/issues/1124
|
||||||
|
|
||||||
|
## Workaround Implementation
|
||||||
|
|
||||||
|
Until the SDK is fixed, here's how to implement NDJSON parsing (for SDK maintainers):
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
async function parseOllamaStreamResponse(response: string): Promise<object> {
|
||||||
|
const lines = response.split('\n').filter(line => line.trim());
|
||||||
|
const mergedMessage = { tool_calls: [] };
|
||||||
|
|
||||||
|
for (const line of lines) {
|
||||||
|
try {
|
||||||
|
const json = JSON.parse(line);
|
||||||
|
if (json.message?.tool_calls) {
|
||||||
|
mergedMessage.tool_calls.push(...json.message.tool_calls);
|
||||||
|
}
|
||||||
|
if (json.message?.content) {
|
||||||
|
mergedMessage.content = json.message.content;
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// Skip malformed lines
|
||||||
|
console.warn('Skipping malformed NDJSON line:', line);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return mergedMessage;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
To verify the fix works:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Test with curl (should work with stream: false)
|
||||||
|
curl -s http://localhost:11434/api/chat \
|
||||||
|
-d '{
|
||||||
|
"model": "qwen3-coder",
|
||||||
|
"messages": [{"role": "user", "content": "Read file README.md"}],
|
||||||
|
"stream": false,
|
||||||
|
"tools": [{"type": "function", "function": {"name": "read", "description": "Read a file", "parameters": {"type": "object", "properties": {"filePath": {"type": "string"}}, "required": ["filePath"]}}}]
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Related Issues
|
||||||
|
|
||||||
|
- **oh-my-opencode**: https://github.com/code-yeongyu/oh-my-opencode/issues/1124
|
||||||
|
- **Ollama API Docs**: https://github.com/ollama/ollama/blob/main/docs/api.md
|
||||||
|
|
||||||
|
## Getting Help
|
||||||
|
|
||||||
|
If you encounter this issue:
|
||||||
|
|
||||||
|
1. Check your Ollama provider configuration
|
||||||
|
2. Set `stream: false` as a workaround
|
||||||
|
3. Report any additional errors to the issue tracker
|
||||||
|
4. Provide your configuration (without secrets) for debugging
|
||||||
198
src/shared/ollama-ndjson-parser.ts
Normal file
198
src/shared/ollama-ndjson-parser.ts
Normal file
@ -0,0 +1,198 @@
|
|||||||
|
/**
|
||||||
|
* Ollama NDJSON Parser
|
||||||
|
*
|
||||||
|
* Parses newline-delimited JSON (NDJSON) responses from Ollama API.
|
||||||
|
*
|
||||||
|
* @module ollama-ndjson-parser
|
||||||
|
* @see https://github.com/code-yeongyu/oh-my-opencode/issues/1124
|
||||||
|
* @see https://github.com/ollama/ollama/blob/main/docs/api.md
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { log } from "./logger"
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ollama message structure
|
||||||
|
*/
|
||||||
|
export interface OllamaMessage {
|
||||||
|
tool_calls?: Array<{
|
||||||
|
function: {
|
||||||
|
name: string
|
||||||
|
arguments: Record<string, unknown>
|
||||||
|
}
|
||||||
|
}>
|
||||||
|
content?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ollama NDJSON line structure
|
||||||
|
*/
|
||||||
|
export interface OllamaNDJSONLine {
|
||||||
|
message?: OllamaMessage
|
||||||
|
done: boolean
|
||||||
|
total_duration?: number
|
||||||
|
load_duration?: number
|
||||||
|
prompt_eval_count?: number
|
||||||
|
prompt_eval_duration?: number
|
||||||
|
eval_count?: number
|
||||||
|
eval_duration?: number
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Merged Ollama response
|
||||||
|
*/
|
||||||
|
export interface OllamaMergedResponse {
|
||||||
|
message: OllamaMessage
|
||||||
|
done: boolean
|
||||||
|
stats?: {
|
||||||
|
total_duration?: number
|
||||||
|
load_duration?: number
|
||||||
|
prompt_eval_count?: number
|
||||||
|
prompt_eval_duration?: number
|
||||||
|
eval_count?: number
|
||||||
|
eval_duration?: number
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse Ollama streaming NDJSON response into a single merged object.
|
||||||
|
*
|
||||||
|
* Ollama returns streaming responses as newline-delimited JSON (NDJSON):
|
||||||
|
* ```
|
||||||
|
* {"message":{"tool_calls":[...]}, "done":false}
|
||||||
|
* {"message":{"content":""}, "done":true}
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* This function:
|
||||||
|
* 1. Splits the response by newlines
|
||||||
|
* 2. Parses each line as JSON
|
||||||
|
* 3. Merges tool_calls and content from all lines
|
||||||
|
* 4. Returns a single merged response
|
||||||
|
*
|
||||||
|
* @param response - Raw NDJSON response string from Ollama API
|
||||||
|
* @returns Merged response with all tool_calls and content combined
|
||||||
|
* @throws {Error} If no valid JSON lines are found
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* const ndjsonResponse = `
|
||||||
|
* {"message":{"tool_calls":[{"function":{"name":"read","arguments":{"filePath":"README.md"}}}]}, "done":false}
|
||||||
|
* {"message":{"content":""}, "done":true}
|
||||||
|
* `;
|
||||||
|
*
|
||||||
|
* const merged = parseOllamaStreamResponse(ndjsonResponse);
|
||||||
|
* // Result:
|
||||||
|
* // {
|
||||||
|
* // message: {
|
||||||
|
* // tool_calls: [{ function: { name: "read", arguments: { filePath: "README.md" } } }],
|
||||||
|
* // content: ""
|
||||||
|
* // },
|
||||||
|
* // done: true
|
||||||
|
* // }
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export function parseOllamaStreamResponse(response: string): OllamaMergedResponse {
|
||||||
|
const lines = response.split("\n").filter((line) => line.trim())
|
||||||
|
|
||||||
|
if (lines.length === 0) {
|
||||||
|
throw new Error("No valid NDJSON lines found in response")
|
||||||
|
}
|
||||||
|
|
||||||
|
const mergedMessage: OllamaMessage = {
|
||||||
|
tool_calls: [],
|
||||||
|
content: "",
|
||||||
|
}
|
||||||
|
|
||||||
|
let done = false
|
||||||
|
let stats: OllamaMergedResponse["stats"] = {}
|
||||||
|
|
||||||
|
for (const line of lines) {
|
||||||
|
try {
|
||||||
|
const json = JSON.parse(line) as OllamaNDJSONLine
|
||||||
|
|
||||||
|
// Merge tool_calls
|
||||||
|
if (json.message?.tool_calls) {
|
||||||
|
mergedMessage.tool_calls = [
|
||||||
|
...(mergedMessage.tool_calls || []),
|
||||||
|
...json.message.tool_calls,
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
// Merge content (concatenate)
|
||||||
|
if (json.message?.content) {
|
||||||
|
mergedMessage.content = (mergedMessage.content || "") + json.message.content
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update done flag (final line has done: true)
|
||||||
|
if (json.done) {
|
||||||
|
done = true
|
||||||
|
|
||||||
|
// Capture stats from final line
|
||||||
|
stats = {
|
||||||
|
total_duration: json.total_duration,
|
||||||
|
load_duration: json.load_duration,
|
||||||
|
prompt_eval_count: json.prompt_eval_count,
|
||||||
|
prompt_eval_duration: json.prompt_eval_duration,
|
||||||
|
eval_count: json.eval_count,
|
||||||
|
eval_duration: json.eval_duration,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
log(`[ollama-ndjson-parser] Skipping malformed NDJSON line: ${line}`, { error })
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
message: mergedMessage,
|
||||||
|
done,
|
||||||
|
...(Object.keys(stats).length > 0 ? { stats } : {}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if a response string is NDJSON format.
|
||||||
|
*
|
||||||
|
* NDJSON is identified by:
|
||||||
|
* - Multiple lines
|
||||||
|
* - Each line is valid JSON
|
||||||
|
* - At least one line has "done" field
|
||||||
|
*
|
||||||
|
* @param response - Response string to check
|
||||||
|
* @returns true if response appears to be NDJSON
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* const ndjson = '{"done":false}\n{"done":true}';
|
||||||
|
* const singleJson = '{"done":true}';
|
||||||
|
*
|
||||||
|
* isNDJSONResponse(ndjson); // true
|
||||||
|
* isNDJSONResponse(singleJson); // false
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export function isNDJSONResponse(response: string): boolean {
|
||||||
|
const lines = response.split("\n").filter((line) => line.trim())
|
||||||
|
|
||||||
|
// Single line is not NDJSON
|
||||||
|
if (lines.length <= 1) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
let hasValidJSON = false
|
||||||
|
let hasDoneField = false
|
||||||
|
|
||||||
|
for (const line of lines) {
|
||||||
|
try {
|
||||||
|
const json = JSON.parse(line) as Record<string, unknown>
|
||||||
|
hasValidJSON = true
|
||||||
|
|
||||||
|
if ("done" in json) {
|
||||||
|
hasDoneField = true
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// If any line fails to parse, it's not NDJSON
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return hasValidJSON && hasDoneField
|
||||||
|
}
|
||||||
Loading…
x
Reference in New Issue
Block a user