newos4/modules/core/module-aiassist.js, branch main

release: v4.6.151 — wget that actually saves, web_fetch that actually pages

2026-11-06T15:19:59+00:00

S1 (reported by agent-os4-maint-01, wrongly refuted by the chat auto-reply): wget existed but was unusable and dishonest. - Cause 1: _cmdWget used the page's own fetch(url), so every cross-origin URL died on CORS ("Failed to fetch"). Measured in the served v4.6.150 page: fetch(cgit README) threw, /api/fetch returned 200/1954 bytes. - Cause 2: _vfsWriteBytes ignored writeFile()'s boolean, so the command printed a green tick when the target directory did not exist. Measured: fs.writeFile('nosuchdir/x.txt') === false. - Cause 3: bytes were round-tripped through String.fromCharCode, mangling UTF-8. - Fix: fetch through /api/fetch (raw source), mkdir -p, writeFile + read-back verification, error instead of a tick on failure. Register curl with -s/-o parsing delegating to the same writer so the two cannot drift. S2 (same reporter): web_fetch could not page and said it could. - Cause: the handler honoured max_chars up to 50 000, then _clipToolResult kept toolResultChars-200 = 11 800, so 38 200 chars vanished silently; offset was not in the schema and was never read, so next_offset pointed at a page the caller could not request again. - Fix: offset/next_offset/total_chars/source_bytes in the contract; the page is built, measured against the clip budget and shrunk until the envelope fits, so next_offset - returned_chars is always 0. - Also fixed: _clipToolResult appended "0 of N chars withheld" when the content already fitted, claiming a loss that did not happen. C1: fetch --save=FILE — the only terminal path from a remote URL to the VFS, so a 90 KB source page no longer has to be read through 3 000-char scrollback. Proxy: /api/extract now returns source_bytes (raw page size) and content_chars_utf16. Python len() counts code points while JS String.length counts UTF-16 units; 7 astral characters in the changelog left a 7-char gap at the end of a 90 KB page (measured). _html_ok declares charset=utf-8. Verified: node --check on all four modules; py_compile; bundles rebuilt; served bytes identical to disk for all five assets at ?v=v4.6.151; live in the browser: curl -s -o review/main.md saved 1954 chars with the correct head, a 404 wrote no file, fetch --save wrote 91 518 chars, web_fetch p1 returned 11 754 with next_offset 11 754 (gap 0), page 2 started at a different offset with next 23 496.

fix(newos4): read_file pages were silently truncated; delegation no longer invents "completed"

2026-11-02T20:21:24+00:00

Two defects found by re-deriving a report that claimed both were still open. read_file paging (module-aiassist.js) - A page larger than the result clip was clipped AFTER read_file had computed next_offset. On the last page next_offset is null, so the model was told truncated:false while its tail had been removed: measured 11 890 of 19 999 chars delivered, returned:19999, no pointer to the rest - 8 109 chars lost with no way to ask for them. * the clip now sets truncated=true and next_offset=offset+keep when it removes the tail of a page that had no continuation (and never splits a surrogate pair) * read_file caps a page at the result budget (min(pageCap, toolResultChars-200)), so a page is always delivered whole and next_offset is exactly contiguous Live v4.6.143: 19 999-char file -> page 11 800 (next_offset 11 800), page 2 8 199 (next_offset null): 11 800 + 8 199 = 19 999, nothing withheld. A model asking for limit:32000 is clamped to 11 800 and told truncated:true. The ~500-char symptom was the 500-char floor: a caller passing limit:200 gets 500, which is why targeted sed -n reads looked better than the tool. Delegation reporting (module-orchestrator.js, module-aiassist.js) - chat() returning {content} was already fixed in 8a9333b, but three other places still substituted the word "completed" for an absent answer, so the symptom survived: delegateTask's outbox wrote status:'completed' even when the worker returned nothing; the delegate_task tool returned {success:true, result:"completed"}; orchestrate-parallel summarised an unmatched agent as {status:'ok', result:'completed (agent recycled)'} and a missing outbox as "completed (no outbox)". - Now: an empty/undefined answer is status 'failed' with error 'agent produced no answer' / 'no result in outbox' / 'agent not found — result unverifiable', and the tool returns success:false with result:null. Tests: test-delegation-reporting.js 6 (RED 3/6 against HEAD: D3/D4 success:true, D5 the literal "completed"), test-tool-context-bound 26 with E6-E8 (RED 24/2: truncated:false, next_offset null). maintain 27/27.

fix(newos4): memory-graph loads stop rewriting user data; unknown sizes say unknown

2026-11-02T20:15:28+00:00

Three follow-ups from the merge review, each with the assertion that pins it. module-memory-graph.js - _normalizeEdges no longer mutates stored weights. It capped legacy 'failed' edges from 0.8 to 0.2 in memory, and the next debounced save made that permanent - an irreversible migration of the user's graph with no schema version. The cap moved to _effectiveWeight(), which ranking (showStats "Strongest relationships"), the session view and prune() now use; the stored record is left exactly as written. - config.json is stamped version: 2 so a future format change can migrate deliberately. ext/webdav.js - the two fallback parsers emit size: null instead of 0. They cannot see the size column, and 0 rendered as "0 B" - the same "every file looks empty" symptom as the parseInt bug fixed in e639c2d, one regex miss away. - _formatSize(null) renders '—' (unknown) rather than '0 B'; 0 still renders '0 B'. module-aiassist.js - the lock timeout now says what to do: "AI is still busy with another turn (waited Ns). Wait for it to finish, or run 'ai-stop' and retry - the running turn's lock is left alone." It used to be a dead-end string. Tests: test-memory-graph 39 (T6.4 rewritten to the read-time contract: stored weight stays 0.8, effective 0.2, success unaffected), test-webdav-sizes 28 (A14d null -> '—'). maintain 26/26, live v4.6.141: _effectiveWeight 0.2 with the edge object unchanged, _formatSize(null) '—', '0 B' still '0 B'.

perf(newos4): delegated agent runs no longer serialise; a waiter can't steal a lock

2026-11-02T19:55:56+00:00

Closes the second half of STEER-20261102-201045-ab08. - chat() gated the main-turn lock on nothing, so orchestrate-parallel's fan-out (each sub-task calling chat() on the same instance with its own agentTag) ran strictly one at a time. Measured 2x300ms delegated runs: 617ms before, 306ms now; the two 200ms user turns still take 400ms (main path stays serialised). - the 10s lock timeout called this._chatLock = false and printed "force released", which cleared a lock the waiter did not own - that released the main turn's lock while it was still running. A waiter now throws and the owner is the only writer. - delegated runs no longer clear the user's cancel flag on entry (an agent start used to reset this._cancelled, disarming a Stop the user had pressed). - tool-log entries carry the run they belong to and _recentToolLog(tag, n) is what feeds the run log line and _postRunReview, so concurrent runs no longer consume each other's entries. Known limit, unchanged: _currentTag is a single field, so under true concurrency an entry can be attributed to the wrong run (telemetry only - the answers are per-session). Namespacing it properly needs per-call state threading. Test: modules/test/test-chat-concurrency.js, 7 assertions, RED before (H1 617ms, H3 force-released, H5 no tags) and GREEN now; maintain 24/24.

fix(newos4): chat() threw on every tool-using turn and never returned its answer

2026-11-02T19:50:25+00:00

Regression from 32608bc, caught by the .chat() caller sweep before it aged: - `recent` was referenced in chat()'s finally block but never declared, so any turn with toolTurns > 0 threw ReferenceError there and chat() REJECTED. delegateTask's catch turned that into "agent failed"; the 'completed' fallback was only ever reachable on tool-free turns. - chat() still had no return at all, so a tool-free turn resolved undefined and module-orchestrator.js:1250 fell through to `|| 'completed'`. Both fixed: `const recent` is now built from _toolCallLog, and chat() returns { content: finalAnswer, toolTurns } before the archive block (a return at the end of the method would let _archiveSession swallow the answer). Verified with /tmp/probe-chat-hotfix.js, 6/6: tool-using turn resolves {content:'WORKER ANSWER', toolTurns:2}, tool-free turn resolves the answer, the orchestrator's own projection returns the answer instead of 'completed', and _chatLock is released. Run against 32608bc the same probe says "recent is not defined" - i.e. it reproduces the shipped defect. Not included: the _chatLock serialisation (orchestrate-parallel still runs sub-tasks one at a time and the 10s force-release can clear the main turn's lock). That stays open in STEER-20261102-201045-ab08.

feat(newos4): configurable tool budgets, per-model defaults, real tool telemetry

2026-11-02T19:46:18+00:00

Closes STEER-20261102-201234-7200 (read_file capped at 6000 while the schema promised 32000). Budgets (module-aiassist.js + new modules/core/model-profiles.js): - one resolver, precedence: call arg -> loop profile -> model profile -> ai-config -> code default. Keys: read-page-chars, read-result-chars, tool-context-budget, fetch-chars. - model classes tiny/small/mid/large keyed by id then by size token; `mid` reproduces the old numbers so nothing moves for a mid model. - _clipToolResult now clips the `content` field, never the serialised JSON envelope: clipping the envelope deleted truncated/next_offset, so a 16k page lost its continuation pointer (the reported symptom). - read_file's schema no longer promises a number the model class may not give. Telemetry: - _recordToolCall keeps sizes, duration, ok and a 120-char arg preview instead of the whole payload; ai-stats prints calls / ok% / avg ms / KB / truncation from localStorage webx11_tool_stats. Server (api/proxy-api.py): - /api/extract honours max_chars (default 12000, cap 100000) and returns content_chars + truncated; the bs4 branch is no longer uncapped while the regex fallback was silently cut at 10 000. BEHAVIOUR CHANGE: num_ctx for a model with no explicit setting now follows the class default (16384 -> 65536 for cloud/large, 8192 for tiny). Cloud models bill on tokens used, so cost is unchanged; local models get a bigger KV cache. Two build traps, both now documented in AGENTS.md: - model-profiles.js must be the FIRST entry in CORE_FILES and must export the MODEL_PROFILES global (a TDZ typeof throws; without the global every budget silently falls back to the code default). - a budget key needs the setter branch AND the constructor loader at module-aiassist.js:49 - persisting without restoring looks fine until reload. Verified: test-tool-context-bound 23/23 (RED before: E1-E4 + TypeError on _toolLimit); live bundle read_page 32000 / result 12000 / budget 48000 / fetch 12000 and _ctxTokens 65536 for gemma4:31b-cloud; /api/extract 28970 -> 500 chars with truncated true; ai-config write+restore probe 6/6.

fix(newos4): send num_ctx to Ollama, and make one auto-connect failure read as one line

2026-10-27T16:08:42+00:00

Two things I had flagged as "worth your attention" and one leftover nit. - num_ctx was never sent. Ollama's server-side default context is 2k-4k depending on the build, so a system prompt + 20 messages + up to 24k chars of tool results could be truncated server-side, which is exactly the "it forgot what it just read" class of report. Every Ollama request now carries options.num_ctx (default 16384, overridable via webx11_ollama_numctx, floor 512); the OpenAI-compatible branch is untouched because num_ctx is not one of its parameters. - Automatic connect prints one outcome instead of one line per endpoint tried (7 -> 3 on a dead endpoint). Interactive `ollamaconnect` keeps the per-endpoint detail. - write_file's failure message no longer echoes a trailing slash. Verification: - test-ollama-numctx.js (7 asserts): generate and chat payloads carry num_ctx=16384, a stored override is used, junk/too-small values fall back, and the /v1/chat/completions branch does not receive it. RED on HEAD: TypeError reading 'num_ctx' of undefined. Suite 20/20. - Live v4.6.117, capturing the real request: {"num_ctx":16384} on /api/generate with model gemma4:31b-cloud, reply "OK". - Live v4.6.118, dead endpoint (127.0.0.1:9, fallback disabled): automatic -> 3 lines ("connecting on first use…", "Connecting to dead…", "Auto-connect failed — …"); manual connect() -> 4 lines including "endpoint http://127.0.0.1:9 failed", so the suppression is conditional, not blanket.

fix(newos4): the loop guard reads the config it documents

2026-10-27T15:07:35+00:00

MAINT-004. The guard read warnAt/blockAt/hardStopAt while one config path wrote `threshold`, and `enabled` was never consulted anywhere — writing {"enabled": false} into webx11_ai_config changed nothing at all. One resolver, _loopPolicy(), now normalises both spellings, guarantees warnAt <= blockAt < hardStopAt, and both the block and the warning are gated on enabled. Live on v4.6.110 (_loopPolicy called in the page): defaults {warn 3, block 6, hardstop 9}; {threshold: 4} -> {warn 2, block 4, hardstop 7}; {enabled: false, threshold: 3} -> enabled false. test-loop-policy.js: 17 asserts, RED on HEAD ("_loopPolicy is not a function"), suite 16/16.

fix(newos4): agent registration is absolute-path and honest about what it stored

2026-10-27T14:33:33+00:00

Three of the four agent-cluster items, each with a test that is red on HEAD. - MAINT-008 (module-agent-registry.js + module-orchestrator.js): registerAgent took args[0] as the agent id and only args[1] as the capability list, while both orchestrator callers passed the command word first — so the boot log read "Agent 'register' registered with 1 capabilities" and four capabilities were stored as one. The command word is now stripped, args[1..] are joined and split on commas (both "a,b" and "a b c" work), and the callers pass [id, ...caps]. - MAINT-007 (registry + aiassist): mkdir/rm used relative paths, so a registration made from /docs created /docs/agents/ and an unregister from any other cwd removed nothing — the leaked directories the report suspected. Every path in the registry (and the four /_sessions writes in module-aiassist.js) is absolute now. - MAINT-009 (listAgents): a directory alone counted as a registered agent, so a hand-made /agents/foo appeared as "foo — unknown (0 capabilities)". Only a readable status.json with an id counts now; the rest are reported as unregistered directories. Verification: test-agent-registry.js (11 asserts). RED on HEAD: 9 of 11 fail, including the exact `✅ Agent 'register' registered with 1 capabilities` line and `hand-made — unknown (0 capabilities)`. Suite 15/15 files green; maintain --fix "node tests 15/15 pass". Live on v4.6.109: boot prints "✅ Agent 'orchestrator-0' registered with 4 capabilities"; `agent-list` warns "1 directory(ies) in /agents have no status.json (not registered): hand-made"; register from / with a comma list stores 3; unregister from /docs removes the directory and leaves no /docs/agents. MAINT-006 stays open with a narrower note: the hard stop was only reachable when every call in a batch was blocked, so a mixed batch (one repeated + one novel call) never ended. The same ceiling now applies on the mixed path, but that branch lives inside agenticLoop and is not covered by a unit test — it needs a live agent run that repeats a call.

fix(newos4): help names the owner of an undocumented command; loop lessons leave /skills

2026-10-27T14:23:38+00:00

Two board items plus the context-window close-out. - MAINT-019 (module-system.js): `help ` answered "No help available for X" for a command that is registered but has no help line, which reads as if the command does not exist. It now says "X — registered by , which publishes no help line for it. Try: help". A command that genuinely does not exist still gets the old message. - MAINT-020 (module-aiassist.js): auto-generated loop lessons were written into /skills with their own index entries, so `skill list` filled up with system noise. Lessons now live in /lessons with /lessons/index.json; the reader still falls back to legacy loop-lesson-* entries in /skills for one release, and the one-time migration deletes them from the skills index. Verification: test-help-and-lessons.js (10 asserts, includes a lint-style guard that no lesson is written into /skills). Suite 14/14 files green; maintain --fix "node tests 14/14 pass", "help coverage 362/362". Live on v4.6.108: `help agent-register` -> "registered by AgentRegistryModule, which publishes no help line for it"; `help a2a-prune` unchanged; `help nope-not-real` still "No help available". MAINT-001 close-out (no code change here): history is capped at 20 messages with a "[context trim]" marker (first trim at turn 11 in a 60-turn probe), token compression took 80 msgs/8080 tok to 17/2625, and since v4.6.107 one turn's tool bodies are clipped to 6 000 chars each and elided past 24 000 with the newest 4 kept verbatim. Residual, recorded on the board: maxContextTokens is never sent to the server as num_ctx, so the real window is whatever the endpoint defaults to. MAINT-018 close-out: not reproducible. Live probes on v4.6.107/108 — `sed -n '1,3p' /probe.txt` prints lines 1-3, `sed -n "2,4p"` prints 2-4, `grep 'line4'` matches, and the same commands inside `script /q.sh` print correctly. The "Command not found: for;" symptom belongs to the line-by-line script runner, which is MAINT-021 and stays open.