newos4/scripts, branch v4.6.168

fix(tooling): two checkers stopped manufacturing false audit signals

2026-11-14T17:04:08+00:00

Both defects are parser bugs in tooling, and both were producing a signal the operator sees on every run. Plan 2026-11-13_062052, implemented as written. 1. `help coverage` WARN was a quote-pairing artefact (every run, always 367/368). `newos4-maintain.py:238-239` captured `this._baseWhitelist = [...]` with a regex and then pulled `'([^']+)'` out of it. A JS comment inside that array literal contains the apostrophe in "don't" (terminal.js:16), which desynchronised pairing: it matched forward to the opening quote of 'ui' and the pseudo-command `t work"\n // …` joined the audited set — so a genuine undocumented command was indistinguishable from parser debris. Fix: strip `/* */` and `//` before extracting, and keep only tokens matching `^[a-z][a-z0-9_-]{0,40}$`. The same shape bug existed for `const internals = [...]` and is fixed by the same helper. The WARN list is now single-line (newlines collapsed) so one bad token cannot reflow the digest. Proven both ways: injected `'zzz-probe-cmd'` into the whitelist -> `WARN: help coverage 367/368 — undocumented: zzz-probe-cmd`; reverted (md5 verified) -> `PASS: help coverage 367/367`. 2. Auto-review "still present in shipped code" fired on generic tokens. `STEER-20261112-195037-1eed [high] cmds.indexOf is not a function` was approved because `identifiers()` kept `indexOf` and `function` (hits: 36 and 74 files) and R4 approved on `any()` present token. `isCommand` — the only name that identifies the fault — was never considered, because stack frames were not parsed at all. It does not exist anywhere in the tree. Fix: JS keywords/builtins/DOM handler names join the skip set; `stack_functions()` extracts frame names (dropping `onclick`, `anonymous`, `eval`); R4 now requires a **distinctive** token (>=6 chars, present in modules/, not generic); and a stack whose TOP frame is `` is classified as generated code (`js-run` / `new Function`), which can never justify "still present in shipped source". The bottom `HTMLButtonElement.onclick (…/main.html)` frame proves only that a click started it — deciding on "does any frame name a real file" got this wrong once already. Proven: `identifiers('cmds.indexOf is not a function')` -> `[]`; `distinctive(['indexOf','function'], {...})` -> `[]`; the real report replayed into a fixture reports `nothing provable (1 left for the operator)` with **no** decision row written; production dry-run `nothing provable (0 left)`. `decisions.jsonl` still carries exactly 1 row for that report (append-only history untouched). Regression cover: `python3 scripts/newos4-feedback-autoreview.py --self-test` -> 6/6. maintain: node tests 42/42, help coverage 367/367, result OK.

chore(newos4): commit accumulated maintenance output

2026-11-09T14:36:23+00:00

The daily collector and the feedback pipeline had left 57 modified files uncommitted (skills, modules-index, CHANGELOG, feedback decisions, debug counters). The public mirror publishes git HEAD, so without this the published repo stayed behind the tree - and it blocked the publisher outright once uncommitted subtrees started failing loudly.

feat(newos4): six interface modes (easy -> cockpit) over one terminal

2026-11-07T17:19:47+00:00

Symptom: newOS4 had exactly one layout. A first-time user had no way in that did not require learning commands, and an operator had no surface showing live runtime state; the only way to see heap/VFS/tool-call telemetry was to type cat /sys/... Root cause: no UI layer existed. main.html hard-codes one flex column, and terminal.js binds the stock nodes by id at construction, so an alternative layout had to either duplicate the output stream or reparent the real nodes. Fix: - modules/core/module-uishell.js (new): mode registry, #ui-root, stock-node snapshot/restore, identity check that reverts to classic when a mode removes or clones #terminal-container, shell-docked #ui-foot (the AI stop button exists in every mode), ui command, gated first-run picker, ?ui= honoured only if the mode actually registers, lazy mode loading via the same loader ext modules use. - modules/ext/ui-{easy,chat,studio,workspace,commander,cockpit}.js (new): six reparenting skins with scoped CSS. Workspace/commander/cockpit read real /sys/* mounts and captured command output; no value on screen is invented. - modules/test/test-ui-shell.js (new): 25 assertions - teardown, clone rejection, failed-mount fallback, footer docking, destructive-mode rejection. - terminal.js: register the shell; make one failing ext module stop aborting the rest of the serial load loop (it did, silently, which is why six ui-* files at the end of extKeys never loaded). - module-loader.js: LATEST_VER v4.6.152. The six ui-* keys are deliberately NOT in extKeys - see the rate-limit note below. - admin/scripts/probe-ui-modes.py (new): 6 modes x 2 viewports plus a --cycle pass that switches all seven states on ONE page and asserts the output node keeps its identity. Also fixed, found while testing (pre-existing, production-affecting): - nginx limit_req zone=webx11 burst 50 -> 200. One newOS4 boot fans out ~40 assets; the bucket emptied mid-boot and every later module got 503. Measured in webx11.error.log: "limiting requests, excess: 50.696 by zone webx11" against modules/ext/ui-chat.js. A reload within ~50 s silently broke modules. Sustained rate unchanged at 60r/m. Verified: node modules/test/test-ui-shell.js -> 25/25; removing both _restoreStock() calls measures 20/25 with five FAILs; probe-ui-modes.py --cycle against production -> ALL PASS, 13 checks; served bytes: v4.6.152 x3 in main.html, class UIShellModule x1 in core-bundle.js, six ui-* packages in packages.json.

fix(changelog): keep the newest commits of a build; wrap long detail URLs

2026-11-06T10:40:54+00:00

Symptom: after bodies were rendered, this fix's own commit was missing from its own entry, and a long URL inside an expanded detail was clipped at the right edge on a 390 px screen. Cause: items took group[:MAX_ITEMS] from a CHRONOLOGICAL list, so a 17-commit build showed its first 14 and dropped the newest three (the count line said "…and 3 more", hiding which three). And the injected detail CSS set no overflow-wrap, so a 47-character URL pushed the paragraph to 320 px of unbreakable text — the document stayed at 0 overflow only because the li clipped it. Fix: show group[-MAX_ITEMS:] and label the omitted ones "…and N earlier commit(s)"; entry.commits follows the same slice. overflow-wrap:anywhere on detail p/ul. The CSS block is now re-emitted from DETAIL_CSS behind BEGIN/END sentinels instead of being write-once, and the first version's sentinel-less block is stripped on sight so no page keeps two copies. Verified: 390 px — p scrollWidth == clientWidth == 320, document overflow 0, URL wraps; newest entry lists c759c50 and this commit; island 83 entries, 14 commit chips matching 14 shown items; CHANGELOG.md byte-identical on a second run.

fix(changelog): render the commit body, not just the subject

2026-11-06T10:37:11+00:00

Symptom: the mirrored CHANGELOG.md (and the page) showed one-line commit summaries while README promises "every shipped fix, symptom -> cause -> fix -> verified". A reviewer measured the gap from the public mirror and could only conclude the claim overpromised. Cause: scripts/build-changelog.py:59 formatted commits with %s only; %b was never read, so the verification prose every fix commit carries (5b55af9 is a typical one) was discarded before rendering. The MD writer then replaced every tag with a backtick, so nothing richer could have survived anyway. Fix: read %b with a %x02 terminator (line-based parser, multi-line body), split it into paragraph/bullet blocks capped at 900 chars with a visible marker, and render it twice from one source: as a native

per item on the page (its renderer already assigns item HTML, so no JS change; CSS injected once behind a sentinel) and as nested bullets under the line in CHANGELOG.md. The "every build" gap note and the page dek were rewritten to describe what the file now contains. Verified: page 100 details render, first opens 23 -> 232 px with 3 paragraphs, no horizontal overflow, island parses (83 entries); two consecutive runs are byte-identical for CHANGELOG.md; the double-escape bug this surfaced (code spans shipped as <sha>) is fixed and grep for < is 0.

fix(changelog): stop prepending the preamble on every run

2026-11-05T19:40:17+00:00

write_md() put the H1 + description inside the generated body and also kept old[:m.start()], so each regeneration added another copy: 44 runs left 44 H1s and pushed the first release to line 179. Only the region between the markers is generated now, an existing file with N copies is collapsed to one, and an assert refuses to write a file containing two generated regions (proven RED by injecting a duplicate). Regenerated: 83 entries, 185 commits, first heading v4.6.146 (was v4.6.135 - the file had not been committed since 2026-11-02).

feat(newos4): configurable tool budgets, per-model defaults, real tool telemetry

2026-11-02T19:46:18+00:00

Closes STEER-20261102-201234-7200 (read_file capped at 6000 while the schema promised 32000). Budgets (module-aiassist.js + new modules/core/model-profiles.js): - one resolver, precedence: call arg -> loop profile -> model profile -> ai-config -> code default. Keys: read-page-chars, read-result-chars, tool-context-budget, fetch-chars. - model classes tiny/small/mid/large keyed by id then by size token; `mid` reproduces the old numbers so nothing moves for a mid model. - _clipToolResult now clips the `content` field, never the serialised JSON envelope: clipping the envelope deleted truncated/next_offset, so a 16k page lost its continuation pointer (the reported symptom). - read_file's schema no longer promises a number the model class may not give. Telemetry: - _recordToolCall keeps sizes, duration, ok and a 120-char arg preview instead of the whole payload; ai-stats prints calls / ok% / avg ms / KB / truncation from localStorage webx11_tool_stats. Server (api/proxy-api.py): - /api/extract honours max_chars (default 12000, cap 100000) and returns content_chars + truncated; the bs4 branch is no longer uncapped while the regex fallback was silently cut at 10 000. BEHAVIOUR CHANGE: num_ctx for a model with no explicit setting now follows the class default (16384 -> 65536 for cloud/large, 8192 for tiny). Cloud models bill on tokens used, so cost is unchanged; local models get a bigger KV cache. Two build traps, both now documented in AGENTS.md: - model-profiles.js must be the FIRST entry in CORE_FILES and must export the MODEL_PROFILES global (a TDZ typeof throws; without the global every budget silently falls back to the code default). - a budget key needs the setter branch AND the constructor loader at module-aiassist.js:49 - persisting without restoring looks fine until reload. Verified: test-tool-context-bound 23/23 (RED before: E1-E4 + TypeError on _toolLimit); live bundle read_page 32000 / result 12000 / budget 48000 / fetch 12000 and _ctxTokens 65536 for gemma4:31b-cloud; /api/extract 28970 -> 500 chars with truncated true; ai-config write+restore probe 6/6.

feat(feedback): chat plane on the dashboard + honest triage ranking

2026-11-02T19:31:05+00:00

Tasks 3-8 of the chat-plane plan: - cron newos4-chat-ingest (0cc4bb4d2f82, */10, script-only, silent when idle) - collect.py: plan_key() normalises the two ledger spellings on read (5 rows carried a plan name without .md and were silently unmatched); per-item bulk_pass for the 2026-10-26 pass; counts.chat_total - feedback.html: 5th tile "from chat"; card shows who closed an item and when, with a bulk marker - triage: UNFILEABLE class -> top_real now ranks actionable errors only (first row went from an 18 203 fetch failure to the 989 toLowerCase runtime error); network/abort signatures move to policy_noise - AGENTS.md: the chat plane documented as an intake Verified: autoreview --dry-run over the 5 chat reports -> "nothing provable (5 left for the operator)" (R6 refuses auto:false); maintain 16 PASS 0 FAIL; feedback.html inline JS node --check OK; collect -> awaiting 5, chat_total 5.

feat(newos4): ingest report-like chat messages into the feedback queue

2026-11-02T19:26:53+00:00

The chat plane (api/chat/post.php -> data/chat/*.json) was read by nothing in the feedback pipeline: 22 report-like messages over 11 days produced 0 queue items. This adds scripts/newos4-chat-ingest.py - a read-only scanner by default, --apply writes RPT-* files in the report.php schema with diagnostics.auto=false so the auto-reviewer leaves them for a human, deduped on (thread, msg_id). First apply filed 5 items (2 x 2026-10-28, 3 x 2026-11-02); snapshot went reports_total 23 -> 28, awaiting_decision 0 -> 5. Second run is silent.

feat(newos4): R5 stale-artefact rule + dashboard queues a decision once

2026-11-02T19:05:14+00:00

Feedback queue triage 2026-11-02: 23 reports, 22 decided, 1 open -- and the open one (RPT-20261029-115241, "[Terminal] Error in ShellModule.test: {\"silent\":true}") was already fixed in v4.6.135 (4472c81, modules/core/terminal.js:1966) yet could never be cleared by automation. Root cause: identifiers() keeps only generic class names (Terminal, ShellModule), both of which necessarily stay in the code, so R2 (every identifier absent) cannot fire; R4 needs a stack and this report has none. A stack-less "Class.method: {json}" console.error whose emitter gets *guarded* -- rather than deleted -- therefore sits "awaiting decision" forever. - newos4-feedback-autoreview.py: new R5 stale artefact -- auto report, no stack, every error count <= 1, and nothing newer than STALE_DAYS=5 in data/debug/errors.json -> resolve with the measured quiet_days as evidence. New --errors knob so the rule can be run against fixtures. RED (pristine copy): "nothing provable (1 left for the operator)" GREEN (aged fixture): "resolve ... quiet 5.3d -- reopen if it recurs" NEG (live store 4.3d, and a report with count 21): both still refuse. - admin/feedback.html: markDecided() patches the local snapshot, so a queued decision leaves the default "Awaiting only" list immediately instead of after the 10-minute sync rebuild. Measured cause of 3x approve on one plan in 3 min (10-30 16:03/16:05/16:05) and 6 items with duplicate rows. - AGENTS.md: documents R5 and the fixture knob (also carries the already-live 2026-11-02 cookie-notice paragraph that was pending in the tree).