From 895d8f5ed57c76f25770f88f290e6ab93949b6f8 Mon Sep 17 00:00:00 2001 From: Agando Date: Fri, 23 Oct 2026 17:38:10 +0200 Subject: fix(newos4): measure predictor accuracy over paired predictions (v4.6.56) The accuracy readout was structurally 0%: module-predictor.js:187 incremented the denominator on every observed run while the numerator only moved when a stored prediction existed, and predictions were stored only by the manual `predict` command (the AI tool predict_task computed one and discarded it). Replaying the AI loop's own call path with 6 tasks printed "Predictions made: 0, Accuracy: 0/6". - pending predictions with a 15m TTL, consumed by threaded id -> task hash -> cli-only similarity >= 0.5; unpaired runs never enter the denominator - duration scored within 30% or 3s (integer predictions vs fractional observations) - errorPct capped at 500% so one miss cannot poison the mean error - schema v2: v1 correct/total counters dropped on load, observed runs and the prediction counter kept; task text stored as an FNV-1a hash only - tuning (confidence-threshold, min-data-points, pair-ttl) persists - min-data-points now scales confidence (was declared, printed, settable, unread) - AI loop registers a prediction per turn and threads its id; predict_task returns predictionId; durations exclude chat-lock queue wait - status/dashboard say "n/a" instead of "N/A%", and carry paired count, mean error, turns error, pending and unpaired counts - test: modules/test/test-predictor-accuracy.js (26 assertions) Live on main.html v4.6.56: fresh session -> "Accuracy: n/a (0/0 paired)"; one paired run -> "100% (1/1 paired) - mean error 5% - turns +-0.0". --- modules/core/module-loader.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'modules/core/module-loader.js') diff --git a/modules/core/module-loader.js b/modules/core/module-loader.js index ff908cd..8b4f606 100644 --- a/modules/core/module-loader.js +++ b/modules/core/module-loader.js @@ -4,7 +4,7 @@ // Core module bundle loaded synchronously in main.html. // This loader only handles version cache-busting, splash, terminal init, and optional ext modules. - const LATEST_VER = 'v4.6.55'; + const LATEST_VER = 'v4.6.56'; const CACHE = '?v=' + LATEST_VER; const stored = localStorage.getItem('webx11_cache_ver'); if (stored !== LATEST_VER) { -- cgit v1.2.3