diff options
| author | Agando <roduccomm@gmail.com> | 2026-10-23 17:38:10 +0200 |
|---|---|---|
| committer | Agando <roduccomm@gmail.com> | 2026-10-23 17:38:10 +0200 |
| commit | 7f4732e5089b1430e736bc6a8b8578a674f31f36 (patch) | |
| tree | c51e8da70e8468d234a60510d678fcfb4a797eab /modules/core/module-simplegui.js | |
| parent | 0c01cf48e83eecca953d0c6cc2dc949aed552f2f (diff) | |
fix(newos4): measure predictor accuracy over paired predictions (v4.6.56)
The accuracy readout was structurally 0%: module-predictor.js:187 incremented the
denominator on every observed run while the numerator only moved when a stored
prediction existed, and predictions were stored only by the manual `predict`
command (the AI tool predict_task computed one and discarded it). Replaying the
AI loop's own call path with 6 tasks printed "Predictions made: 0, Accuracy: 0/6".
- pending predictions with a 15m TTL, consumed by threaded id -> task hash ->
cli-only similarity >= 0.5; unpaired runs never enter the denominator
- duration scored within 30% or 3s (integer predictions vs fractional observations)
- errorPct capped at 500% so one miss cannot poison the mean error
- schema v2: v1 correct/total counters dropped on load, observed runs and the
prediction counter kept; task text stored as an FNV-1a hash only
- tuning (confidence-threshold, min-data-points, pair-ttl) persists
- min-data-points now scales confidence (was declared, printed, settable, unread)
- AI loop registers a prediction per turn and threads its id; predict_task
returns predictionId; durations exclude chat-lock queue wait
- status/dashboard say "n/a" instead of "N/A%", and carry paired count, mean
error, turns error, pending and unpaired counts
- test: modules/test/test-predictor-accuracy.js (26 assertions)
Live on main.html v4.6.56: fresh session -> "Accuracy: n/a (0/0 paired)";
one paired run -> "100% (1/1 paired) - mean error 5% - turns +-0.0".
Diffstat (limited to 'modules/core/module-simplegui.js')
0 files changed, 0 insertions, 0 deletions