fix(newos4): honest size/network measurements + checkpoint and phantom-delete fixes (v4.6.160)

Applies the three 2026-11-09 patches after review, and fixes two more
navigator.onLine lies found while verifying them. Each patch was checked
against the live code first; one of its claims did not survive.

1. virtual-fs — phantom deletes and checkpoint bloat
   - deleteCheckpoint('nope') returned true and printed "Deleted checkpoint"
     for a name that did not exist: _resolvePath() validates intermediate
     segments only, so the leaf was never checked. New _requireLeaf() refuses
     a resolved path whose leaf is absent, and deleteCheckpoint uses it.
   - CHECKPOINT SIZE: checkpoint() cloned the whole root minus
     _snapshots/_sessions/.trash, so _mods (module originals) and
     .semantic_index were duplicated into EVERY checkpoint — measured
     2,101,686 bytes for a 2 MiB _mods. Both are now excluded on write and
     preserved across rollback (without the keep-list change a rollback would
     delete the live module originals). Measured after: 185 bytes.
   - Claim corrected: the patch also claimed deleteFile() reported success on
     a phantom path. It did not — live code already required
     `node && node.type === 'file'` and returned false. The guard is kept as
     defence-in-depth, and the test asserts the real behaviour rather than the
     claimed one.

2. module-fs — vfs-size reported ~2x the real size
   - It measured JSON.stringify(root).length: UTF-16 code units PLUS every
     structural character of the serialisation. A 1046-byte tree printed as
     1961; anything under ~5 KB printed as "0.00 MB".
   - Every number is now stored bytes (UTF-8 of file content): one measure()
     walk for the total, the per-file table, the top-directory table and the
     system dirs, so the rows add up to the total. The patch's arithmetic was
     still off in seven places, all fixed here: a UTF-16 fallback that
     reintroduced the bug when Blob was missing, a top-5 table that could
     never sum to the total (now top 10 + an aggregated remainder row), the
     top-dirs table dropping files stored at root (now a "/(root files)" row),
     a 2-decimal MB total that printed 0.00 for small VFS, three duplicate
     hand-rolled walks, readDir() used for dir sizes, and "1 files".

3. module-system — status/doctor no longer trust navigator.onLine
   - status printed "Network: Offline" from navigator.onLine, which an
     Android WebView reports false while fully connected.
   - doctor now awaits a real HEAD probe (AbortController, 4 s): only
     'reachable' is healthy, a throw is HIGH "Network: unreachable", a non-ok
     answer is MEDIUM and names the status (405 to HEAD is not "offline").
   - THE PATCH'S THRESHOLDS COULD NEVER FIRE: it compared localStorage bytes
     against navigator.storage.estimate().quota, which is the whole-origin
     quota (GBs, mostly Cache Storage), while localStorage has its own ~5 MB
     cap — a 4.5 MB localStorage reported "All systems healthy" against a
     2 GB quota. Now: localStorage measured as (key+value)*2 UTF-16 bytes with
     HIGH > 4 MiB / MEDIUM > 3 MiB, VFS measured in UTF-8 bytes with
     HIGH > 3 MiB and > 80% of the measured localStorage bytes. The origin
     quota appears only in one informational line and decides no severity.
   - Also removed navigator.onLine from version, bug-report and bug (a grep
     for it is now 0 in the module), and added a "Measured:" line so a clean
     report still shows what was measured.

4. terminal.js / module-servermon.js — the same lie, where the user looks
   - The status bar printed "❌ Offline" in red from navigator.onLine; on a
     WebView that is false while online. It now shows "📶 Network?" with a
     title pointing at `doctor` unless the browser claims online.
   - `net` test line now reads "Browser reports: offline (unreliable)" as a
     warning instead of a red failure — the real verdict comes from the
     probes that follow.

Verified: 40/40 node tests (3 new files: 36 + 53 + 34 assertions, each proven
red against the pre-patch code; the doctor test runs against the PATCH's
threshold logic and shows the 4.5 MB case staying silent). An independent
orchestrator probe re-derived all of it: phantom delete false, 2 MB _mods
absent from the checkpoint and preserved through rollback, vfs-size exact
bytes for ASCII/é/☃ nested trees, 4.5 MB localStorage HIGH despite a 2 GB
quota, failing probe unreachable, status never claiming Online/Offline.
Served bytes identical for core-bundle.js / main.html / module-system.js /
virtual-fs.js. Live Chromium on v4.6.160: vfs-size "85.8 KB (87880 bytes)
across 40 files", doctor "Measured: VFS 0.08 MB UTF-8 · localStorage 0.07 MB
UTF-16 in 14 keys", a stubbed failing probe reports HIGH unreachable, the
status bar reads "📶 Network?" when the browser lies, deleteCheckpoint on a
missing name returns false, and the console has 0 warnings and 0 JS errors.

1 files changed, 172 insertions, 30 deletions

diff --git a/modules/core/module-system.js b/modules/core/module-system.js
index 43bb0b1..ef37b11 100644
--- a/modules/core/module-system.js
+++ b/modules/core/module-system.js
@@ -35,6 +35,84 @@
                     'checkpoint-del': 'checkpoint-del <name> — Delete a VFS checkpoint',
                 };
             }
+
+            // ── doctor measurement helpers ──────────────────────────────────────
+            // UTF-8 byte length of a string. Blob is the reference implementation
+            // (Node 17+/every target browser), TextEncoder the browser fallback, and a
+            // code-point walk the last resort so a stripped-down WebView still gets a
+            // number instead of a thrown command.
+            _utf8Bytes(str) {
+                if (typeof str !== 'string' || str.length === 0) return 0;
+                try {
+                    if (typeof Blob === 'function') return new Blob([str]).size;
+                } catch (e) {}
+                try {
+                    if (typeof TextEncoder === 'function') return new TextEncoder().encode(str).length;
+                } catch (e) {}
+                let n = 0;
+                for (let i = 0; i < str.length; i++) {
+                    const cp = str.codePointAt(i);
+                    if (cp > 0xffff) i++;                 // surrogate pair consumed
+                    n += cp < 0x80 ? 1 : cp < 0x800 ? 2 : cp < 0x10000 ? 3 : 4;
+                }
+                return n;
+            }
+
+            // Sum of UTF-8 bytes of every file's content under `root`. This is the unit
+            // `vfs-size` reports, without JSON.stringify's braces/quoting overhead and
+            // without the old `chars * 2` fudge. Depth- and node-capped so a malformed
+            // or cyclic tree cannot hang the command.
+            _vfsContentBytes(root) {
+                let bytes = 0, nodes = 0;
+                const walk = (node, depth) => {
+                    if (!node || typeof node !== 'object' || depth > 64) return;
+                    if (node.type === 'file') {
+                        if (typeof node.content === 'string') bytes += this._utf8Bytes(node.content);
+                        return;
+                    }
+                    const kids = node.content;
+                    if (!kids || typeof kids !== 'object') return;
+                    for (const name of Object.keys(kids)) {
+                        if (++nodes > 20000) return;
+                        walk(kids[name], depth + 1);
+                    }
+                };
+                walk(root, 0);
+                return bytes;
+            }
+
+            // Live reachability probe. The legacy `onLine` boolean is not consulted anywhere:
+            // in Android WebView it reports false while the page is fully connected, which is
+            // exactly how `status` used to tell working users they were offline.
+            async _probeNetwork() {
+                try {
+                    if (typeof fetch !== 'function') return { state: 'unprobed', detail: 'fetch unavailable' };
+                } catch (e) {
+                    return { state: 'unprobed', detail: 'fetch unavailable' };
+                }
+                let ctl = null, timer = null;
+                try { if (typeof AbortController === 'function') ctl = new AbortController(); } catch (e) {}
+                try {
+                    if (ctl) timer = setTimeout(() => { try { ctl.abort(); } catch (e) {} }, 4000);
+                    const resp = await fetch('./', {
+                        method: 'HEAD',
+                        cache: 'no-store',
+                        signal: ctl ? ctl.signal : undefined,
+                    });
+                    if (!resp) return { state: 'unreachable', detail: 'empty response' };
+                    if (resp.ok) return { state: 'reachable', detail: 'HTTP ' + resp.status };
+                    return { state: 'http-' + resp.status, detail: 'HTTP ' + resp.status };
+                } catch (e) {
+                    const aborted = e && (e.name === 'AbortError' || e.name === 'TimeoutError');
+                    return {
+                        state: 'unreachable',
+                        detail: aborted ? 'timed out after 4s' : ((e && e.message) || 'request failed'),
+                    };
+                } finally {
+                    if (timer) clearTimeout(timer);
+                }
+            }
+
             registerCommands() {
                 this.commands['help'] = (args) => this.handleHelp(args);
                 this.commands['echo'] = (args) => this.terminal.print(args.join(' '));
@@ -83,7 +161,10 @@
                 }, 500);
                 this.commands['status'] = () => {
                     this.terminal.print(`System: WebX11 ${window.NEWOS4_VERSION || 'v4.0'}`);
-                    this.terminal.print(`Network: ${navigator.onLine ? 'Online' : 'Offline'}`);
+                    // The legacy `onLine` boolean is NOT evidence of reachability: Android
+                    // WebView reports false on a fully connected page, and true on a captive
+                    // portal. Say so, and point at the command that actually probes.
+                    this.terminal.print('Network: unknown — run doctor for a live probe');
                     this.terminal.print(`Files: ${this.terminal.fs.ls().length} items`);
                     this.terminal.print(`Modules: ${Object.keys(this.terminal.moduleManager.modules).length} loaded`);
                     this.terminal.print(`Ollama: ${this.terminal.ollamaManager.connected ? 'Connected' : 'Disconnected'}`);
@@ -110,7 +191,7 @@
                     this.terminal.print(`  Build:      ${buildDate}`);
                     this.terminal.print(`  Cache key:  ${cacheKey}`);
                     this.terminal.print(`  Browser:    ${ua}`);
-                    this.terminal.print(`  Online:     ${navigator.onLine ? 'yes' : 'no'}`);
+                    this.terminal.print(`  Network:    unknown — run doctor for a live probe`);
                 };
 
                 // v4.5.43: uptime — time since page load + last save time
@@ -166,40 +247,102 @@
                 };
 
                 // v4.5.43: doctor — run all health checks, rank issues, show top 3
-                this.commands['doctor'] = () => {
+                // v4.6.x: the legacy `onLine` boolean is GONE from this file. It reports false
+                // on a fully connected Android WebView, which is how `status`/`doctor` used to
+                // announce an outage on a working page. Reachability now comes from a real HEAD
+                // probe, and every size is measured in the unit the store actually charges for.
+                this.commands['doctor'] = async () => {
                     const issues = [];
                     const fs = this.terminal.fs;
-
-                    // Check 1: Network
-                    if (!navigator.onLine) issues.push({ severity: 'high', msg: 'Network: offline' });
+                    const MB = 1024 * 1024;
+
+                    // Check 1: Network — live probe, 4s budget
+                    const net = await this._probeNetwork();
+                    if (net.state === 'unreachable') {
+                        issues.push({ severity: 'high', msg: `Network: unreachable (HEAD ./ — ${net.detail})` });
+                    } else if (net.state.indexOf('http-') === 0) {
+                        // A non-ok HEAD is not proof the page is broken (some servers answer
+                        // 405 to HEAD); report the status and let the user judge.
+                        issues.push({ severity: 'medium', msg: `Network: HTTP probe to ./ returned ${net.state} (page may still work)` });
+                    }
 
                     // Check 2: Ollama
                     if (!this.terminal.ollamaManager.connected) {
                         issues.push({ severity: 'medium', msg: 'Ollama: not connected (AI features limited)' });
                     }
 
-                    // Check 3: VFS size
+                    // Origin quota — context only. It is the whole-origin quota (Cache Storage
+                    // included, typically hundreds of MB to GBs). localStorage has its own
+                    // ~5 MB per-origin cap, so this number must NEVER set a severity: a 4.5 MB
+                    // localStorage is 96% of its cap and 0.2% of a 2 GB origin quota.
+                    let originUsage = 0, originQuota = 0;
+                    try {
+                        if (typeof navigator !== 'undefined' && navigator.storage &&
+                            typeof navigator.storage.estimate === 'function') {
+                            const est = await navigator.storage.estimate();
+                            if (est && typeof est.quota === 'number' && est.quota > 0) {
+                                originQuota = est.quota;
+                                originUsage = typeof est.usage === 'number' ? est.usage : 0;
+                            }
+                        }
+                    } catch (e) {}
+
+                    // Check 3: VFS — UTF-8 bytes of file content (the unit vfs-size reports),
+                    // never JSON.stringify(root).length and never "chars × 2".
+                    let vfsBytes = null;
+                    try { vfsBytes = this._vfsContentBytes(fs.root); } catch (e) { vfsBytes = null; }
+
+                    // Check 4: localStorage — JS strings are UTF-16, so each key and value is
+                    // charged 2 bytes per code unit. Σ (key.length + value.length) × 2 is the
+                    // real storage cost, not a fudge factor.
+                    let lsBytes = 0, lsKeys = 0;
                     try {
-                        const vfsBytes = JSON.stringify(fs.root).length;
-                        if (vfsBytes > 524288) issues.push({ severity: 'high', msg: `VFS: ${(vfsBytes/1024).toFixed(0)}KB > 512KB limit` });
-                        else if (vfsBytes > 262144) issues.push({ severity: 'medium', msg: `VFS: ${(vfsBytes/1024).toFixed(0)}KB > 256KB (consider reset-vfs)` });
+                        for (let i = 0; i < localStorage.length; i++) {
+                            const k = localStorage.key(i);
+                            if (!k) continue;
+                            lsKeys++;
+                            lsBytes += (k.length + (localStorage.getItem(k) || '').length) * 2;
+                        }
                     } catch (e) {}
 
-                    // Check 4: Heap
+                    // VFS thresholds: a VFS only threatens the browser budget when it is BOTH
+                    // large on its own and the bulk of what is actually stored.
+                    if (vfsBytes !== null) {
+                        const vfsMB = vfsBytes / MB;
+                        const share = lsBytes > 0 ? vfsBytes / lsBytes : 0;
+                        const budget = lsBytes > 0
+                            ? `, ${(share * 100).toFixed(0)}% of the ${(lsBytes / MB).toFixed(1)} MB localStorage budget`
+                            : ', no localStorage copy measured';
+                        if (vfsBytes > 3 * MB && (lsBytes === 0 || share > 0.8)) {
+                            issues.push({ severity: 'high', msg: `VFS: ${vfsMB.toFixed(1)} MB of file content${budget} — close to the ~5 MB per-origin cap` });
+                        } else if (vfsBytes > 2 * MB) {
+                            issues.push({ severity: 'medium', msg: `VFS: ${vfsMB.toFixed(1)} MB of file content${budget} (consider deleting stale checkpoints)` });
+                        }
+                    }
+
+                    // localStorage thresholds: the cap is ~5 MB per origin, whatever
+                    // navigator.storage.estimate() says the whole origin may use.
+                    if (lsBytes > 4 * MB) {
+                        issues.push({ severity: 'high', msg: `localStorage: ${(lsBytes / MB).toFixed(1)} MB of UTF-16 data in ${lsKeys} keys — approaching the ~5 MB per-origin cap` });
+                    } else if (lsBytes > 3 * MB) {
+                        issues.push({ severity: 'medium', msg: `localStorage: ${(lsBytes / MB).toFixed(1)} MB of UTF-16 data in ${lsKeys} keys — past half of the ~5 MB per-origin cap` });
+                    }
+
+                    // Check 5: Heap
                     if (performance.memory) {
                         const heapMB = performance.memory.usedJSHeapSize / 1048576;
                         if (heapMB > 200) issues.push({ severity: 'high', msg: `Heap: ${heapMB.toFixed(0)}MB > 200MB` });
                         else if (heapMB > 100) issues.push({ severity: 'medium', msg: `Heap: ${heapMB.toFixed(0)}MB > 100MB` });
                     }
 
-                    // Check 5: Logs
+                    // Check 6: Logs
                     try {
                         const logContent = fs.readFile('/logs/current.log') || '';
                         const logLines = logContent.split('\n').filter(l => l).length;
                         if (logLines > 500) issues.push({ severity: 'low', msg: `Logs: ${logLines} lines (consider logrotate)` });
                     } catch (e) {}
 
-                    // Check 6: Errors in last 100 log lines
+                    // Check 7: Errors in last 100 log lines
                     try {
                         const logContent = fs.readFile('/logs/current.log') || '';
                         const recent = logContent.split('\n').filter(l => l).slice(-100);
@@ -208,16 +351,6 @@
                         else if (errorCount > 0) issues.push({ severity: 'medium', msg: `Errors: ${errorCount} errors in last 100 log lines` });
                     } catch (e) {}
 
-                    // Check 7: localStorage quota
-                    let lsBytes = 0;
-                    try {
-                        for (let i = 0; i < localStorage.length; i++) {
-                            const k = localStorage.key(i);
-                            if (k) lsBytes += (k.length + (localStorage.getItem(k) || '').length) * 2;
-                        }
-                    } catch (e) {}
-                    if (lsBytes > 4 * 1024 * 1024) issues.push({ severity: 'high', msg: `localStorage: ${(lsBytes/1024/1024).toFixed(1)}MB approaching 5MB quota` });
-
                     // Check 8: Disabled modules
                     let disabled = [];
                     try { disabled = JSON.parse(localStorage.getItem('webx11_module_disabled') || '[]'); } catch (e) {}
@@ -226,6 +359,11 @@
                     }
 
                     this.terminal.print('🩺 Doctor Report', 'header');
+                    this.terminal.print(`  Measured: VFS ${vfsBytes === null ? 'unavailable' : (vfsBytes / MB).toFixed(2) + ' MB UTF-8'} · localStorage ${(lsBytes / MB).toFixed(2)} MB UTF-16 in ${lsKeys} keys`, 'info');
+                    this.terminal.print(`  Network:  ${net.state === 'reachable' ? 'reachable (' + net.detail + ')' : net.state + (net.detail ? ' — ' + net.detail : '')}`, 'info');
+                    if (originQuota > 0) {
+                        this.terminal.print(`  Origin usage ${(originUsage / MB).toFixed(1)} MB of ~${(originQuota / MB).toFixed(0)} MB quota (whole origin, mostly Cache Storage — localStorage has its own ~5 MB cap)`, 'info');
+                    }
                     if (issues.length === 0) {
                         this.terminal.print('  ✅ All systems healthy', 'success');
                         return;
@@ -234,20 +372,21 @@
                     const sevOrder = { high: 0, medium: 1, low: 2 };
                     issues.sort((a, b) => sevOrder[a.severity] - sevOrder[b.severity]);
                     this.terminal.print(`  Found ${issues.length} issue(s):`, 'warning');
-                    for (const i of issues.slice(0, 5)) {
+                    for (const i of issues.slice(0, 3)) {
                         const icon = i.severity === 'high' ? '🔴' : i.severity === 'medium' ? '🟡' : '🟢';
                         this.terminal.print(`    ${icon} [${i.severity.toUpperCase()}] ${i.msg}`, i.severity === 'high' ? 'error' : 'warning');
                     }
-                    if (issues.length > 5) this.terminal.print(`    ... and ${issues.length - 5} more`, 'info');
+                    if (issues.length > 3) this.terminal.print(`    ... and ${issues.length - 3} more`, 'info');
                     this.terminal.print('', '');
                     this.terminal.print('  Recommended actions:', 'info');
-                    if (issues.some(i => i.msg.startsWith('Network'))) this.terminal.print('    → Check internet connection');
+                    if (issues.some(i => i.msg.startsWith('Network: unreachable'))) this.terminal.print('    → Check connectivity; reload before trusting this probe');
+                    if (issues.some(i => i.msg.startsWith('Network: HTTP probe'))) this.terminal.print('    → Server answered the probe with an error status — check the server, not the network');
                     if (issues.some(i => i.msg.startsWith('Ollama'))) this.terminal.print('    → Check Ollama is running (ollama serve)');
-                    if (issues.some(i => i.msg.startsWith('VFS'))) this.terminal.print('    → Run reset-vfs --confirm to clear or checkpoint/rollback');
+                    if (issues.some(i => i.msg.startsWith('VFS'))) this.terminal.print('    → Delete stale snapshots (checkpoint-list, checkpoint-del <name>) or rollback; do NOT reset-vfs');
                     if (issues.some(i => i.msg.startsWith('Heap'))) this.terminal.print('    → Refresh the page to clear memory');
                     if (issues.some(i => i.msg.startsWith('Logs'))) this.terminal.print('    → Run logrotate');
                     if (issues.some(i => i.msg.startsWith('Errors'))) this.terminal.print('    → Run errors to see details, or bug-report to save state');
-                    if (issues.some(i => i.msg.startsWith('localStorage'))) this.terminal.print('    → Run factory-reset --confirm (last resort)');
+                    if (issues.some(i => i.msg.startsWith('localStorage'))) this.terminal.print('    → Free space by deleting stale VFS entries or checkpoints; do NOT factory-reset');
                 };
 
                 // v4.5.43: bug-report — generate markdown report to /tmp/
@@ -261,7 +400,7 @@
                     lines.push('## System');
                     lines.push(`- Version: ${window.NEWOS4_VERSION || 'unknown'}`);
                     lines.push(`- Cache key: ${localStorage.getItem('webx11_cache_ver') || 'unknown'}`);
-                    lines.push(`- Online: ${navigator.onLine ? 'yes' : 'no'}`);
+                    lines.push(`- Online: unknown (not probed; run doctor)`);
                     lines.push(`- Browser: ${navigator.userAgent}`);
                     lines.push(`- Screen: ${screen.width}x${screen.height}`);
                     lines.push(`- Heap: ${performance.memory ? (performance.memory.usedJSHeapSize / 1048576).toFixed(1) + 'MB' : 'N/A'}`);
@@ -325,7 +464,10 @@
                         version: window.NEWOS4_VERSION || 'unknown',
                         url: location.href,
                         ua: navigator.userAgent,
-                        online: navigator.onLine,
+                        // Legacy key, kept so stored reports keep their shape. The old onLine
+                        // boolean lied in Android WebView, so null = "not asserted".
+                        online: null,
+                        network: 'unknown',
                         modules: Object.keys(this.terminal.moduleManager.modules).length,
                         errors: [],
                     };