diff options
| author | Agando <roduccomm@gmail.com> | 2026-10-23 19:29:31 +0200 |
|---|---|---|
| committer | Agando <roduccomm@gmail.com> | 2026-10-23 19:29:31 +0200 |
| commit | 313f13955ad9c72895b0914ac828c61b3e469715 (patch) | |
| tree | 8d46bba5452cfe539348a2f1b3fd963b61b45ec4 | |
| parent | 1ff8df65ffefbd2eaa9100374fbcfb7828fdb717 (diff) | |
feat(chat): token-scoped chat endpoints, contract, host CLI, old POST retired (v4.6.71 phase B)
- api/chat/{lib,post,poll}.php: thread = sha256(token)[0:16]; flock read-modify-write;
tmp+rename; ms epoch cursor; per-thread + per-IP windows with retry_after_ms;
401/413/507/405 carry a machine code, only rate_limited is retryable
- _docs/chat-contract.md: frozen shapes, incl. why the cursor is ms and not ts
- scripts/chat.py: list/show/reply, UTC, locked, atomic replace
- api/debug/agent-replies.php POST -> 410 Gone with pointers (one write path)
Measured: 401 no-token, 201+id, poll 1 msg, poll>since 1 new, data/chat 403,
4001 chars 413 (codepoints), 429 with Retry-After 576, round trip via chat.py.
| -rw-r--r-- | _docs/chat-contract.md | 90 | ||||
| -rw-r--r-- | api/chat/lib.php | 157 | ||||
| -rw-r--r-- | api/chat/poll.php | 27 | ||||
| -rw-r--r-- | api/chat/post.php | 57 | ||||
| -rw-r--r-- | api/debug/agent-replies.php | 40 | ||||
| -rwxr-xr-x | scripts/chat.py | 125 |
6 files changed, 470 insertions, 26 deletions
diff --git a/_docs/chat-contract.md b/_docs/chat-contract.md new file mode 100644 index 0000000..ee2d006 --- /dev/null +++ b/_docs/chat-contract.md @@ -0,0 +1,90 @@ +# newOS4 ⇄ Hermes chat contract + +**Status:** frozen 2026-10-23. Any change here changes both sides. +**Scope:** one conversation per browser token. No accounts, no cookies, no admin session. + +## Identity + +| Item | Value | +|---|---| +| Token | 64 lowercase hex, generated by the browser (`crypto.getRandomValues`, 32 bytes) | +| Header | `X-Chat-Token: <64 hex>` | +| Thread | `substr(sha256(token), 0, 16)` — computed server-side, **never** sent by the client | +| Storage | `spaces/newos4/data/chat/<thread>.json`, one file per thread, atomic replace (tmp + rename) | +| Reachability | `data/chat/` is **not** web-reachable. No listing endpoint exists. | + +## POST `/spaces/newos4/api/chat/post.php` + +Request: + +```text +X-Chat-Token: <64 hex> +Content-Type: application/json + +{"text": "message body", "kind": "operator"|"agent"} +``` + +`kind` defaults to `operator`. + +Response `201`: + +```json +{"success": true, "id": "MSG-20261023-184000-a1b2c3", "epoch": 1792778400123, "ts": "2026-10-23T18:40:00+00:00", "thread": "a1b2c3d4e5f60718"} +``` + +## GET `/spaces/newos4/api/chat/poll.php?since=<epoch>` + +`since` is **epoch milliseconds**, an integer. Omitted or `0` returns the full thread (up to 50 newest). Seconds were not enough: two messages in the same second made a strict `>` drop the second one. + +Response `200`: + +```json +{"success": true, "thread": "a1b2c3d4e5f60718", + "messages": [{"id": "MSG-…", "epoch": 1792778400123, "ts": "2026-10-23T18:40:00+00:00", "author": "operator", "text": "…"}], + "awaiting": true, + "cursor": 1792778400123} +``` + +- `awaiting` = the last message in the thread was **not** written by `hermes`. There is no unread + counter: the server keeps no read state, so a counter could never clear. +- `cursor` = epoch of the newest message in the thread. Clients store it and pass it as `since`. + +**The cursor is epoch, never `ts`.** `ts` carries an offset (`+00:00`), and comparing strings with +mixed offsets silently drops messages in the same second and reorders others. + +## Authors + +| `author` | Written by | +|---|---| +| `operator` | the human in the app (`chat` / `feedback` command) | +| `agent` | the in-page AI (`chat_hermes` tool) | +| `hermes` | Hermes, host-side (`scripts/chat.py reply`) | + +## Errors — no silent success + +| Code | When | Body | +|---|---|---| +| `401` | token missing or not 64 hex | `{"success": false, "error": "missing or malformed X-Chat-Token"}` | +| `413` | body > 32 KB, or text > 4000 chars | `{"success": false, "error": "text too long (max 4000 chars)"}` | +| `429` | 20 messages per thread / 40 per IP per 10 min | `{"success": false, "error": "rate limit…"}` | +| `507` | more than 500 threads on the host | `{"success": false, "error": "thread limit reached"}` | +| `405` | wrong method | `{"success": false, "error": "Method not allowed"}` | + +Rules that follow from this table: + +- **Auth is checked before anything touches disk.** A `401` means no file was opened. +- **Text is rejected, never truncated.** A shortened question gets a wrongly scoped answer. +- Rate limits are two windows, per thread **and** per IP: a token is mintable in a loop, an IP is not. + +## Retention + +Newest 200 messages per thread, 30 days. Older messages are dropped on append and swept by the +relay cron. A conversation is not an archive. + +## What this replaces + +`feedback <msg>` and `chat <msg>` are the **same code path**; `feedback` is a deprecated alias. +Neither writes to the bug-report intake any more. `chat-poll` = `feedback-inbox`. + +`POST /api/debug/agent-replies.php` is retired in favour of this contract: after the acceptance +round trip it answers `410 Gone` with a pointer to `/api/chat/post.php`. One write path, one truth. diff --git a/api/chat/lib.php b/api/chat/lib.php new file mode 100644 index 0000000..6a0f923 --- /dev/null +++ b/api/chat/lib.php @@ -0,0 +1,157 @@ +<?php +// api/chat/lib.php — conversation store for the newOS4 <-> Hermes channel. +// One file per thread: data/chat/<thread>.json, written atomically (tmp + rename). +declare(strict_types=1); + +require_once __DIR__ . '/../debug/config.php'; // error_response() / json_response() / load_json() + +define('CHAT_DIR', __DIR__ . '/../../data/chat'); +define('CHAT_MAX_MESSAGES', 200); +define('CHAT_MAX_TEXT', 4000); +define('CHAT_RATE_MAX', 20); // messages per thread … +define('CHAT_RATE_WINDOW', 600); // … per 10 minutes +define('CHAT_IP_MAX', 40); // a token is mintable, an IP is not +define('CHAT_MAX_THREADS', 500); // hard ceiling on disk growth + +if (!is_dir(CHAT_DIR)) { + @mkdir(CHAT_DIR, 0775, true); + @chgrp(CHAT_DIR, 'www-data'); +} + +// Machine-readable failure: the sender must be able to tell "try again" from +// "your text is wrong" without parsing prose. 401/413/507 are permanent for that +// payload; 429 is the only one worth retrying. +function chat_error(string $msg, int $status, string $code, array $extra = []): void { + if (!headers_sent()) { + http_response_code($status); + header('Content-Type: application/json'); + } + $body = array_merge(['success' => false, 'code' => $code, 'error' => $msg], $extra); + echo json_encode($body, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES); + exit; +} + +function chat_token(): string { + $t = $_SERVER['HTTP_X_CHAT_TOKEN'] ?? ''; + if (!preg_match('/^[a-f0-9]{64}$/', $t)) { + chat_error('missing or malformed X-Chat-Token', 401, 'bad_token'); + } + return $t; +} + +function chat_thread(string $token): string { + return substr(hash('sha256', $token), 0, 16); +} + +function chat_path(string $thread): string { + if (!preg_match('/^[a-f0-9]{16}$/', $thread)) { + error_response('invalid thread', 400); + } + return CHAT_DIR . '/' . $thread . '.json'; +} + +function chat_read(string $thread): array { + $all = load_json(chat_path($thread)); + return is_array($all) ? $all : []; +} + +// Atomic replace: a reader sees the old file or the new one, never a truncated one. +function chat_write(string $thread, array $messages): bool { + $path = chat_path($thread); + $tmp = $path . '.tmp-' . bin2hex(random_bytes(4)); + $json = json_encode(array_values($messages), JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES); + if ($json === false) return false; + $fp = fopen($tmp, 'wb'); + if (!$fp) return false; + $ok = (fwrite($fp, $json) === strlen($json)); + fflush($fp); + fclose($fp); + if (!$ok) { @unlink($tmp); return false; } + @chmod($tmp, 0664); + return @rename($tmp, $path); +} + +// Append under a real lock: read-modify-write on one handle so two concurrent +// POSTs cannot each write back a stale copy (lost update). +function chat_append(string $thread, string $author, string $text): array { + $lock = fopen(CHAT_DIR . '/.lock-' . $thread, 'c'); + // FPM creates as www-data 0644; the host-side CLI (darkbot, group www-data) must be + // able to open the same lock, so group gets write too. + if ($lock) @chmod(CHAT_DIR . '/.lock-' . $thread, 0664); + if (!$lock || !flock($lock, LOCK_EX)) { + if ($lock) fclose($lock); + error_response('failed to lock thread', 500); + } + $msg = []; + try { + $messages = chat_read($thread); + $msg = [ + 'id' => 'MSG-' . gmdate('Ymd-His') . '-' . bin2hex(random_bytes(3)), + 'ts' => gmdate('c'), // UTC, always — never a local offset + 'epoch' => (int)round(microtime(true) * 1000), // ms — second precision drops same-second messages + 'author' => $author, + 'text' => $text, + ]; + $messages[] = $msg; + $cutoff = (time() - 30 * 86400) * 1000; + $messages = array_values(array_filter( + $messages, + fn($m) => (int)($m['epoch'] ?? (strtotime((string)($m['ts'] ?? '')) * 1000)) >= $cutoff + )); + if (count($messages) > CHAT_MAX_MESSAGES) { + $messages = array_slice($messages, -CHAT_MAX_MESSAGES); + } + if (!chat_write($thread, $messages)) { + error_response('failed to persist message', 500); + } + } finally { + flock($lock, LOCK_UN); + fclose($lock); + } + return $msg; +} + +// Two windows: per thread (the conversation) and per IP (a token is mintable, so the +// thread window alone is bypassable by rotating tokens). Returns the reason and the +// wait, so the caller can send Retry-After instead of an unparseable sentence. +function chat_rate_check(string $thread): array { + $ip = $_SERVER['REMOTE_ADDR'] ?? '0.0.0.0'; + $t = chat_window_check('t-' . $thread, CHAT_RATE_MAX); + if (!$t['ok']) return ['ok' => false, 'scope' => 'thread'] + $t; + $i = chat_window_check('i-' . hash('sha256', $ip), CHAT_IP_MAX); + if (!$i['ok']) return ['ok' => false, 'scope' => 'ip'] + $i; + return ['ok' => true, 'scope' => '', 'retry_after_ms' => 0]; +} + +// Sliding window. The wait is the time until the oldest hit leaves the window, +// which is the shortest time after which a retry can succeed. +function chat_window_check(string $key, int $max): array { + $file = CHAT_DIR . '/.rate-' . $key . '.json'; + $fp = fopen($file, 'c+'); + if (!$fp) return ['ok' => true, 'retry_after_ms' => 0]; // never drop a message over a stats file + if (!flock($fp, LOCK_EX)) { fclose($fp); return ['ok' => true, 'retry_after_ms' => 0]; } + $raw = stream_get_contents($fp); + $hits = json_decode($raw ?: '[]', true); + if (!is_array($hits)) $hits = []; + $now = time(); + $hits = array_values(array_filter($hits, fn($t) => (int)$t > $now - CHAT_RATE_WINDOW)); + sort($hits); + $ok = count($hits) < $max; + if ($ok) $hits[] = $now; + ftruncate($fp, 0); + rewind($fp); + fwrite($fp, json_encode($hits)); + fflush($fp); + flock($fp, LOCK_UN); + fclose($fp); + @chmod($file, 0664); + if ($ok) return ['ok' => true, 'retry_after_ms' => 0]; + $oldest = (int)($hits[0] ?? $now); + $wait = max(0, ($oldest + CHAT_RATE_WINDOW) - $now) * 1000; + return ['ok' => false, 'retry_after_ms' => $wait]; +} + +function chat_thread_count(): int { + $g = glob(CHAT_DIR . '/*.json'); + return $g ? count($g) : 0; +} diff --git a/api/chat/poll.php b/api/chat/poll.php new file mode 100644 index 0000000..601beca --- /dev/null +++ b/api/chat/poll.php @@ -0,0 +1,27 @@ +<?php +// api/chat/poll.php — read back the caller's own thread. The thread comes from the +// token, never from a query parameter: no id to guess, no IDOR, no listing. +// Contract: _docs/chat-contract.md +require_once __DIR__ . '/lib.php'; +// Note: poll is deliberately NOT rate limited. A 30 s poller is 120 GETs/hour; +// sharing the write window would make the client throttle itself into silence. + +$token = chat_token(); +$thread = chat_thread($token); +$since = isset($_GET['since']) ? (int)$_GET['since'] : 0; // epoch seconds, numeric + +$all = chat_read($thread); +$messages = $since > 0 + ? array_values(array_filter($all, fn($m) => (int)($m['epoch'] ?? 0) > $since)) + : $all; +$lastAll = $all ? end($all) : null; + +json_response([ + 'success' => true, + 'thread' => $thread, + 'messages' => array_slice($messages, -50), + // `awaiting` replaces `unread`: the server keeps no read state, so an unread + // counter could never clear. Who spoke last is knowable. + 'awaiting' => (bool)($lastAll && ($lastAll['author'] ?? '') !== 'hermes'), + 'cursor' => $lastAll ? (int)($lastAll['epoch'] ?? 0) : 0, +]); diff --git a/api/chat/post.php b/api/chat/post.php new file mode 100644 index 0000000..07cee65 --- /dev/null +++ b/api/chat/post.php @@ -0,0 +1,57 @@ +<?php +// api/chat/post.php — append one message to the caller's own thread. +// Contract: _docs/chat-contract.md +require_once __DIR__ . '/lib.php'; + +if ($_SERVER['REQUEST_METHOD'] !== 'POST') { + chat_error('method not allowed', 405, 'method'); +} + +// auth first: a 401 must mean nothing touched disk +$token = chat_token(); +$thread = chat_thread($token); + +$raw = file_get_contents('php://input'); +if (!$raw) error_response('Empty request body'); +if (strlen($raw) > 32 * 1024) chat_error('payload too large (max 32KB)', 413, 'too_large'); + +$payload = json_decode($raw, true); +if (!is_array($payload)) chat_error('invalid JSON', 400, 'bad_json'); + +$text = trim((string)($payload['text'] ?? '')); +if ($text === '') chat_error('text is required', 400, 'text_required'); +// Reject rather than truncate: a shortened question earns a wrongly scoped answer. +// mb_strlen = codepoints, not bytes: 4000 emoji are 4000 chars and 16 KB. +if (mb_strlen($text) > CHAT_MAX_TEXT) { + chat_error('text too long (max ' . CHAT_MAX_TEXT . ' chars)', 413, 'too_long', + ['limit' => CHAT_MAX_TEXT, 'unit' => 'codepoints', 'actual' => mb_strlen($text)]); +} + +$kind = (string)($payload['kind'] ?? 'operator'); +if (!in_array($kind, ['operator', 'agent'], true)) $kind = 'operator'; + +if (!is_file(chat_path($thread)) && chat_thread_count() >= CHAT_MAX_THREADS) { + chat_error('thread limit reached', 507, 'thread_limit', ['limit' => CHAT_MAX_THREADS]); +} +$rate = chat_rate_check($thread); +if (!$rate['ok']) { + // The only retryable failure. Everything else (401/413/507) is permanent for this payload. + header('Retry-After: ' . (int)ceil($rate['retry_after_ms'] / 1000)); + chat_error( + 'rate limit: ' . ($rate['scope'] === 'thread' ? CHAT_RATE_MAX . ' per thread' : CHAT_IP_MAX . ' per IP') . ' per 10 min', + 429, 'rate_limited', + ['scope' => $rate['scope'], + 'retry_after_ms' => $rate['retry_after_ms'], + 'limit' => $rate['scope'] === 'thread' ? CHAT_RATE_MAX : CHAT_IP_MAX, + 'window_s' => CHAT_RATE_WINDOW] + ); +} + +$msg = chat_append($thread, $kind, $text); +json_response([ + 'success' => true, + 'id' => $msg['id'], + 'ts' => $msg['ts'], + 'epoch' => $msg['epoch'], + 'thread' => $thread, +], 201); diff --git a/api/debug/agent-replies.php b/api/debug/agent-replies.php index 070c0e6..2d2e2b2 100644 --- a/api/debug/agent-replies.php +++ b/api/debug/agent-replies.php @@ -29,29 +29,17 @@ if ($method !== 'POST') { error_response('Method not allowed', 405); } -$raw = file_get_contents('php://input'); -if (!$raw) error_response('Empty request body'); -if (strlen($raw) > 16384) error_response('Payload too large (max 16KB)', 413); - -$p = json_decode($raw, true); -if (!is_array($p)) error_response('Invalid JSON'); - -$body = trim((string)($p['body'] ?? '')); -if ($body === '') error_response('body is required'); -if (mb_strlen($body) > 2000) $body = mb_substr($body, 0, 2000); - -$list = load_json(REPLIES_FILE); -if (!is_array($list)) $list = []; - -$author = (($p['author'] ?? 'hermes') === 'newos4') ? 'newos4' : 'hermes'; -$entry = [ - 'id' => 'REP-' . gmdate('Ymd-His') . '-' . substr(bin2hex(random_bytes(3)), 0, 6), - 'created_at' => gmdate('c'), - 'author' => $author, - 'body' => $body, -]; - -$list[] = $entry; -save_json(REPLIES_FILE, array_slice($list, -200)); - -json_response(['success' => true, 'id' => $entry['id']], 201); +// v4.6.71: retired. Two write paths meant two truths — messages written here never +// reached the chat thread, so they looked like a conversation nobody answered. +// One write path now: /api/chat/post.php (contract: _docs/chat-contract.md). +http_response_code(410); +header('Content-Type: application/json'); +echo json_encode([ + 'success' => false, + 'code' => 'gone', + 'error' => 'POST /api/debug/agent-replies.php is retired', + 'use' => '/spaces/newos4/api/chat/post.php', + 'contract'=> '/spaces/newos4/_docs/chat-contract.md', + 'read' => '/spaces/newos4/api/chat/poll.php', +]); +exit; diff --git a/scripts/chat.py b/scripts/chat.py new file mode 100755 index 0000000..e19fb6b --- /dev/null +++ b/scripts/chat.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +"""scripts/chat.py — read and reply to newOS4 chat threads as Hermes. + + chat.py list [--json] # threads, counts, who spoke last + chat.py show <thread> # full transcript + chat.py reply <thread> "…" # append a Hermes reply (atomic, locked) + +Store: data/chat/<thread>.json — contract: _docs/chat-contract.md +Thread id is 16 lowercase hex = sha256(browser token)[0:16]. +""" +import json, sys, os, re, datetime, secrets, fcntl + +CHAT_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', 'data', 'chat') +THREAD_RE = re.compile(r'^[a-f0-9]{16}$') + + +def utc_iso(): + # UTC, matching the PHP side. A local offset here breaks every comparison downstream. + return datetime.datetime.now(datetime.timezone.utc).strftime('%Y-%m-%dT%H:%M:%S+00:00') + + +def now_ms(): + return int(datetime.datetime.now(datetime.timezone.utc).timestamp() * 1000) + + +def load(path): + try: + with open(path) as f: + return json.load(f) + except Exception: + return [] + + +def threads(): + out = [] + if not os.path.isdir(CHAT_DIR): + return out + for name in sorted(os.listdir(CHAT_DIR)): + if not name.endswith('.json') or name.startswith('.'): + continue + msgs = load(os.path.join(CHAT_DIR, name)) + if not msgs: + continue + last = msgs[-1] + out.append({ + 'thread': name[:-5], + 'count': len(msgs), + 'last_ts': last.get('ts', ''), + 'last_author': last.get('author', ''), + 'awaiting': last.get('author') != 'hermes', + 'preview': (last.get('text') or '')[:100].replace('\n', ' '), + }) + return out + + +def reply(thread, text): + # A typo'd id must fail, not silently create a thread nobody reads. + if not THREAD_RE.match(thread): + raise SystemExit('invalid thread id (expected 16 lowercase hex chars)') + path = os.path.join(CHAT_DIR, thread + '.json') + if not os.path.exists(path): + raise SystemExit('no such thread: ' + thread) + lockpath = os.path.join(CHAT_DIR, '.lock-' + thread) + with open(lockpath, 'a+') as lock: + fcntl.flock(lock, fcntl.LOCK_EX) + try: + msgs = load(path) + msg = { + 'id': 'MSG-' + datetime.datetime.now().strftime('%Y%m%d-%H%M%S') + '-' + secrets.token_hex(3), + 'ts': utc_iso(), + 'epoch': now_ms(), + 'author': 'hermes', + 'text': text, + } + msgs.append(msg) + msgs = msgs[-200:] + tmp = path + '.tmp-' + secrets.token_hex(4) + with open(tmp, 'w') as fh: + json.dump(msgs, fh, ensure_ascii=False) + os.chmod(tmp, 0o664) + finally: + fcntl.flock(lock, fcntl.LOCK_UN) + os.replace(tmp, path) # atomic: a poll never sees a half-written file + return msg + + +def main(): + if len(sys.argv) < 2: + print(__doc__) + return 1 + cmd = sys.argv[1] + + if cmd == 'list': + rows = threads() + if '--json' in sys.argv: + print(json.dumps(rows, indent=2)) + return 0 + if not rows: + print('(no threads)') + return 0 + for t in rows: + flag = 'AWAITING' if t['awaiting'] else 'replied ' + print(f"{t['thread']} {flag} {t['count']:>3} msg {t['last_ts']} {t['preview']}") + return 0 + + if cmd == 'show' and len(sys.argv) >= 3: + msgs = load(os.path.join(CHAT_DIR, sys.argv[2] + '.json')) + if not msgs: + print('(empty or unknown thread)') + return 1 + for m in msgs: + print(f"[{m.get('ts','')}] {m.get('author','')}: {m.get('text','')}") + return 0 + + if cmd == 'reply' and len(sys.argv) >= 4: + msg = reply(sys.argv[2], ' '.join(sys.argv[3:])) + print('replied', msg['id']) + return 0 + + print(__doc__) + return 1 + + +if __name__ == '__main__': + sys.exit(main()) |