v1.1.0 Pi v1.1.0 源码分析 Analysis of Pi v1.1.0 · 2026-10-07
env env
Pi Durable 的远程执行环境(经 SSH 部署的小型守护进程) Remote execution environments for Pi Durable: a small daemon deployed over SSH and its ExecutionEnv client
Infra Infra Infra
← → 翻页 · F 全屏 · O 目录 · T 明暗 · L 语言 ← → navigate · F fullscreen · O overview · T theme · L language
01 · 两半:TypeScript 客户端与 Rust 守护进程 01 · Two halves: a TypeScript client and a Rust daemon Infra
两半:TypeScript 客户端与 Rust 守护进程 Two halves: a TypeScript client and a Rust daemon pi-env: a 2,073-line TypeScript client + a 4,027-line Rust daemon 本地:Durable worker 所在的机器 Local: where the Durable worker runs src/*.ts 远端:工具真正运行的机器 Remote: where the tools actually run daemon/src/*.rs 包与平台 Package and platforms 939 remote-env.ts RemoteExecutionEnv:实现 Durable 的 ExecutionEnv RemoteExecutionEnv: a Durable ExecutionEnv 530 ssh.ts 主机密钥 · 探测平台 · 部署 · 启动 host keys · detect · deploy · launch 384 connection.ts Connection:帧、请求 id、心跳、重连 Connection: frames, request ids, ping, restart 158 watch.ts RemoteWatcher:断线后重新打开 RemoteWatcher: reopens after a lost link 34 errors.ts toFileError:错误码映射 toFileError: error-code mapping 28 index.ts 公开 API public API 516 main.rs serve:同步行、调度、工作线程 serve: sync line, scheduling, workers 778 watch.rs NodeFileWatcher 的移植 a port of NodeFileWatcher 483 exec.rs 命令、进程组、超时、溢出文件 commands, process groups, timeouts, spill 269 window.rs 输出窗口与节流 output window and pacing 473 fs.rs · scan.rs 文件操作 · 行扫描 file ops · line scans 988 sys/unix.rs · sys/windows.rs 平台差异 platform specifics @earendil-works/pi-env chord ^1.1.0 pi-durable ^1.1.0 Linux macOS Android (Termux) Windows x86-64 arm64 ssh stdin/stdout 随包附带 bin/pi-env-{platform}-{arch}/pi-env ships bin/pi-env-{platform}-{arch}/pi-env README:Durable worker、存储与凭据留在本地 README: the Durable worker, storage and credentials stay local 标签 = 行数 tag = lines 本地 TypeScript local TypeScript 远端 Rust remote Rust 请求 requests 结果与事件 results & events
支持的远端:Linux、macOS、Android(Termux)与 Windows,x86-64 与 arm64。Windows 上字符串命令像 NodeExecutionEnv 那样通过 Git Bash 运行,argv 命令直接运行。 Supported remotes: Linux, macOS, Android (Termux) and Windows, on x86-64 and arm64. On Windows, string commands run through Git Bash as NodeExecutionEnv runs them; argv commands run directly.
02 · 分工与语义 02 · Split of work and semantics Infra
分工与语义 Split of work and semantics Split of work: results must match NodeExecutionEnv on the remote machine 客户端(TypeScript,本地) Client (TypeScript, local) 守护进程(Rust,远端) Daemon (Rust, remote) 路径解析 Path resolution ~(远端 home)、file://、相对 cwd;Windows 驱动器相对路径 ~ (remote home), file://, relative to cwd; Windows drive-relative 取消检查点 Abort checkpoints 每个方法的检查点与 NodeExecutionEnv 相同 each method's checkpoints, as NodeExecutionEnv 错误码映射 Error-code mapping ENOENT → not_found …(toFileError) ENOENT → not_found … (toFileError) readFile 的分块与解码 readFile chunking & decoding StringDecoder,保留 BOM StringDecoder, BOM kept writeFile 512 KiB 一块,每块前检查取消 512 KiB writes, abort check before each 行读取器 Line reader 定位读取 + Durable 的 StreamDecoder positional reads + Durable's StreamDecoder 超时与结果优先级 Timeouts & result precedence 回调错误 > 超时 > 取消 > 溢出失败 > 退出码 callback error > timeout > abort > spill failure > exit code 系统调用 System calls lstat、open、pread、rename、rm … lstat, open, pread, rename, rm … 进程 Processes 每条命令一个新会话(进程组) a new session (process group) per command 溢出文件 Spill files pi-output-<uuid>.log,在新的 tmp- 目录里 pi-output-<uuid>.log in a fresh tmp- directory 行扫描 Line scans scanLines:在文件旁边数行 scanLines: counted next to the file 输出解码 Output decoding 每个流独立,WHATWG UTF-8(encoding_rs) per stream, WHATWG UTF-8 (encoding_rs) 文件监视 Watching 快照、原生事件、轮询:NodeFileWatcher 的移植 snapshots, native events, polling: a NodeFileWatcher port 两边由 Durable 的 env 一致性套件和 test/differential.test.ts 一起检查:随机操作序列在 RemoteExecutionEnv 与本机 NodeExecutionEnv 上比对。 Both sides are checked by Durable's env conformance suite and test/differential.test.ts: random operation sequences compared with NodeExecutionEnv. 只有错误消息可以不同;错误码才是契约。 Only error messages may differ; codes are the contract. 客户端 client 守护进程 daemon
docs/semantics.md 记录了标准库不遵守、但这里必须遵守的规则:递归 mkdir 遇到已有文件报 EEXIST;rm 目录不带 recursive 报 ERR_FS_EISDIR;listDir 按字节序;mkdtemp 的前缀用 POSIX path.join……docs/semantics.md records rules the standard libraries do not follow but this must: a recursive mkdir over an existing file fails with EEXIST; rm of a directory without recursive gives ERR_FS_EISDIR; listDir is in byte order; mkdtemp joins its prefix with POSIX path.join…
03 · 通过 SSH 连接 03 · Connecting over SSH Infra
通过 SSH 连接 Connecting over SSH Reaching a machine over SSH: trust → detect → deploy → start deployDaemon 的细节(POSIX) deployDaemon in detail (POSIX) src/ssh.ts sshArguments():每次调用都锁死这些选项 sshArguments(): these options on every call scanHostKey 用临时 known_hosts 抓取主机密钥,给用户比对指纹 capture the host key with a temp known_hosts; owner checks fingerprints acceptHostKey 写进应用自己的 known_hosts,固定别名 store it in the app's own known_hosts, under a fixed alias detectPlatform POSIX:sh -c uname …;Windows:PowerShell POSIX: sh -c uname …; Windows: PowerShell deployDaemon 按 SHA-256 命名、校验后才放到位 named by SHA-256, verified before it is put in place pi-env serve --token <hex> 经 ssh 启动;等到 "PI-ENV <token>" 这一行 started over ssh; wait for the "PI-ENV <token>" line BatchMode=yes ClearAllForwardings=yes ForwardAgent=no ForwardX11=no ControlMaster=no ControlPath=none RemoteCommand=none PermitLocalCommand=no SendEnv=-* ServerAliveInterval=15 GlobalKnownHostsFile=none HashKnownHosts=no IdentitiesOnly=yes StrictHostKeyChecking=yes UserKnownHostsFile=<app file> HostKeyAlias=<alias> connectSsh():立即探测与部署(3–4),守护进程在第一个请求时启动;sshConnection():全部懒执行, connectSsh(): detects and deploys now (3–4), the daemon starts on the first request; sshConnection(): all lazily; 失败就是该操作的错误,下一次操作再试。forgetHostKey() 删除已存的密钥。 a failure is that operation's error and the next one retries. forgetHostKey() removes a key. f=~/.pi/mobile/tools/pi-env-{sha256 前 32 位} f=~/.pi/mobile/tools/pi-env-{first 32 of sha256} # 检查:present → 直接用 · missing → 上传 · nohash → 报错 # check: present → use it · missing → upload · nohash → error t=$(mktemp "$d/.pi-env.XXXXXX"); cat > "$t" [ "$(hash "$t")" = {sha256} ] || 删除并报 corrupt [ "$(hash "$t")" = {sha256} ] || rm, "upload is corrupt" chmod 700 "$t"; mv -f "$t" "$f" # 删除其他 pi-env-*(运行中的保持打开直到退出) # remove other pi-env-* (running ones stay open until exit) hash 依次尝试 sha256sum、shasum、openssl;Windows 用 Get-FileHash 与 base64 上传。 hash tries sha256sum, shasum, openssl; Windows uses Get-FileHash and a base64 upload. 不转发任何东西、不复用连接、不执行配置的命令、不发送本地 locale。 No forwarding of any kind, no shared connections, no configured commands, no local locale. accept-new 只在 scanHostKey 的临时文件里使用。 accept-new is used only against scanHostKey's temporary file. 1 2 3 4 5 信任 trust 部署与启动 deploy & start 顺序 order
守护进程放在 ~/.pi/mobile/tools/pi-env-<sha256 前 32 位>:按内容命名,版本之间不会冲突;每次启动前都先校验哈希。 The daemon lives at ~/.pi/mobile/tools/pi-env-<first 32 of sha256>: named by content so versions never collide, and its hash is checked before every start.
04 · 线上协议 04 · The wire protocol Infra
线上协议 The wire protocol The wire format: after the sync line, only frames 同步 Sync 一帧(两个方向都一样,大端序,最多 16 MiB) One frame (both directions, big-endian, at most 16 MiB) 6 种帧类型 The 6 frame types connection.ts · frame.rs shell 启动文件可能打印的杂项… whatever shell startup files print… PI-ENV 3f9a…c1\n frame frame frame … length 之后的字节数 bytes after this field type 1–6 id 请求 id;0 = 不属于请求 request id; 0 = none jsonLength json JSON 对象 a JSON object bytes 文件内容 / 输出 file contents / output 1 2 3 4 5 6 客户端丢弃同步行之前的 The client discards 一切输出 everything before it u32 u8 u32 u32 UTF-8 raw bytes 长度 = length − 9 − jsonLength;JSON 解析失败 → EINVAL;帧格式错误 → 断开连接。 bytes length = length − 9 − jsonLength; unparsable JSON → EINVAL; a framing error ends the connection. request 客户端 → 守护进程 client → daemon json.op 指明操作 json.op names the operation result 守护进程 → 客户端 daemon → client 请求成功并结束 success; the request is finished error 守护进程 → 客户端 daemon → client { code, message, syscall?, path? } event 守护进程 → 客户端 daemon → client 进度:exec 输出、watch 变化;不结束请求 progress: exec output, watch changes; not finishing cancel 客户端 → 守护进程 client → daemon 取消请求 id cancel request id ping 双向 both 心跳;接收方忽略 liveness; ignored by the receiver 同步行 sync line 帧字段 frame fields 帧类型 frame types
文件内容和命令输出走原始字节负载,不做 base64;更大的传输由客户端拆分。 File contents and output travel as raw payload bytes, not base64; larger transfers are split by the client.
05 · 连接的生命周期 05 · The connection lifecycle Infra
连接的生命周期 The connection lifecycle The life of a connection: lazy start, heartbeats, restart after loss 连接丢失时(客户端) When the connection is lost (client) connection.ts 上层如何感知 What the layers above see 第一个请求 first request 懒启动 lazy start ssh … serve --token <hex> PI-ENV <token> 同步行 sync line hello { protocol: 1 } 60 s 内必须完成 within 60 s { os, arch, home, … } 请求 / 结果 / 事件 requests / results / events 工作线程 · exec · watch workers · exec · watch 双向 ping,每 5 s ping both ways, every 5 s 30 s 没收到任何字节 30 s without any bytes 杀掉所有进程组并退出 kill every process group, exit 客户端 Connection client Connection 守护进程 daemon ssh stdin 结束时守护进程也退出: The daemon also exits when stdin 断线 = 远端命令停止。 ends: a dropped link stops commands. 进行中的请求失败 Requests in flight fail { code: "unknown", lost: true } 变更类请求的结果因此未知 A mutation's outcome is then unknown 写入可能已发生,也可能没有 the write may or may not have happened 下一个请求启动新的守护进程 The next request starts a new daemon 再次同步 + hello sync line + hello again 旧会话的句柄不会落到新进程上 Old handles never reach the new daemon 它们的请求直接失败 their requests fail instead 原因:进程退出、帧损坏("Corrupt frame from pi-env")、30 s 静默、hello 协议不符。 Causes: process exit, a corrupt frame, 30 s of silence, a hello protocol mismatch. isConnectionLost(err) 区分"断线"和"守护进程拒绝了请求" tells "lost" apart from "the daemon refused" RemoteExecutionEnv 把丢失映射成 FileError / ExecutionError 的 unknown maps a loss to FileError / ExecutionError unknown RemoteWatcher 新守护进程启动后重新打开监视并报告 overflow reopens the watch on the new daemon, reports overflow sshConnection() 懒连接:网络或密钥错误只是那一次操作的错误 lazy: a network or key failure fails that operation only 请求 request 回应 reply 丢失 loss
丢失时正在进行的变更(写入、改名…)结果未知:可能执行了,也可能没有。 For mutations in flight when the link is lost (writes, renames…), the outcome is unknown: they may or may not have happened.
06 · 命令与输出 06 · Commands and output Infra
命令与输出 Commands and output exec: commands, output windows, spill files and cancellation 输出:event { kind: "output", stream } Output: event { kind: "output", stream } 调度:回复先于大块数据 Scheduling: replies before bulk data 命令怎样结束 How a command settles exec { command | argv, cwd, env, … } timeoutMs? · spill? · window? 字符串 → shell string → shell shellPath,否则 /bin/bash、which bash、sh;-c shellPath, else /bin/bash, which bash, sh; -c argv 数组 → 直接运行 argv array → run directly Windows 按 libuv 规则搜索,拒绝批处理 Windows: libuv search, batch files refused 新会话(进程组) new session (process group) 默认信号处理 · 空信号掩码 default signal dispositions · empty mask 控制队列 control queue 结果 · 错误 · ping · watch 事件 results · errors · pings · watch 大块队列 bulk queue 命令输出 · > 64 KiB 的结果 command output · results > 64 KiB stdout → ssh → 客户端 → ssh → client { exitCode } timeout aborted 137 spawn_error shell_unavailable 每个流独立解码为 UTF-8,作为负载字节发送。 Each stream is decoded to UTF-8 and sent as payload bytes. window { maxBytes, maxLines, minIntervalMs, bytesPerSecond } 合并并按调用方的提交节奏发送;每条命令最多一帧未发出; coalesced and paced like the caller's commits; at most one unsent frame; 超出窗口的部分在下一帧里变成 skipped { bytes, newlines, … } output beyond the window becomes skipped { bytes, newlines, … } spill { afterBytes, afterLines } 越线后完整原始输出写入 tmp-…/pi-output-<uuid>.log; once crossed, the full raw output goes to tmp-…/pi-output-<uuid>.log; 结果带 spillPath(超时 / 取消时也带)。 the result carries spillPath (also on timeout / abort). 进程退出后继续收集输出,直到两个管道都关闭或 100 ms 无输出。 After exit, output is collected until both pipes end or 100 ms pass silently. 没有 window 的命令:未发输出超过 4 MiB 就等待——像管道写满一样拖慢命令。 Without a window: output waits while more than 4 MiB is unsent, slowing the command like a full pipe. 正常退出;被信号杀死报告 128 + 信号 normal exit; killed by a signal → 128 + signal 超时:SIGKILL 整个进程组 timeoutMs passed: SIGKILL the group cancel 帧:杀进程组,以 aborted 结束 a cancel frame: kill the group, settle aborted cleanup():cancel { mode: "kill" },不 abort cleanup(): cancel { mode: "kill" }, no abort cwd 不存在等 missing cwd, etc. 配置的 shellPath 不存在 the configured shellPath is missing 请求 request 输出 output 控制优先 control first 结束方式 settlements
调度:结果、错误、ping 和 watch 事件排在命令输出之前,超过 64 KiB 的结果也排到输出后面——慢链路只延迟大块数据,不延迟回复。 Scheduling: results, errors, pings and watch events go before queued command output, and results above 64 KiB queue behind output too — a slow link delays bulk data, not replies.
07 · 文件操作 07 · File operations Infra
文件操作 File operations File operations: ExecutionEnv methods → the daemon's 17 file ops RemoteExecutionEnv 守护进程的文件 op The daemon's file ops protocol.md 客户端的分块与流水线 Client chunking and pipelining remote-env.ts 错误码映射(toFileError) Error mapping (toFileError) errors.ts 标准库不遵守、但这里遵守的规则(节选) Rules the standard libraries do not follow (excerpt) semantics.md lstat realpath write writeChunk truncate fsync rename mkdir rm mkdtemp open pread fstat scanLines opendir readdir close readTextFile / readBinaryFile → open(mode:"read") → pread… openBinaryReader → open(noFollow) · pread · fstat openTextLineReader → pread + StreamDecoder writeFile / appendFile → write(keep) · writeChunk… listDir / openDirReader → opendir · readdir(max) fileInfo / exists → lstat canonicalPath → realpath renameFile / remove → rename · rm createDir / createTempDir → mkdir · mkdtemp truncateFile / flushFile → truncate · fsync watch → watch 全部在 16 个工作线程的固定池里运行; All run on a fixed pool of 16 worker threads; 同一句柄的分块写按到达顺序执行。 chunk writes to one handle run in order. 句柄是数字,close 或连接结束前有效; Handles are numbers, valid until close or 打开超过 4096 个 → EMFILE。 the connection ends; over 4096 → EMFILE. 路径在客户端解析成绝对路径后再发送。 Paths are made absolute by the client. READ_CHUNK 256 KiB × 8 二进制读,8 个并发 binary reads, 8 in flight WRITE_CHUNK 512 KiB × 8 writeFile,8 个并发 writeFile, 8 in flight READ_FILE_CHUNK 512 KiB readFile 每次读 each readFile read LINE_CHUNK 64 KiB 行读取器每次读 each line-reader read ENOENT → not_found EACCES · EPERM → permission_denied ENOTDIR → not_directory EISDIR → is_directory EINVAL … → invalid · mkdir -p:已有文件 → EEXIST recursive mkdir: an existing file → EEXIST · rm 目录不带 recursive → ERR_FS_EISDIR rm of a dir without recursive → ERR_FS_EISDIR · listDir:按字节序排序名字 listDir: names in byte order · openBinaryReader:noFollow → O_NOFOLLOW openBinaryReader: noFollow → O_NOFOLLOW 客户端方法 client methods 守护进程 op daemon ops 错误映射 error mapping
write 默认像递归 mkdir 那样创建缺失的父目录;带 keep 时返回句柄供后续 writeChunk,一块失败后后续块都报 EBADF。write creates missing parents like a recursive mkdir by default; with keep it returns a handle for later writeChunks, and after a failed chunk later chunks fail with EBADF.
08 · 文件监视 08 · Watching Infra
文件监视 Watching Watching: NodeFileWatcher running next to the files 变化怎样被发现 How changes are found event 帧(请求一直持续到被取消) Event frames (the request lasts until cancelled) 断线之后(客户端 watch.ts) After a lost link (client watch.ts) env.watch(targets, …) RemoteWatcher(客户端) RemoteWatcher (client) watch { targets, mode?, pollIntervalMs?, … } targets:{ path, recursive, exclude } targets: { path, recursive, exclude } 守护进程 watch.rs daemon watch.rs Durable NodeFileWatcher 的移植 a port of Durable's NodeFileWatcher 原生事件 native events inotify / FSEvents 去抖 50 ms debounce 50 ms FSEvents 再等 500 ms FSEvents settles 500 ms 重新扫描 rescan 最多 10,000 个目录 up to 10,000 directories 比较快照 diff snapshots ≤ 256 KiB 的文件比哈希 hash files ≤ 256 KiB change { paths } 快照差异 + 事件路径 snapshot diff + event paths 原生事件只触发重新扫描;读取不算事件。Windows、网络与 FUSE 文件系统改用轮询(默认 2000 ms)。 Native events only trigger a rescan; reads are not events. Windows, network and FUSE file systems poll instead (2000 ms by default). { kind: "ready", mode } 覆盖范围建立;之后才有变化 coverage established; changes follow { kind: "change", paths } 一批变化的路径 a batch of changed paths { kind: "change", overflow: true, mode: "polling" } 原生监视用尽 → 转轮询 native watches ran out → polling { kind: "error", code, message } 之后不再有任何事件 nothing follows 新守护进程启动后,RemoteWatcher 重新打开同样的监视, Once a new daemon starts, RemoteWatcher reopens the same watch 并报告一次 overflow——期间可能漏掉了变化。 and reports overflow: changes may have been missed meanwhile. 重连退避:1 s 起,最多 30 s。 Reconnect backoff: from 1 s up to 30 s. cancel 帧让 watch 以 {} 结束。 A cancel frame finishes the watch with {}. 收到 overflow 的使用方应当整体重新读取, A consumer that sees overflow should re-read 而不是只看路径列表。 everything rather than trust the path list. 客户端 client 守护进程 daemon 事件 events
去抖 50 ms(FSEvents 再等 500 ms 稳定);轮询默认每 2000 ms;最多 10,000 个目录;≤ 256 KiB 的文件比较哈希。 Debounce 50 ms (FSEvents settles another 500 ms); polling every 2000 ms by default; at most 10,000 directories; files ≤ 256 KiB are compared by hash.
扩展点 Extension points pi-coding-agent
env 的扩展点 Where to plug into env 1
RemoteExecutionEnv({ connection, id, cwd }) RemoteExecutionEnv({ connection, id, cwd })
任何 Connection 都行:ssh、管道或测试里的本地进程。 Any Connection works: ssh, a pipe or a local process in tests.
2
sshConnection / connectSsh(options) sshConnection / connectSsh(options)
懒或立即;binary 可替换,loginShell 可选。 Lazy or eager; the binary can be replaced, loginShell is optional.
3
scanHostKey · acceptHostKey · forgetHostKey scanHostKey · acceptHostKey · forgetHostKey
把主机密钥信任交给你的 UI。 Put host-key trust in your own UI.
4
deployDaemon(target, remote, binary?) deployDaemon(target, remote, binary?)
单独部署,或部署自己构建的守护进程。 Deploy on its own, or deploy a daemon you built.
5
PI_ENV_DAEMON PI_ENV_DAEMON
测试时指定守护进程二进制(CI 用发布构建)。 Point tests at a daemon binary (CI uses release builds).