https://gitlab.synchro.net/main/sbbs/-/commit/7471d1b776cb2d7ea274a41e
Added Files:
docs/zmodem_comparison.md
Log Message:
docs: add ZMODEM implementation comparison (sexyz vs lrzsz vs rzsz)

A performance, scalability, and robustness analysis of Synchronet's ZMODEM (sexyz + the shared zmodem.c, which SyncTERM also uses) benchmarked against lrzsz 0.12.21rc and Chuck Forsberg's final rzsz 3.73 (modern branch).

Headline findings, all empirically measured:

- sexyz's sender is ~25x slower than lrzsz's lsz on a fast link (~8 vs
~204 MB/s), from two stacked overheads isolated by linking the real
zmodem.o behind two send paths: zmodem.c's per-byte callback + escape +
CRC design caps any sender at ~26 MB/s (shared, so SyncTERM inherits it),
and sexyz.c's ring-buffer + output-thread drops that a further ~3x to
~8 MB/s (sexyz-only; a futex storm + ~84-byte writes). SyncTERM avoids
the second layer, so its upload runs ~26 MB/s.
- A windowed (-w) send hangs the instant the file offset crosses 2 GiB:
zmodem.c narrows transmit-window/ACK positions to signed int32_t. Shared
with SyncTERM. The data path (fseeko/int64) is otherwise 64-bit clean;
4 GiB is a hard protocol ceiling (32-bit wire position field).
- sexyz's receiver is fine past 2 GB; Forsberg's sender runs ~92 MB/s but
its receiver won't complete headlessly (serial-tty assumptions).

Includes a conditions matrix (block size, CRC, latency, bandwidth
asymmetry, injected errors) and per-finding attribution to sexyz.c vs the shared zmodem.c vs SyncTERM's term.c. Relates to the sexyz-throughput and 2-GiB-window GitLab issues.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/f29244a40663ce7f99d3e4e6
Modified Files:
docs/zmodem_comparison.md
Log Message:
docs: note the sexyz SINGLE_THREADED build is not a throughput fix

Document that sexyz.c:84 still carries a source-level
`#define SINGLE_THREADED FALSE` (no makefile toggle), but its
single-threaded send_byte (sexyz.c:682) writes one byte per write()
syscall via sendbuf() Ä trading the two-thread futex storm for a
write-syscall storm, generally worse. Add a three-way send-architecture
table (threaded / single-threaded / buffered) and a recommendation caveat:
the fix is output buffering (flush per subpacket, as SyncTERM's term.c
does), not thread count.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/5c0e802235bfc797afbdadb1
Modified Files:
docs/zmodem_comparison.md
Log Message:
docs/zmodem: correct throughput magnitudes to steady-state; version-tag

The initial throughput figures were measured on 32 MB transfers whose
short duration was dominated by per-transfer startup, which deflated and distorted the rates. Re-measured at 256 MB steady-state:

- sexyz sender was ~18x (not ~25x) slower than lsz: ~11 vs ~204 MB/s.
- The two layers are ~7.5x sexyz.c (ring per-byte, 85->11, now fixed by
the send-batching change -> ~66 MB/s) and ~2.4x zmodem.c (per-byte send
design, 204->85, shared with SyncTERM), not the ~3x/~8x quoted before.
- The SyncTERM-model (ztx_buf) sender runs ~85 MB/s, not ~26; sexyz
receiver ~130; Forsberg sender ~107.

The mechanism findings (futex storm, ~84-byte writes, ring-per-byte) were unaffected -- only the magnitudes changed.

Also: tag every number to a component version (sexyz.c revision 3.3->3.4, zmodem.c rev 2.2->2.3) since Synchronet's side is actively changing, and
mark the recommendations with status (Fix A and the 2 GB fix done, Fix B
and the adaptive-block-length model pending).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/9c73c787d0f036e0eff35085
Modified Files:
docs/zmodem_comparison.md
Log Message:
docs/zmodem: record that the send-batching fixes were reverted

Multi-run error-recovery testing showed both batching prototypes regress
error recovery under heavy injected corruption: the shipped per-byte
sender passes 3/3 (~50 s), the Fix A prototype (batch spans into the ring, sexyz.c) timed out 0/3, and the Fix B prototype (send_buf span callback, zmodem.c) hard-failed 1/3. Root cause: batching a whole subpacket before
one blocking flush starves the back-channel during retransmit storms (the sender is stuck in the flush and can't service ZRPOS), stalling the
transfer; the original per-byte-to-ring keeps the pipeline draining and
stays responsive.

Both prototypes were reverted. The only shipped code change remains the
2 GB signed-position fix in zmodem.c (rev 2.3, GitLab #1196); sexyz.c is unchanged (stays 3.3). The ~66/~88 MB/s figures are retained in the doc, relabeled as reverted prototypes, because the root-cause analysis (ring per-byte -> futex storm / 84-byte writes) is still valid and the eventual
fix must service the back-channel while sending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/142d6aa509d91e348bd28dd2
Modified Files:
docs/zmodem_comparison.md src/bench/zmodem/README.md
Log Message:
docs/zmodem, bench: correct SyncTERM/localhost/recall claims (per Deuce)

Deuce (SyncTERM's author) corrected three framings; retained with caveats:

- "SyncTERM ~85 MB/s (faithful ztx_buf model)" is wrong. ztx_buf models the
SHAPE of SyncTERM's send path but under-measures its throughput ~3x
(Python-relay harness + poll-per-subpacket). SyncTERM's speed is a
deliberate BDP / socket-buffer choice (~75% of a 1Gb LAN), not an emergent
send-path fact. Relabelled ztx_buf as a "buffered floor (harness)" and
flagged ~85 as a localhost floor, not SyncTERM, throughout.
- The "recall stale bytes / abort-aware purge on ZRPOS" root-cause framing
(bench README) was a wrong garden path: you can't recall bytes once they
hit the network buffers, and lrz doesn't either yet recovers. The batched-
sender failures were implementation bugs, not inherent to batching.
- These are localhost CPU microbenchmarks; below ~8ms RTT the socket buffer
dominates the send loop. Optimizing it for localhost is largely beside the
point; the send-loop rewrite pays off only on fast LANs.

Also fixed a stale "fixed" on the ring-per-byte row (the batching prototype
was reverted, not shipped).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/9627d6504cecd3c53ff66c47
Modified Files:
docs/zmodem_comparison.md src/bench/zmodem/README.md
Log Message:
docs/zmodem, bench: re-measure after Deuce's send-path work; root-cause sexyz

Every throughput figure is re-measured in one interleaved batch (256 MB,
three passes, spread under 2%) so the tables are internally consistent
instead of splicing runs from different days: lrzsz 203.9, zmodem.c
rev 2.3 buffered floor 115.8, sexyz receiving 113.3, Forsberg 96.9,
zmodem.c rev 2.2 buffered floor 91.9, sexyz sending 11.5 MB/s.

Deuce's 2026-07-24 work on the shared send path (class-table byte
classifier, slicing-by-4 CRC-32, hoisted escape mask with noinline cold
paths, buffered fcrc32) moved the buffered floor 91.9 -> 115.8, +26%,
which SyncTERM inherits. The doc's old "second, shared lever, pending"
framing is replaced with what actually shipped.

Root-cause the sexyz sender properly rather than asserting it.
send_byte() takes the ring mutex twice per byte while output_thread
hot-loops on the same mutex: 44 CPU-seconds per 256 MB against lsz's
0.95, 1,464,130 voluntary context switches against 1,402, one futex per
17 bytes with 37% contended, and ~94-byte socket writes. A full sweep
of OutbufSize, OutbufHighwaterMark and OutbufDrainTimeout is flat inside
noise, so this is not a tuning problem and should not be chased as one.

Record the prototype ladder and what each one proved. Buffering the
producer removes the entire sexyz-specific penalty -- 115.8 MB/s at 1.0 CPU-second, exactly the engine ceiling -- but six variants now fail the
3x error-injection gate that the shipped per-byte sender passes. Two
results are worth keeping: rate-capping the batched sender to the
shipped sender's own 11 MB/s still fails while the shipped sender passes
3/3, so it is not an artifact of the higher data rate; and going single-threaded with a buffered send_byte holds full speed, drops
context switches to lsz's level, and reaches 2/3, leaving blocking
sendbuf() as the one missing piece. The recommendation is now what lrz actually is: single-threaded, buffered, non-blocking output with select
on both directions.

Also note the failure modes that cost time to rediscover (a void flush
callback swallowing a failed write, a sticky error latch spinning on zmodem_send_raw ERROR, position accounting desync) and that a
single-stream localhost benchmark cannot see per-byte CPU wins above
~100 MB/s -- two of Deuce's optimizations measure zero here because his
six interleaved 1 GiB transfers are CPU-saturated and this is not.

No code changes: all prototypes were measured out-of-tree and discarded.
Refs #1195, #1196.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/3d5c82cfc2265f8f131969c1
Modified Files:
docs/zmodem_comparison.md src/bench/zmodem/README.md
Log Message:
docs/zmodem, bench: 115.8 MB/s is a harness stall, not an engine ceiling

The previous commit treated the 115.8 MB/s "buffered floor" as zmodem.c's
CPU limit and built three conclusions on it. Measuring what the senders actually consume shows that is wrong:

ztx_buf (rev 2.3 + buffered send) 115.8 MB/s 0.69 CPU-s 30% CPU
lsz 203.9 MB/s 0.98 CPU-s 77% CPU

The engine does the same 256 MB in 30% less CPU than lrzsz and sits idle
for 70% of the wall clock; extrapolated to their CPU limits it is roughly
371 MB/s against lrzsz's 261. The receiver is not the constraint either,
since the same lrz absorbs 203.9 from lsz. So after Deuce's 2026-07-24 send-path work the engine is cheaper per byte than lrzsz's inlined loop,
and 115.8 is where ztx_buf's own I/O pattern stalls -- a blocking write()
per zmodem_flush and a poll() per subpacket in data_waiting. Under
strace -w, 93% of its in-syscall wall time lands in those polls; the
exact stall mechanism is not isolated and the doc says so.

Three claims are retracted. The 115.8 figure is not an engine ceiling.
The residual gap to lrzsz is not an escape/CRC deficit, so handing
send_byte a span would shave CPU that is not being spent -- that
recommendation is downgraded from "the second lever" to "done, and no
longer the lever". And the sexyz prototypes were not pinned by the
engine: all of them ran at 30-36% CPU because they share the same blocking-flush-per-subpacket shape, so a genuinely non-blocking sender's ceiling is unmeasured and plausibly above lrzsz's 203.9. That makes the transport rewrite more attractive than the previous commit implied, and
makes the next useful measurement a non-blocking prototype rather than
more engine micro-optimization.

The bench README gains the lesson that caused this: always record CPU
alongside goodput. A goodput number with no CPU figure beside it cannot distinguish fast code from idle code, and ztx_buf needs its own I/O
pattern fixed before it can serve as a reference at all.

No code changes. Refs #1195.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/c55eef72078c25e2b7322309
Modified Files:
docs/zmodem_comparison.md src/bench/zmodem/README.md
Log Message:
docs/zmodem, bench: lrzsz uses blocking stdio; a real flush() is not the fix

Two revisions of these documents told the reader that lrzsz's sender uses non-blocking output with select() on both directions, and that supplying
sexyz with the same would close the error-recovery gap. Reading the
source shows the first claim is false, and building the thing shows the
second one does not follow.

What lsz actually does: output is putchar/fwrite into stdout
(zm.c:109-110), flushmo() is fflush(stdout) (zglobal.h:411), and there is
no O_NONBLOCK, fcntl or FIONBIO anywhere in lsz.c or zm.c. The only
select() in the sender (lsz.c:754) is the pre-handshake purge that drains
stale input before ZRQINIT, not part of the data path. Escaping is a
lookup table with fwrite() of unescaped runs (zm.c:285-320) -- both the class-table idea Deuce landed this week and the "hand send_byte a span"
idea, present since the 1990s. Its per-subpacket loop (lsz.c:2093-2120)
is ZSDATA, then fflush(stdout), then a while (rdchk(fd)) loop that drains
the back-channel to empty. The property is "everything is on the wire
before you look for a reply", with blocking I/O throughout.

That did suggest a real difference: sexyz's flush() callback is a no-op
on the socket path -- it only fflush(stdout)s, and only in stdio mode --
so zmodem_flush() returns with the bytes still sitting in the ring.
zmodem.c already has the matching drain loop and already calls
zmodem_flush() per subpacket, so sexyz was the only piece not holding up
its end. Prototype v8 supplies a flush() that waits for the ring to
drain: full speed at 115.8 MB/s, and the error gate still fails, 1/3.
Worth fixing on its own merits, but it is not the cause.

Eight prototypes in, the useful output is the exclusion list, now
recorded: it is not the data rate (rate-capped control), not the batch
size (64 B to 4 KB all fail), not dropped or latched errors, not
flush-to-wire ordering, and only partly the output thread (removing it
moves 0-1/3 to 2/3 without fixing it). The mechanism is unidentified,
and the recommendation changes accordingly -- the next step is to
instrument a failing run and find what the receiver sees after a ZRPOS
that it does not see with the per-byte sender, rather than build a ninth prototype.

No code changes. Refs #1195.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net