• exec/load/filecontents_li

    From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Sun Sep 20 03:31:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/043587351c9f3c6f0078c857
    Added Files:
    exec/load/filecontents_lib.js exec/tests/filecontents_test.js
    Modified Files:
    exec/archive.js
    Log Message:
    Store archive content listings per file area in data/dirs/<code>.contents

    Viewers re-opened and enumerated an archive on every view. On a public file base under crawler load that is nearly all wasted work: across six days, 72%
    of ?view= requests were the first ever for that URL, so an expiring cache
    would sit near a 28% hit rate. It is also repeated exposure to the mozjs185 out-of-memory failure that d777bda5c5 (2026-06-01) only deflects crawlers
    away from.

    filecontents_lib.js keeps the extraction outcome in a per-area JSON file
    beside the base it derives from, keyed by filename and validated against the file's size and mtime. Records carry a format version and an extractor-set version. The second guards the verdict rather than the encoding, so a stored failure is re-examined when the extractor set changes while a stored listing
    is not. Whole-base projection is about 82 MB, so listings are stored uncapped.

    Failures are stored too. 171 of 600 sampled archive-extension files are .arc, which libarchive cannot read at all, so each one re-attempted a doomed extraction on every view. Those now fall back to lsar, which read 80 of 80 sampled .arc files, and are enumerated once.

    The external tool runs via system.exec() with its output redirected to a file rather than system.popen(), whose _popen() needs a console and so fails inside a Windows service. ILLEGAL_FILENAME_CHARS already excludes " and \, but $ and
    a backtick are legal in a file-base filename and are still expanded by sh inside double quotes, so a path containing either is never handed to a shell.

    archive.js is given a path and cannot know which area holds it, and cmdstr() has no specifier that could pass one, so the library maps the path back to an area itself. A file outside every area is enumerated but not stored, there being nowhere to persist it. Verbose listings still read the archive, since
    CRC and compression format are not stored.

    #1247

    Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Sun Sep 20 03:31:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/32d4aaf66581350ee596b795
    Modified Files:
    exec/load/filecontents_lib.js exec/tests/filecontents_test.js
    Log Message:
    filecontents: record which external lister reached a failure verdict

    The store is shared between hosts, and those hosts do not necessarily have
    the same tools installed. A host with no external lister would record "unreadable" for an ARC, which libarchive cannot read in any form, and a host that does have one would then trust that verdict and refuse to list a file it could read perfectly well.

    A negative record now carries the external lister that reached it, and a
    record whose lister differs from what this host has is treated as stale.
    The version is part of the identity, since builds years apart are in the wild (1.8.1 and 1.10.8 both are) and a format the older one rejects may be one the newer one reads. Successful listings are unaffected: a listing is a listing whoever produced it.

    Detection redirects stdout only, so a missing command reports on stderr and leaves the probe file empty, which holds for both /bin/sh and cmd.exe. The result is cached, and a host with no lister now skips the per-file shell it used to spawn only to fail.

    #1247

    Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Sun Sep 20 03:31:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/5c3806d896c2de5b9d373ea0
    Modified Files:
    exec/load/filecontents_lib.js exec/tests/filecontents_test.js
    Log Message:
    filecontents: qualify a successful listing with the extractor version too

    The failure path recorded the external lister with its version while the success path recorded a bare "lsar", on the reasoning that a listing is never re-examined so the version cannot matter. That misses the one thing the field is good for: identifying, after the fact, listings produced by a build that turned out to be untrustworthy. lsar returns a partial listing for a truncated archive rather than failing, and builds years apart differ on where they draw that line.

    Both paths now use the same identity string. libarchive exposes no version to JavaScript, so the native path stays unqualified, which is honest rather than tidy.

    #1247

    Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Tue Sep 29 17:07:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/e5c5b69e1c724c2fcf25a39b
    Modified Files:
    exec/load/filecontents_lib.js src/sbbs3/js_filebase.cpp
    Log Message:
    FileBase.update(): don't take a name's case difference for a rename

    FileBase lookups ignore case, but update() compared the caller's name with
    the stored one exactly. Updating useredit.exe (stored lowercase) under its on-disk spelling UserEdit.exe was taken as a rename to useredit.exe, which "already exists in base" -- the same record -- and threw. Had the check
    passed, the file on disk would have been renamed to lowercase.

    update() now carries on with the stored name when the caller's differs
    only in case. filecontents_lib.js also names the record by its stored
    spelling when saving an archive listing, so viewing such a file no longer
    fails with this error on builds without the fix.

    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Fri Oct 16 21:19:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/b958104a194d89c54072acc3
    Modified Files:
    exec/load/filecontents_lib.js exec/tests/filecontents_test.js src/sbbs3/filedat.c filedat.h js_archive.cpp
    Log Message:
    Read bare compressed files (.gz, .bz2, .xz, .zst) as single-file archives

    The Archive class and the file-base archive code could not open a bare compressed file, even with its extension listed as a supported archive
    type: they failed with "-30 Unrecognized archive format". libarchive only decompresses a non-archive when its "raw" format is enabled, and archive_read_support_format_all() deliberately leaves raw out.

    All four libarchive read sites now get their reader from
    new_archive_reader(), which adds the raw format, and pass each header
    through check_archive_entry(). Raw matches any input, so a raw result is accepted only when a decompression filter was applied; an uncompressed non-archive still fails as unrecognized. The entry is named after the file minus its compression extension (log.txt.gz -> log.txt) rather than from
    the gzip header, whose stored name is optional and may contain a path.

    A raw entry's size is unknown until decompressed, so Archive.list() reads
    the data to supply it.

    filecontents_lib.js caches "unreadable" verdicts that only a change in EXTRACTORS re-examines, so bump it to 2: .gz files previously recorded as unreadable get listed on their next view. Hosts sharing file areas should
    run this build before then, or an older build re-records the failure under
    the new version.

    Fixes #1278

    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Fri Oct 16 21:19:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/9de2c48d0265b07e1a8406d8
    Modified Files:
    exec/load/filecontents_lib.js
    Log Message:
    filecontents: treat bare .xz and .zst files as viewable archives

    libarchive now reads a bare compressed file as a single-file archive, and ARCHIVE_TYPES already listed gz and bz2, so add the other two compressors libarchive commonly supports. This lets the web file index and inventory_archives.js list their contents.

    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net