Fizz design decisions and rationale
=======================================

This file records intentional design choices, rejected alternatives, and
known trade-offs. It is not a TODO and not a changelog. The purpose is to
let a new contributor (or a future session) understand why things are the
way they are.


Goals and non-goals
-------------------

Goal: the smallest possible TCP network filesystem that works well on a
trusted home LAN, including machines as old as an Amiga 1200.

Non-goals: authentication, encryption, multi-user permissions, extended
attributes, high concurrency, WAN use.

The guiding principle is: if a real Amiga 1200 with 2 MB RAM can implement
it, it belongs. Everything else is out of scope unless it can be added
without burdening the Amiga side.


Protocol
--------

Binary, big-endian, request/response over a persistent TCP connection.
A sequence number in every frame enables pipelining; no client uses it yet
but the infrastructure is there (the FUSE client's reader thread was added
specifically for this).

Stateful (persistent connection) over stateless: file handles require
state. Stateless would force path re-sending on every read/write, which is
costly over a slow link.

Operations: STAT, READDIR, OPEN, CREATE, CLOSE, READ, WRITE, MKDIR,
UNLINK, RMDIR, RENAME, TRUNCATE. This is the minimum needed for a working
read/write mount. Nothing else has been added.

No OP_READLINK: a deliberate omission, not an oversight. See symlinks
section below.


Filesystem size (OP_STATFS)
---------------------------

OP_STATFS (opcode 15, protocol v3) returns total, used, and available byte
counts for the share. The Linux server calls statvfs(g_root) and reports
f_blocks/f_bavail in terms of f_frsize; used = total - avail (bavail, not
bfree, to exclude space reserved for root). The Amiga server calls Info()
on the root lock and derives the same three values from InfoData.

The FUSE client maps the result to struct statvfs with f_frsize = 512; Linux
df shows correct size, used, and percentage.

The Amiga client fetches OP_STATFS live on every ACTION_DISK_INFO (no
caching). AmigaDOS InfoData fields are 32-bit, so values are scaled: both
total and used block counts are right-shifted together until total fits in
a signed 32-bit LONG, and id_BytesPerBlock is multiplied by 2 for each
shift so the reported size in bytes remains correct. Without this
adjustment the block count is correct but the implied capacity is not.
For filesystems <= 2 GB no scaling is needed. For filesystems > 2 GB the
percentage display in Workbench is correct; the raw size shown by the Info
command is inherently limited by the 32-bit Amiga representation.


Path validation
---------------

safe_path() on the server enforces: must start with '/', no empty
components, no '.', no '..', no ':', no '\', no control bytes (0x00-0x1f).
These rules are intentionally strict so that the same validator is safe on
AmigaOS (where ':' is a volume separator and '//' means parent), Windows
(where '\' is a separator and ':' introduces alternate data streams), and
POSIX systems.

The assert at the end of safe_path() checks that the constructed host path
begins with g_root as a string. This is a sanity check on the string
construction, not a symlink-escape defence. See symlinks below.

h_readdir applies the same character rules to each entry name before
emitting it. Any name that would fail safe_path() (colon, backslash,
control byte) is silently skipped. This prevents a class of broken
listings where a name appears in readdir but every subsequent operation
on it returns EACCES because the path cannot be encoded in a valid request.


Reconnect
---------

Both clients attempt to reconnect automatically after a server disconnect,
retrying every RECONNECT_INTERVAL (5 s) for up to RECONNECT_ATTEMPTS (12)
tries (~60 s cap). Operations block during this window rather than failing
immediately, resuming transparently on success.

No open-file-handle state is restored after reconnect. The server assigns
new file handle numbers per connection; any handle from the previous
connection is stale and will produce EBADF on the next use. The
application must close and reopen the file. All path-based operations
(stat, readdir, open, mkdir, unlink, rename) resume without intervention.

The FUSE client (fizz-mount) implements reconnect in the reader thread,
which owns the socket lifecycle. do_rpc() and do_rpc_nowait() wait on
g_reconnect_cv instead of returning EIO immediately when g_dead is set;
the reader thread signals the condvar on success or on permanent failure
(g_perm_dead).

The Amiga client is single-threaded; reconnect runs inline inside
conn_dead(), which is called on any socket I/O error. The reconnect loop
uses Delay() between attempts. The triggering DOS packet always fails
(conn_dead always returns -EIO for the current op); subsequent packets
proceed normally after a successful reconnect.


Symlinks
--------

The current behaviour is intentionally "dirty but practical":

- lstat() is used for STAT and READDIR, so symlinks are reported as
  NT_LINK to clients.
- Regular (symlink-following) syscalls are used for OPEN, READDIR
  (opendir), MKDIR, RENAME, TRUNCATE, and UNLINK. A path that crosses a
  symlink will work on the server even though the symlink itself is
  reported as NT_LINK.
- There is no OP_READLINK. Clients cannot resolve symlink targets.
- NT_LINK entries should be skipped by clients that do not support them.

Two cleaner alternatives were considered:

  Fully transparent (stat everywhere, never emit NT_LINK): removes the
  inconsistency and needs no OP_READLINK. Rejected because directory
  symlinks can cause infinite traversal loops in recursive client
  operations (cp -r, backup tools). The server is stateless between
  requests and cannot detect these loops. No inode numbers are exposed
  in the protocol, so clients cannot detect them either.

  Drop all symlinks (hide NT_LINK from readdir/stat, return ENOENT):
  eliminates loop risk entirely and is clean for non-POSIX clients (Amiga
  has no POSIX symlinks). Rejected because it silently breaks things users
  have on disk, with no indication to them of why files are missing.

The current approach is a pragmatic middle ground: symlinks are visible
and usable if the client handles NT_LINK, and silently ignorable if it
does not. The known limitations (no OP_READLINK, loop risk, symlink
escape) are documented in protocol.h and in README Known Limitations.


Security model
--------------

No authentication. Any host that can reach the TCP port has full
read/write access. This is intentional and documented; the target use case
is a crossover cable, home switch, or VPN with trusted hosts only.

safe_path() blocks path traversal (.., //, etc.) but does not block
symlinks pointing outside the share root. This is also intentional: for a
trusted-LAN prototype, the complexity of per-component O_NOFOLLOW path
walking is not justified. The caveat is documented.

--readonly mode rejects all write ops at the server with EROFS. It does
not affect read access or the auth model.


Amiga server: concurrency model
--------------------------------

The Amiga server uses CreateNewProc (not CreateTask) so each connection
has its own full Process context and an isolated IoErr() / pr_Result2.

Three concurrency models were considered for the Amiga server:

  Exec tasks, one per connection: maps directly onto handle_conn(). The
  complication is socket handoff between tasks: on AmigaOS/bsdsocket each
  task is expected to have its own library base, and descriptors need
  explicit handoff. With CreateNewProc and NP_UserData the startup
  handoff is manageable.

  Multiplexed I/O (single task, WaitSelect on all sockets): clean in
  principle but undermined by the fact that AmigaDOS I/O calls (Read,
  Write, Seek) are synchronous and block the whole event loop. Ruled out
  unless async DOS packets are used.

  Async DOS packets + WaitSelect on combined socket+message-port mask:
  architecturally the most native AmigaOS approach. Issues ACTION_READ
  packets without blocking, waits for whichever of socket readiness or
  filesystem reply arrives first. No tasks needed. Higher implementation
  complexity. Deferred, not ruled out - the right long-term answer for
  the Amiga server.

Current choice: CreateNewProc, one per connection. The structure mirrors
the POSIX server (pthread per connection), the complexity is bounded, and
it can be replaced by the async approach later without touching protocol
or op handlers.


Amiga client: read-ahead
------------------------

AmigaDOS requests ~512 bytes per ACTION_READ. At 1 ms RTT that caps
sequential throughput at ~500 KB/s regardless of available bandwidth.
The client pre-fetches up to RA_SIZE (64 KB) per OP_READ and serves
subsequent ACTION_READ calls from the cache. This is the largest single
performance win for the backup/copy use case and was implemented with
~60 lines of new code in the handler (no protocol or server change).


Async write pipeline
--------------------

The dominant performance bottleneck for writes is round-trip latency: a
synchronous write model issues one OP_WRITE and waits for the server reply
before issuing the next. Over a 100 Mbit LAN with ~1 ms RTT this limits
write throughput to roughly (write size) / (RTT), well below line rate.

Both clients address this with a fire-and-forget write pipeline:

  FUSE client: do_rpc_nowait() sends an OP_WRITE and returns immediately
  without waiting for the reply. Pending write replies are drained by
  nfs_flush() and nfs_fsync(), which wait on a condvar signalled by the
  reader thread when all outstanding writes have been acknowledged.

  Amiga client: single-threaded, so true concurrency is not possible.
  Instead, drain_async_writes() is called at the start of do_rpc() before
  any non-write RPC, and explicitly in h_end() before OP_CLOSE, to drain
  any outstanding write replies before issuing the next operation. Write
  errors are propagated as DOSFALSE from the Close() handler.

Write errors from pipelined writes are deferred: the application gets an
error on Close(), not on Write(). This matches the behaviour of buffered
POSIX I/O (fclose returns EOF on flush error) and is acceptable for the
target use case.

A write-coalescing buffer (accumulate writes up to RA_SIZE before sending)
was tried on the Amiga client and reverted. The buffer split application
Write() calls larger than RA_SIZE into multiple RPCs, doubling round-trips
for large-block writers. Applications writing in chunks at or above RA_SIZE
already maximise RPC payload; buffering cannot help them.


FUSE client: pipelining
-----------------------

The FUSE client has a dedicated reader thread and a per-seq pending slot
table so that FUSE op threads can send requests and wait on individual
condvars without serialising through a global mutex. This makes the FUSE
client a useful performance testbed for the pipelined protocol path, and
also makes it a correct concurrent client (FUSE dispatches VFS ops from
multiple threads simultaneously).


Filename length limits on Amiga
--------------------------------

fib_FileName in the NDK FileInfoBlock is 108 bytes (BCPL string: one
length byte, up to 107 chars). The Amiga client truncates incoming names
at 106 chars (sizeof(fib_FileName) - 2). Names longer than 106 chars are
silently truncated in directory listings; any attempt to open such a file
by the truncated name will return an error from the server.

Standard OFS/FFS enforces a 30-character name limit at the filesystem
level. Extended filesystems (PFS3, SFS) support up to 107 chars within
the same struct. The Amiga client does not enforce the 30-char limit
itself - it truncates at 106 - so names between 31 and 106 chars will
appear and work correctly on PFS3/SFS volumes but may confuse programs
that assume the standard 30-char maximum.

A POSIX server can serve files with names up to NAME_MAX (typically 255
chars). Names between 107 and 255 chars are silently truncated on the
Amiga side.


File sizes > 2 GB on Amiga
--------------------------

AmigaDOS file offsets are signed 32-bit. The Amiga client uses ULONG (32
bits) for file positions and sizes. Files larger than 2 GB will not work
correctly. This matches the inherent AmigaOS limit and is documented; the
POSIX server and the wire protocol are 64-bit clean.


Amiga client: path conventions in build_path()
----------------------------------------------

AmigaDOS uses '/' differently from POSIX. In an AmigaDOS path string:
- A leading '/' (or any '/' that follows another '/') signals "go up one
  level from the current point in the path". So "/" alone means parent,
  "//" means grandparent, and "/foo" means parent then into foo.
- '/' between two named components is a separator, as in POSIX.
- A trailing '/' after a named component is a directory-type hint ("lock
  this as a directory") not a navigation step, and is stripped.
- '.' and '..' are ordinary legal filenames on AmigaDOS. Neither has a
  special meaning in path resolution the way they do on POSIX.

build_path() implements this:
- Leading '/' characters are consumed one at a time, each calling
  strip_last() on the running path (stops at root).
- A lone '.' component after consuming any leading slashes is treated as
  a current-directory reference (no-op). This is a pragmatic concession
  to POSIX-aware tools that construct "./" paths; it is not an AmigaDOS
  convention.
- '..' is not treated specially. It is a legal AmigaDOS name that simply
  does not exist on any served POSIX filesystem, so operations on it fail
  with an appropriate error from the server.
- Trailing '/' is stripped only when preceded by at least one non-'/'
  character, preserving pure-slash strings for the leading-slash loop.


Permissions
-----------

Owner r/w/x bits are mapped in both directions via OP_CHMOD (protocol v2).
Group and other bits are not mapped; the Amiga server sends fixed group/other
values (055 for directories, 044 for files) regardless of fib_Protection.

  POSIX server -> FUSE client: real Unix mode bits sent and applied correctly.
  nfs_chmod() sends OP_CHMOD to the server (no-op if server_version < 2).
  chown/utimens return success without forwarding (no protocol ops for these).

  Amiga server -> any client: STAT and READDIR derive the owner r/w/x bits
  from fib_Protection (FIBF_READ/WRITE/EXECUTE, active-low). h_chmod()
  converts the received Unix mode back to fib_Protection via SetProtection().

  Any server -> Amiga client: fill_fib() converts the received Unix mode to
  fib_Protection (owner r/w/x only). ACTION_SET_PROTECT sends OP_CHMOD if
  server_version >= 2; falls back to silent success against a v1 server.

The delete bit (FIBF_DELETE) has no Unix equivalent and is not mapped; it
is always left clear (deletable) in both directions.


Timezone offset (TZOFFS)
--------------------------

AmigaDOS datestamps are local time; Unix timestamps are UTC. Without
correction, files show with a time that is offset by the TZ+DST difference.

Both the Amiga client and Amiga server accept a TZOFFS argument (seconds,
signed integer, e.g. 3600 for CET, 7200 for CEST, -18000 for EST). The
convention is: local time = UTC + TZOFFS.

  Amiga server: datestamp_to_unix() subtracts TZOFFS from the Amiga
  local time before sending the Unix timestamp to the client.

  Amiga client: unix_to_ds() adds TZOFFS to the received UTC timestamp
  before converting to an Amiga DateStamp.

Both adjustments must use the same TZOFFS value to round-trip correctly.
The value must be set by the user; there is no automatic TZ detection.

No timezone support exists in the protocol itself; the POSIX server and
FUSE client operate entirely in UTC and are not affected.


Locking
-------

Client-side locking is implemented in the Amiga client behind a compile-time
flag (#define LOCKING in amiga-client.c) and the LOCKING/S mount argument.
When enabled, the client maintains a per-mount lock table and enforces full
AmigaDOS conflict semantics in three directions:

  exact: EXCLUSIVE_LOCK conflicts with any existing claim on the same path;
  SHARED_LOCK conflicts only with an existing exclusive claim.

  prefix: any exclusive lock on a directory blocks all access to paths
  within that subtree.

  descendant: an exclusive lock request is refused if any claim exists on
  any path within the target subtree.

All operations that interact with the lock table: Lock/UnLock, DupLock,
FindInput/FindOutput/FindUpdate, End (close), CreateDir, DeleteFile, Rename.
File opens participate as implicit locks (FINDINPUT = shared,
FINDOUTPUT/FINDUPDATE = exclusive).

Cross-client enforcement (between separate connections to the same server)
is not implemented. The server has no lock table; two clients locking the
same path independently do not conflict. A future server-side implementation
would use an in-memory lock table (no OS-backed locking: AmigaDOS Lock()
has no non-blocking variant; Linux fcntl is a compile-time option at best),
keyed per file-handle token rather than per connection (multiple Amiga
applications share one connection). See TODO for the full design.


Test suite
----------

Two layers:

  run.lua (Lua, POSIX): integration tests that drive fizz-serve and
  fizz-mount end-to-end. 22 tests. FreeBSD note: run.lua uses
  /proc/mounts for mount detection (Linux-only); the local-server test
  mode is affected on FreeBSD.

  fizz-test.c (C99, portable): protocol-level client test program that
  compiles and runs on both POSIX and AmigaOS. 11 tests covering write,
  read, seek, truncate, rename, delete, mkdir, large file. Completes in
  under 30 seconds on a real Amiga.

