Part 5: Anatomy of a Tunnel & Workflow

This part maps the contracts from Parts 1-4 onto the actual files you edit. Once you know which file owns which responsibility, adding or fixing a tunnel becomes mechanical: make the smallest complete root-cause change in the right place and keep the lifecycle intact.

Directory Layout

Tunnel files use the following layout. Callback files and their directories exist only where the node implements behavior beyond its constructor defaults:

tunnels/MyTunnel/
  CMakeLists.txt
  description.md
  include/MyTunnel/
    interface.h          # public WW_EXPORT API + custom callback prototypes
    structure.h          # tstate / lstate structs, size enums, helpers
  instance/
    create.c             # tunnelCreate(), assign callbacks, parse + validate settings
    node.c               # node metadata: type, flags, layer group, padding, createHandle
    chain.c              # custom onChain hook, when needed
    index.c              # custom onIndex hook, when needed
    prepair.c            # onPrepare hook
    start.c              # onStart hook
    stop.c               # lifecycle-v2 quiesce, drain, and stop hooks
    destroy.c            # onDestroy hook: free state, then tunnelDestroy()
    api.c                # runtime API entry (tunnelApi)
  common/
    helpers.c            # shared protocol helpers
    flow.c               # shared progress coordination, when needed
    input.c              # shared parsing/extraction, when needed
    line_state.c         # initialize + destroy per-line state
  upstream/
    init.c  est.c  payload.c  pause.c  resume.c  fin.c
  downstream/
    init.c  est.c  payload.c  pause.c  resume.c  fin.c

Real layout vs. the old summary

Two details matter in practice: every tunnel has an instance/api.c, and the prepare hook file, when present, is spelled prepair.c (the function is ...OnPrepair). The upstream/downstream finish files are fin.c, not finish.c.

Keep inherited callbacks at their defaults; do not add forwarding wrappers or empty lifecycle files for them. Compare against the actual constructor: packet payload defaults and disabled adapter directions intentionally abort. tunnelCreate() also terminates on upstream Est and downstream Init by default; nodes that support these events must explicitly install handlers. packettunnelCreate() preserves lifecycle forwarding in both directions. Other ordinary flow defaults forward. State changes, event absorption, cleanup, and error handling require custom callbacks even when their bodies are short.

For an ordinary middle node, a custom Est handler must still promptly relay adapter establishment. Protocol negotiation, request emission and payload readiness belong in separate state, not in a delayed or replacement Est. Follow Part 2's Est contract when mapping distinct lines or adding a protocol-completion callback.

Choose flow state by role, not by copying another protocol's pump. Synchronous transforms use direct Payload and signal forwarding without redundant output queues. Ordinary middle nodes inherit transparent Pause/Resume forwarding, including repeated notifications; timers and deferred backlogs do not require a consumer-pause gate. Parsers keep incomplete input, and every backlog preserves FIFO and its own hard limits. Adapters, exact-line heads/tails, SpeedLimit and bandwidth-enabled UserController retain their flow-control responsibilities. Keep any reentry guard attached to the specific ordering obligation it protects; see Part 2.

Remove obsolete callback implementations when they have no registrations or runtime callers, together with their declarations and build-source references. Check public entrypoints and dynamic loading conventions before declaring a function unused; retain callbacks and APIs that remain reachable.

Keep one file per custom {direction}/{event}. Each callback may contain its direction-specific dispatch, admission checks and direct forwarding so buffer ownership and callback direction remain visible. Share parsing, framing, progress coordination and teardown in cohesive common/ files, such as input.c, flow.c, fallback.c or lifecycle.c, as their responsibilities warrant. Keep file-local helpers static and expose only necessary cross-file helpers through the private header. Callbacks should remain readable without hiding directional decisions behind a boolean dispatcher; common code need not all live in helpers.c.

QuicClient and QuicServer keep their constructors, lifecycle hooks and directional callbacks in this layout. Each node selects application or carrier input explicitly and maps generated wire data, decoded application data and application pressure in common/output.c. Internals/QuicCommon shares configuration validation, framing and the protocol engine, with its C runtime split into context/lifetime, input, flow and timer modules. The shared driver retains the borrowed line and defers engine destruction across reentrant callbacks; node callbacks keep Init and Finish forwarding visible in their own direction.

Most production presets enable unity builds, combining a target's C source files into one translation unit. Give file-local functions, types and objects names that remain distinct within that target; static does not isolate a name from another included source file. Check the target in unity mode when adding or extracting helpers, as ordinary separate-file and native-unit builds do not expose these collisions.

What Each File Owns

File	Responsibility
`instance/node.c`	Returns the `node_t` describing this tunnel type: `type`, `version`, `createHandle`, `flags`, `required_padding_left`, `layer_group`, and neighbor layer constraints.
`instance/create.c`	`tunnelCreate(node, sizeof(tstate), sizeof(lstate))`, assign only the flow callbacks and lifecycle hooks that differ from the constructor defaults, parse and validate JSON settings, allocate per-worker resources, clean up on any failure.
`instance/chain.c` / `index.c`	The `onChain` / `onIndex` hooks. Most tunnels keep the framework defaults. Static internal topology known before solving belongs in `onChain`. A tunnel whose private topology depends on solved adjacent layers may also set optional `onSolvedTopology` from `create.c`. It receives the current solved snapshot, which is not necessarily final. The hook may be called repeatedly and returns `true` if and only if it changed topology; the snapshot then becomes stale and the manager solves again before invoking another hook. Once a complete pass reports no changes, `onIndex` runs once over immutable topology and may consume the final solved layers.
`instance/prepair.c` / `start.c` / `stop.c`	`onPrepare` / `onStart` and the lifecycle-v2 shutdown hooks. Bootstrap in `onStart`; close admission in `onQuiesceRequest`; detach worker-affine work in `onWorkerQuiesce`; wait for external producers in `onQuiesceWait`; drain owned lines in `onWorkerStop`; release remaining shared resources in `onStop`.
`instance/destroy.c`	`onDestroy`: free settings, SSL contexts, pools, thread-local arrays, then call `tunnelDestroy(t)`.
`instance/api.c`	The `tunnelApi` runtime entry. It receives an `sbuf_t` message, must recycle it, and returns an `api_result_t`.
`common/line_state.c`	`LinestateInitialize` (called from `Init`) and `LinestateDestroy` (zeroes the aligned line-state region).
`common/*.c`	Shared algorithms and coordination grouped by responsibility: protocol helpers, parsing, framing, progress, fallback or lifecycle.
`upstream/.c`, `downstream/.c`	One custom flow callback each. Omit files for inherited callbacks. Keep directional dispatch, admission and direct forwarding here; delegate shared algorithms and coordination to cohesive common files.
`include/MyTunnel/structure.h`	`mytunnel_tstate_t`, `mytunnel_lstate_t`, size enums, and small inline helpers.
`include/MyTunnel/interface.h`	`WW_EXPORT` create/api and custom destroy prototypes, plus prototypes for implemented callbacks.

wwapi.h provides all ww headers needed by nodes; use it instead of individual ww headers, directly or through the node's own interface/structure headers. Node implementations still include their intended logger-selection header explicitly. Do not include logger-selection headers in ww library .h files: that would select the logger for consuming nodes and override their intended selection.

Caller preconditions and failure handling

Callers establish an internal helper's preconditions. A required tunnel, line, buffer or output pointer is not an optional input merely because C can represent NULL. Do not turn a programmer error into an ordinary false, NULL, success or silent return, or duplicate argument validation at every layer.

Choose the check according to what can actually cause the condition:

Condition	Handling
Caller-owned precondition or programmer-only invariant	Use `assert()` to document and check it in Debug, or remove a redundant check when the surrounding contract already establishes it.
Critical internal invariant requiring release-build protection, such as corrupted resource ownership	Keep a runtime check, emit `LOGF`, then call `abortProgramNow()`. Do not disguise corruption as a recoverable protocol failure or request orderly shutdown and continue.
WaterWall memory-manager heap allocation failure	The memory manager calls `abortProgramNow(1)` without normal logging or orderly cleanup; see heap allocation failure.
Expected failure, external input or supported state	Keep ordinary runtime validation and the documented failure/cleanup path. Examples include unrepresentable sizes, I/O refusal, malformed peer data, exhausted budgets or fixed resource inventories, incomplete frames and optional NULL arguments.

Trace callers before classifying a condition. A NULL callback context after detachment, a stale association, a canceled timer, or state released during a reentrant callback can be expected. Keep those guards and exact-line lifetime checks. A reference preserves the allocation, not the logical life of the line. An entry assertion never replaces a required check after calling another tunnel. Required internal arguments normally belong in the first row; the possibility that caller misuse could crash does not by itself justify another runtime guard.

Split mixed checks instead of deleting the whole expression. For example, a TLS output helper can assert that its tunnel, line, state and write BIO are non-null, while still returning failure if TLS output fails or a callback releases its state. Document nullable parameters explicitly. Keep operations with side effects outside assert() so Release builds perform them too; assertions disappear under NDEBUG.

Avoiding redundant checks and state

A check is redundant only when an established precondition, earlier validation or API postcondition guarantees it on every path reaching that check, and intervening operations cannot invalidate that guarantee. Trace error paths, reentry and teardown as well as ordinary progress. A branch being rare or absent from tests does not make it impossible.

Use a helper's actual guarantees. For example, addresscontextIsDomain() already establishes a non-null domain and nonzero stored length; it does not validate protocol-specific hostname syntax or agreement with the C-string length. Keep checks for properties the helper does not establish.
Preserve a shared checked API's documented invalid-input, incomplete-input and capacity-refusal outcomes even when one caller prevalidates its input. A local cleanup may narrow an internal helper's contract only when its complete caller set establishes the preconditions. Document a non-obvious size bound or ordering guarantee where future callers need it.
Store independent facts rather than flags that merely duplicate a phase or another flag. Check every assignment and transition for the exact line role before removing state; similar client and server implementations need not have identical reachable states. Whether a neighbor received Init, for example, remains a separate fact when local setup can fail before dispatch.

When a redundant branch is removed, also remove private fields or helper results that lose their last purpose. Use the assertion policy above for programmer-only invariants; retain real input, admission and callback-lifetime failures. For allocation postconditions and incoming-buffer checks, see Part 3.

Heap allocation failure

WaterWall's memory manager does not support recovery from heap-allocation failure. A valid heap allocation returns usable storage or terminates the process at that boundary with abortProgramNow(1). This applies to initial allocation, growth and duplication, including startup and optional facilities. Do not add nullable OOM APIs, caller-side OOM checks, retries, fallback modes or per-line cleanup solely to recover from heap exhaustion.

Route runtime-owned heap storage through memoryAllocate*(), memoryCalloc(), memoryReAllocate() and the matching memoryFree*() APIs. The normal backend is mimalloc; do not bypass this boundary with CRT allocation to obtain a recoverable OOM result. This includes lwIP engine control storage, worker-engine slots and caller-owned Windows adapter-query buffers. Storage supplied by an OS or third-party API still uses that API's matching release function. WaterWall-owned TLS ex-data, session-cache records and custom BIO state use WaterWall allocation and matching frees in their destructor callbacks; attaching them to an SSL/BIO object does not transfer their allocation policy to the TLS library. Keep failure handling for the actual SSL/BIO objects and ex-data registration operations.

Runtime executables replace all 20 global C++17 allocation/deallocation overloads: scalar and array new/delete, including nothrow, sized-delete and aligned forms. Ordinary storage uses memoryAllocate()/memoryFree(); aligned storage uses memoryAllocateAligned()/memoryFreeAligned(). This covers QUICHE and linked C++ containers and preserves the selected mimalloc or ALLOCATOR_BYPASS backend. Heap exhaustion remains fatal even for nothrow allocation. Aligned geometry refusal remains distinct: throwing forms raise std::bad_alloc, while nothrow forms return nullptr. Constructor-failure cleanup uses the matching allocator. Direct C allocation calls and class-specific allocators retain their own contracts; global replacement does not intercept them. The standalone launcher and build-time host tools do not acquire the runtime override.

TcpOverUdp's shared FEC implementation retains its private C++ allocator for objects, nested vectors and shared-pointer control blocks; it also routes that storage through WaterWall allocation. Its C-facing boundary terminates immediately on std::bad_alloc, while invalid geometry, malformed shards and refused output remain ordinary failures. Preserve that distinction when changing its exception handling.

Tunnel-local helpers must not expose failure results or rollback solely for WaterWall heap exhaustion. Fixed, representable buffer-queue initialization and credential copying are infallible. Helpers that also validate geometry, admit bounded work or create native resources remain fallible for those reasons. Tests must not simulate recoverable OOM by returning NULL above the allocator gate; exercise actual admission/resource refusal there and test fatal heap failure below the gate.

Runtime cJSON uses the same fatal allocator. Creating an object or array and adding fresh items with valid keys and values needs no OOM rollback. Keep parse, serialization-size and invalid-argument failures; a nullable JSON result is not necessarily an allocation failure. Likewise, inserting an absent key into an already-reserved map needs no allocation recovery when admission proves it fits. Keep duplicate-key and capacity refusals wherever callers can reach them.

The standalone launcher does not link the runtime allocator. Shared configuration validation follows the existing WW_CJSON_CRT_ALLOCATOR build selection: runtime builds use WaterWall allocation, while standalone launcher builds retain CRT allocation and their recoverable failure result. Do not link the runtime into the launcher solely to share its allocator.

Shared integer JSON readers use cJSON's exact signed-integer provenance. Parsing accepts plain integer tokens through INT64_MAX without a double round trip; decimal and exponent spellings remain usable floats but are rejected by integer readers, including nonzero fractional underflow. Unsigned decimal strings extend through UINT64_MAX. Use cJSON_CreateInt64() / cJSON_SetInt64Value() or jsonAddUint64ToObject() for exact integer creation. Number setters validate their supplied double, duplication preserves validity, and direct field writes cannot establish it. Parser and standalone builds share this representation.

For fields that require exact C strings, use jsonObjectFieldStringsAreExact(object, key) before consuming them. It checks every matching member, including duplicates, for embedded NULs in its name or string value and for invalid UTF-8. cJSON retains decoded-NUL evidence through duplication and references; replacing a name or value through its setter clears only that string's evidence. This check is opt-in: presence, type, ranges and recognized setting names remain the consumer's responsibility. QUIC applies it in Internals/QuicCommon/config.c; the generic config loader preserves malformed raw variable values during substitution without knowing QUIC's setting names. Serialization does not preserve decoded-NUL evidence, so validate before printing and reparsing a tree.

Zero-sized allocation/calloc requests return at least one freeable byte. memoryReAllocate(ptr, 0) instead frees ptr and returns NULL. Invalid aligned allocation geometry and overflowing memoryCalloc() multiplication return NULL before the backend allocator is called. OPENSSL_memory_alloc() likewise returns NULL before allocation when adding its bookkeeping header would overflow. Preserve checks for these documented results when the caller can supply such requests.

The OOM path must not use the normal logger, allocate more memory, acquire application locks or call cleanup callbacks. It does not request orderly shutdown: requestProgramShutdown() returns to its caller, while an allocation caller cannot continue without usable storage. abortProgramNow() immediately terminates through the independent fatal path and deliberately skips orderly cleanup; see Part 7.

Validate external sizes, integer arithmetic and configured admission limits before allocating. Keep documented refusal for unrepresentable geometry, exhausted byte/entry budgets, empty fixed inventories, OS operations and third-party or standalone allocator interfaces that retain their own contracts. Keep optional-null and explicitly free-only operation results. Those results do not imply that a WaterWall heap allocator can return after OOM. Split checks with mixed causes and preserve ownership on every supported refusal; removing allocation recovery never authorizes removing bounds or lifetime checks.

Typed internal capabilities versus runtime messages

Use include/<Tunnel>/interface.h for a typed capability exchanged directly between compatible tunnels already known to share a chain. For example, UdpListener exposes a dynamic UDP endpoint provider as a value-handle API: the caller opens, activates, queries, and closes an endpoint on its validated owner worker. The capability must define ownership, worker affinity, stale-handle behavior, and callback routing; it must not expose a usable foreign line_t *.

Do not turn such a local capability into instance/api.c. That file is the external runtime message entry: it accepts an sbuf_t, recycles it, and returns an api_result_t. It has different admission, serialization, and lifecycle requirements. A typed internal capability remains ordinary local C calls and must still preserve the normal tunnel callback chain.

Address-context metadata ownership

address_context_t.optional_flags.detected_protocols carries two logical classes of optional metadata. Router/sniffer-owned bits record observed application protocols; route-control bits carry a one-shot internal instruction such as an already-negotiated SOCKS UDP relay endpoint. A Router starting a new sniffing window must call addresscontextClearDetectedProtocols(), which clears only observed protocol bits. addresscontextClearOptionalFlags() is the full reset for a caller that genuinely replaces the context. The designated outbound adapter consumes route-control metadata exactly once and clears it after setup; rules must never treat route-control bits as sniffed protocols.

Framework-owned worker messages, timers, WIO callbacks, and async DNS requests settle before component destruction under lifecycle-v2, so those contexts may carry tunnel_t * directly when they also validate and use their explicit owner worker. This is not a general lifetime rule: a callback root outside the worker lifecycle, such as a device reader thread, must close local admission and wait for active operations through an owner-specific mechanism such as quiescence_gate_t before the tunnel or device state is reclaimed.

`node.c`: Declaring the Node

node.c is where a tunnel announces itself to the runtime. Here is TcpListener's, which is a good template for an adapter:

node_t nodeTcpListenerGet(void)
{
    const char *type_name = "TcpListener";
    node_t node = {
        .type                  = stringDuplicate(type_name),
        .hash_type             = calcHashBytes(type_name, stringLength(type_name)),
        .version               = 0001,
        .createHandle          = tcplistenerTunnelCreate,
        .flags                 = kNodeFlagChainHead,
        .required_padding_left = 0,
        .layer_group           = kNodeLayer4,
        .layer_group_next_node = kNodeLayerAnything,
        .layer_group_prev_node = kNodeLayerAnything,
        .can_have_next         = true,
        .can_have_prev         = true,
    };
    return node;
}

The fields you will set most often (all defined in ww/objects/node.h):

Field	Meaning
`type` / `hash_type`	The string used in JSON `"type"`, plus its precomputed hash.
`createHandle`	The `TunnelCreateHandle` that builds the instance.
`flags`	One or more `node_flags`: `kNodeFlagChainHead`, `kNodeFlagChainEnd`, `kNodeFlagNoChain`, `kNodeFlagSingleton`, `kNodeFlagSupportsSplice`, `kNodeFlagBlocksSplice`.
`required_padding_left`	Left-padding budget this tunnel may prepend into (see Part 3).
`layer_group`	The layer category this node operates on: `kNodeLayer3` (packet), `kNodeLayer4` (stream), `kNodeLayerAnything` (supports both), or `kNodeLayerNone`. (Note: `kNodeLayerSameAsNext`, `kNodeLayerSameAsPrev`, `kNodeLayerOppositeNext`, and `kNodeLayerOppositePrev` are forbidden for `layer_group` and only allowed in `layer_group_next_node` / `layer_group_prev_node`).
`layer_group_next_node` / `layer_group_prev_node`	Required layer group for next/previous neighbor (`kNodeLayer3`, `kNodeLayer4`, `kNodeLayerAnything`, `kNodeLayerSameAsNext`, `kNodeLayerSameAsPrev`, `kNodeLayerOppositeNext`, `kNodeLayerOppositePrev`, or `kNodeLayerNone`). Note: Opposite flags must be combined with a base layer group (e.g. `kNodeLayerAnything
`can_have_next` / `can_have_prev`	Whether a neighbor is permitted on each side (`true`/`false`).

Layer Groups & Strict Topology Validation

During chain startup, NodeManager strictly enforces node capabilities and layer rules via validateTunnelChains and the bounded monotonic layer constraint solver (nodeLayerSolveChain):

Capability constraints: If a node specifies .can_have_next = false (or .can_have_prev = false), having an adjacent neighbor on that side results in immediate startup failure. Unchained nodes marked with kNodeFlagNoChain must not have adjacent neighbors.
Metadata shape validation: layer_group must be concrete (kNodeLayerNone, kNodeLayer3, kNodeLayer4, kNodeLayerAnything). Forward relative flags (kNodeLayerSameAsNext, kNodeLayerOppositeNext) are allowed only in layer_group_prev_node, and backward relative flags (kNodeLayerSameAsPrev, kNodeLayerOppositePrev) are allowed only in layer_group_next_node. Opposite flags must always include a base layer group (e.g. kNodeLayerAnything | kNodeLayerOppositePrev).
Edge-domain constraint propagation: The runtime constructs a directed edge graph representing callback connections between adjacent tunnels, seeds each edge domain ({L3}, {L4}, or {L3, L4}) from participating node and neighbor layer requirements, and propagates SameAs and Opposite relations monotonically to a fixed point. Any edge domain converging to {} immediately fails startup with a clear diagnostic identifying both nodes and sides.
Conditional SameAs vs Strict Opposite: SameAs relative constraints (kNodeLayerSameAsPrev, kNodeLayerSameAsNext) are conditional: when the referenced side is absent, no equality relation is added, and the node's placement flags (kNodeFlagChainHead, kNodeFlagChainEnd, kNodeFlagNoChain) determine whether boundary placement is permitted. In contrast, Opposite constraints (kNodeLayerOppositePrev, kNodeLayerOppositeNext) are strict and always require both participating sides to be present.
State-held cross-tunnel relations: For nodes whose callback targets are established dynamically at runtime (such as paired Bridge tunnels), tunnels register logical layer relations directly on the chain via tunnelchainRegisterLayerRelation(). Registered relations couple the participating edge domains (e.g. equating Bridge A prev edge with Bridge B prev edge, and Bridge A next edge with Bridge B next edge) during solver execution and persist across chain combinations (tunnelchainCombine()).
Fixed and relative bridges: Fixed-side bridges explicitly declare concrete layer requirements on their adjacent edges rather than claiming kNodeLayerAnything. ConnectionToPackets is L4 on prev and L3 on next; PacketsToConnection is L3 on prev and L4 on next; StreamToPackets is L4 on prev and L3 on next. Dynamic bidirectional bridges such as WireGuardDevice declare Opposite relations that couple opposite edge domains without prematurely collapsing unresolved intermediate subchains.
Packet chain classification & bootstrap: A chain allocates worker packet lines and sets contains_packet_node = true only when at least one edge is forced to {L3}, a node or edge requirement is fixed to L3, or an Opposite relation exists. Persistent packet line Init messages are queued at runtime only when the chain head's next edge is resolved to {L3}.
Failure semantics: Any validation or solver failure logs a fatal diagnostic (LOGF) and records a startup failure (startupFailureRecord(1)), causing WaterWall to reject the configuration and perform a clean startup rollback.

Bridge and reverse-node adjacency

The reverse topology requires both direct adjacencies:

ReverseServer -> Bridge: Bridge must be ReverseServer's immediate next node.
Bridge -> ReverseClient: Bridge must be ReverseClient's immediate previous node.

No node may intervene in either pair, including a transparent transform, logger, or limiter. Both Bridge -> SpeedLimit -> ReverseClient and ReverseServer -> SpeedLimit -> Bridge are forbidden. ReverseClient creates a line toward its previous side using downstream Init, and Bridge translates that callback into upstream Init on its paired branch. These are chain-design requirements that users must follow. The code does not automatically enforce either adjacency rule. The direct connection keeps the exceptional direction change at the reverse/Bridge boundary; ordinary stream nodes outside it retain their usual left-owner topology assumptions. See Part 2 for the scope of those assumptions and the corresponding Finish order.

Node Flags

UdpStatelessSocket is a chain-head adapter by design. It requires next, forbids prev, and creates normal peer lines using upstream Init only. Outgoing datagrams and callbacks from the chain arrive downstream from next. Tail placement and downstream peer initialization are intentionally disabled; this is the supported topology contract, not a temporary workaround. WireGuard layouts using this socket place its transport side toward prev. No runtime role flag is needed; upstream callbacks retain the adapter's disabled defaults. The idle handle distinguishes owned peer lines from borrowed transport or packet lines, whose downstream lifecycle callbacks have no peer state to manage.

Flag	Meaning
`kNodeFlagNone`	No node capabilities selected; use alone.
`kNodeFlagChainHead`	May start a chain (an ingress adapter).
`kNodeFlagChainEnd`	May end a chain (an egress adapter).
`kNodeFlagNoChain`	Works without being in a chain (e.g. a database/user-auth node).
`kNodeFlagSingleton`	Only one instance allowed in the whole config.
`kNodeFlagSupportsSplice`	Advertises support for splice payloads.
`kNodeFlagBlocksSplice`	Explicitly disables splice for the entire chain; takes precedence over `kNodeFlagSupportsSplice`.

Use kNodeFlagNone only when no other node flags apply. When capabilities are selected, combine those flags without adding kNodeFlagNone.

tunnel_chain_t.supports_splice is false during assembly. At tunnelchainFinalize(), it becomes true only when WW_HAVE_SPLICE == 1, misc.splice is not disabled, the solved chain has no packet nodes (contains_packet_node == false), and the chain is nonempty with every tunnel's node advertising kNodeFlagSupportsSplice and none advertising kNodeFlagBlocksSplice. Packet chains and builds without splice support always leave the chain flag false, regardless of node flags. The node check includes internal helpers and tunnels brought in by chain merges. Finalization follows the complete onChain phase and the solve/onSolvedTopology expansion loop; onChain alone is too early because topology-dependent helpers can still join.

kNodeFlagSupportsSplice is a two-direction safety promise, not a per-callback converter: every reachable payload, retention, error, and close path must accept splice input safely, and the node must also accept ordinary fallback input. Advertising the flag does not promise that every path remains zero-copy.

kNodeFlagBlocksSplice records an explicit node-level exclusion. It wins even if both splice flags are set on a node. A node with neither flag still prevents splice because support must be explicitly advertised. The blocker applies to the final expanded/merged topology, including internal helpers, and does not depend on which runtime mode or branch a particular connection uses.

TlsClient, TlsServer, EncryptionClient, EncryptionServer, RealityClient, RealityServer, ObfuscatorClient, ObfuscatorServer, Disturber, TcpOverUdpClient, TcpOverUdpServer, HttpClient, and HttpServer explicitly block splice. Internal node copies must preserve these capability flags, including RealityClient's embedded TlsClient.

The chain flag applies build/startup/topology restrictions. Final eligibility requires WW_HAVE_SPLICE == 1, misc.splice not disabled, no packet node, a nonempty finalized chain, support on every tunnel and no explicit blocker after helper/topology expansion. Eligibility alone does not activate splice I/O; the adapters still select it. WW_HAVE_SPLICE reports Linux splice API availability independently of the event watcher.

TcpListener, TcpConnector, DomainResolver, MuxClient, MuxServer, Bgp4Client, Bgp4Server, HttpProxyServer, VlessClient, VlessServer, TrojanClient, TrojanServer, HeaderClient, Bridge, UserController, BlackHole, ReverseClient, and JunkDatagramSender, UdpListener, UdpConnector, UdpStatelessSocket, TcpUdpListener, TcpUdpConnector, Router, SniffRouter, ReverseServer, Socks5Client, Socks5Server, HeaderServer, KeepAliveClient, KeepAliveServer, SoftIpLimiter, SpeedLimit, UdpOverTcpClient, and UdpOverTcpServer, ConnectionFisherClient, and ConnectionFisherServer advertise the flag. UdpOverTcp preserves exact framed bodies as UDP datagrams over a TCP carrier and leaves UDP socket fallback to the sender. SoftIpLimiter materializes only during initial identity inspection; SpeedLimit preserves private pipes in its bounded queues and splits exact token grants with ordinary fallback. HeaderServer materializes only while reading its initial header; KeepAlive uses the fixed-header splice stream for ongoing framing. See Part 3 for their ownership, reentry and fallback contracts. This includes the internal DomainResolver inserted by TcpConnector or Router's resolve-domains setting. Router and SniffRouter materialize complete deliveries while route selection is undecided or nested input waits behind startup replay, then forward opaque buffers in both directions. Eligible next-path routes permanently retire after replay; alternate branches retain routing state. See route retirement. Every configured route branch and helper remains part of the whole-chain capability check, so an unselected blocking branch also disables splice. See Part 3 for their ordinary pending storage and exact replay contract. Mux retains valid queued bodies under local queue-capacity charges. Incoming assembly shares the parent receive limit with attached child queues; beneficial pressure compaction and actual pipe-transfer fallback remain available. Its fixed-header stream and atomic large splice batches preserve both callback directions. This does not promise zero-copy for every frame. BufferStream and HTTP parsers require ordinary memory. HttpClient and HttpServer explicitly block splice. HttpProxyServer materializes at admission except for ready, unpaused CONNECT/fallback whole-buffer relay; its session slots remain ordinary in both directions (see Part 3).

ConnectionFisher accepts splice-backed startup input through whole-delivery materialization before ordinary parsing, then preserves opaque bodies in application FIFOs and ready forwarding. Its fixed probe marker is separate from the following application-data budget. Handshake streams and empty setup queues retire after setup/drain; client main/child mapping and Finish ownership remain active. The server permanently bypasses after next Init and replay settle, with its full line state already destroyed. See Parts 2 and 3.

HalfDuplexClient and HalfDuplexServer also advertise splice support. The client requires 17 bytes of left padding; the server requires zero. Client first-intro construction reuses ordinary buffers when headroom permits and materializes splice input. Server parsing/waiting storage materializes complete deliveries; their direct paths preserve wrappers. The server's PipeTunnel shares the child's node metadata and participates in the expanded-chain capability check. This capability does not promise zero-copy setup, change pairing or waiting limits, or add read preferences. See Part 3.

Socks5Client and Socks5Server keep negotiation input ordinary, including nested parser admission, and preserve opaque TCP bodies after setup. Their UDP paths fully materialize datagrams and request ordinary reads on each associated line. Client local DNS and server authenticated UserController helpers retain their splice flags; AuthenticationClient lookup is outside the payload chain. See Part 3 for the exact read-preference timing and unchanged protocol/backlog bounds.

TrojanClient supports opaque TCP and staged variable-header UDP decoding, including its dependent owned carrier and destination-context selection. Its internal DomainResolver remains eligible. TlsClient still blocks a chain containing it; TrojanServer supports staged request authentication, opaque CONNECT, UDP backends and exact fallback replay. Eligibility includes its UserController and all reachable fallback/connector helpers; the AuthenticationClient lookup dependency is not a payload hop. TlsServer remains a blocker. See the Trojan buffer contracts in Part 3.

VlessClient supports both callback directions for TCP, UDP and destination-context selection, including retained input and dependent carrier cleanup. Its internal DomainResolver remains eligible. The response/addon barrier and UDP length parser use node-local bounded metadata storage; ordinary fallback remains valid. VlessServer supports staged request authentication, opaque TCP, fixed-destination UDP and exact fallback replay. Its expanded chain includes UserController and fallback helpers; AuthenticationClient is a lookup dependency. TlsClient and TlsServer remain chain-wide blockers. See Part 3 for exact retention limits and representation handling.

VmessClient and VmessServer advertise kNodeFlagBlocksSplice with zero required left padding. Their shared codec lives in Internals/VmessCommon; neither endpoint depends on the other. Client construction prepares one immutable UUID-derived key, validates exact setting types and unambiguous aliases, and shares command/framing validation with the codec. It is an ordinary L4 middle node supporting prev and next. Its owned DomainResolver is inserted before it when local resolution is requested; transport composition remains configured through the existing TLS/HTTP nodes. Client-only construction has no AuthenticationClient, UserController, replay table, credential vector or authentication pump. Persistent codecs remain active even for NONE/zero; see Parts 2 and 3 for bootstrap, ownership and ordinary-buffer bounds.

VMess credential normalization is private to VmessCommon. Preserve strict dashed or compact UUID parsing; exact nonempty UTF-8 IDs of at most 30 bytes use UUIDv5 with the nil namespace and provider SHA-1. Do not trim, case-fold arbitrary IDs, normalize Unicode or accept malformed UUID-sized strings as short IDs. Local server duplicate admission compares normalized UUID bytes. Managed AuthenticationClient passwords remain canonical UUIDs; do not scan raw passwords or alter shared UUID indexes.

Client security=auto resolves once at construction using proven runtime CPU/OS AES/carryless-multiply capabilities and direct-provider cipher availability, with a conservative ChaCha preference when acceleration is unproved. Explicit unavailable ciphers fail clearly; never infer TLS or automatically select NONE/zero. Server allowed-security is a nonempty unique strict list of actual body modes; reject an allow-list with no usable mode for its enabled commands. New options retain exact JSON type/encoding/duplicate validation and introduce no negotiation bits.

Both endpoints register the read-only stats operation at construction and unregister after admitted callbacks settle, before target state is reclaimed. The historical TunnelApi callback alone does not expose it. Accept only a bounded empty JSON object; return fixed-schema effective configuration and aggregate counters, preserving uint64 precision as decimal strings. Load synchronized published counters; never traverse live lines or owner-only destination tables on the API caller's worker. Server auth/ cache/replay occupancy uses bounded nonblocking snapshots and returns busy on lock contention. Respect deadline, working-memory and actual result-pool charges. Snapshots are approximate during traffic. Application counters exclude packetaddr envelopes; wire counters include protocol framing. Add no reset/mutation operation or stats scheduler. Verbose diagnostics use bounded per-node sampling and fixed reason categories, without credentials, keys, plaintext or per-user/per-destination metric labels.

BGP supports both representations through in-place framing, bounded frame FIFOs and a 19-byte fixed-header receive stream. OPEN inspection reads only its bounded metadata; private-pipe creation or transfer refusal selects ordinary fallback. The fixed per-direction retention policy is independent of pool sizing.

HeaderClient prepends its one-time header only in real left padding, preserving existing prefixes and the private-pipe body. Bridge transfers the original buffer and line through its paired callback mapping; eligibility includes the merged chain. UserController preserves opaque payloads and accounts logical bytes, including prefixes, preserving reverse-direction accounting. Its shared shaping machinery also serves SpeedLimit. Finite user rates retain bounded FIFOs, split streams through exact range moves and preserve datagrams whole. Internal creators bind their local payload-mode resolver immediately after construction; manual controllers require an explicit payload-mode. See Parts 2 and 3 for classification, delivery commits and resource bounds. Rejected traffic and BlackHole input recycle through the line's pool, discarding unread pipe bytes; healthy empty pipes may remain cached. BlackHole's active Init still closes the line and does not admit payload afterward.

ReverseClient forwards application buffers between its owned normal lines and recycles held input if activation closes the pair. Its handshake and JunkDatagramSender's generated junk/delayed duplicates stay ordinary; forwarded application bodies can stay in private pipes. Existing callback directions, ordering, lifecycle, and direct Bridge -> ReverseClient adjacency remain required. ReverseServer materializes complete deliveries in both directions until pairing is complete; handshake inspection and waiting/replay storage remain ordinary. After pairing it forwards opaque application buffers, including through its PipeTunnel wrapper, which shares the child's capability flags. Waiting limits and direct ReverseServer -> Bridge adjacency remain required. HeaderClient support does not imply HeaderServer support. Ordinary fallback stays valid; packet configurations of dual-layer nodes still disable splice globally. These capabilities make no performance promise without measurement.

UDP capability covers sending existing private-pipe bodies; UDP receives stay ordinary. Combined TCP/UDP wrappers retain both children's flags and include both branches plus inserted helpers in final eligibility, even for a TCP line. An eligible TCP branch can therefore use its existing splice I/O without changing protocol selection. Datagram failure and full fallback follow Part 3; packet configurations remain excluded globally.

External-node ABI

External libraries must export waterwallNodeLifecycleAbiVersion() returning WW_EXTERNAL_NODE_ABI_VERSION, currently 7. The loader rejects missing or mismatched descriptors before resolving or invoking a node getter. Rebuild external libraries against the current headers: packet checksum requests use sbufSetRecalculateChecksum() / sbufGetRecalculateChecksum(), and line_t no longer contains that state or its old inline accessors. socket_filter_option_t is passed by value and no longer includes shared_balance_table. Balancing state belongs to private SocketManager filters, not the public options structure. WW_EXTERNAL_NODE_LIFECYCLE_ABI_VERSION remains an alias for this complete gate; the lifecycle callback semantics themselves are unchanged.

`create.c`: Building the Instance

create.c is the only place a tunnel is allocated. The pattern, distilled from TlsClient:

tunnel_t *t = tunnelCreate(node, sizeof(tstate), sizeof(lstate));
Check t immediately and return NULL if creation failed.
Assign only callbacks whose behavior differs from the chosen constructor defaults.
Read settings from node->node_settings_json, validating each one.
Allocate per-worker resources (e.g. one SSL context per worker).
On any later failure, clean up and bail out — destroy partial state and the tunnel, return NULL.

tunnelCreate(), adapterCreate(), and packettunnelCreate() are nullable by contract. They return NULL on invalid or overflowing state sizes. Heap allocation follows the fatal policy. Never assign a callback or call tunnelGetState() until the result has been checked.

The cleanup discipline is real and worth following. The distilled shape is:

tunnel_t *tlsclientTunnelCreate(node_t *node)
{
    tunnel_t *t = tunnelCreate(node, sizeof(tlsclient_tstate_t), sizeof(tlsclient_lstate_t));
    if (! t)
    {
        return NULL;
    }

    configureTunnelCallbacks(t);

    tlsclient_tstate_t *ts       = tunnelGetState(t);
    const cJSON        *settings = node->node_settings_json;

    if (! getandvalidateSniSetting(ts, settings))
    {
        goto fail;
    }
    // ... more validated settings ...

    if (! createSslContextPool(&ts->threadlocal_ssl_contexts, /* ... */))
    {
        goto fail;
    }

    return t;

fail:
    tlsclientTunnelstateDestroy(ts);
    tunnelDestroy(t);
    return NULL;
}

Inserting an internal helper before a tunnel

During onChain, use tunnelInsertBefore(target, inserted, chain) to place an already-created helper between a tunnel and its previous neighbour. Both tunnels must be distinct and unchained, the target must have a previous neighbour, the helper must have no neighbours, and the chain must still be mutable. These are caller preconditions, checked with Debug assertions. An already-pending startup failure returns false without changing the chain or links.

The helper first registers inserted in the chain. A chain-capacity or aggregate size failure records a startup failure and returns false without changing the neighbour links; the caller must return. On success it connects previous -> inserted -> target. It changes previous->next only when that link previously pointed to target, preserving a branch owner's separate default route. Branch callers obtain the callable entry with tunnelGetBranchEntry().

The operation neither registers target nor calls either tunnel's onChain. The caller still handles optional helpers, deferral until a previous neighbour exists, configured-next validation, its own insertion and onward traversal. Check startup failure after insertion before continuing. A nested onChain may merge and destroy the supplied chain; reacquire it through tunnelGetChain() before subsequent work, as with ordinary chaining.

The lifecycle-v2 shutdown stages

TlsClient's configured fragment is an example of a private helper after its owner. Copy the nested settings, configure the child with nodeConfigureChild() and the owner's next link, then insert TlsClient -> StreamFragmenter during chaining. The child's normal traversal binds the configured next entry, including that entry's own helpers. The child participates in indexing, padding, layer solving and lifecycle traversal; TlsClient retains its splice-blocking metadata. Destroy the child through tunnelOwnedChildDestroy() before releasing its settings and node strings, including construction rollback. Do not manually repeat its chain-driven shutdown hooks or allocate another line for this composition.

Lifecycle ownership is explicit and context-bearing. Process shutdown invokes these stages for every tunnel in every loaded chain. Within a chain, NodeManager uses storage order; that order is not guaranteed to be source-to-tail:

onQuiesceRequest runs on the main thread, closes local admission, issues producer stop requests, and must not wait.
onWorkerQuiesce runs on each owner worker and detaches that worker's timers, watchers, and component-specific queued handles.
onQuiesceWait runs on the main thread after every worker is quiesced and joins or detaches already-admitted external callback roots.
onWorkerStop runs on each owner worker during drain. Every source or internal normal-line owner closes its authoritative inventory when visited; another hook must not assume a neighbouring owner has already run or require a borrowed registry to be empty.
onStop runs on the main thread after every worker is drained and releases remaining shared resources.
onDestroy runs on the main thread only after no worker, message, line, or external producer can reference the instance.

Every callback receives a read-only ww_lifecycle_context_t. Construction rollback, owner-driven private-child teardown, and local device restart use their own scopes and abortive close policy. Components must not infer a scope by polling process-global shutdown state. Live configuration removal is not a supported lifecycle scope.

Use the project's JSON helpers (getStringFromJsonObject, getBoolFromJsonObjectOrDefault, getIntFromJsonObjectOrDefault, etc.) and LOGF a clear message on bad config — misconfiguration should fail loudly at startup, not silently at runtime.

For per-node MTU settings, seed an int64_t from CORE_DEFAULT_MTU, strictly parse any override with jsonGetObjectIntegerInRange(), and validate the final value before narrowing or allocating MTU-dependent resources. Missing settings must not bypass validation: the JSON helper does not validate an inherited seed. Store the result in immutable tunnel state and use it for all runtime work, including delayed callbacks and lazy protocol objects. Invalid inherited values must name core misc.mtu and the supported node override in the diagnostic. See Core configuration ownership.

`structure.h`: tstate and lstate

structure.h defines the two state structs and their sizes. Keep settings and runtime fields clearly grouped, as TlsClient does:

typedef struct tlsclient_tstate_s
{
    // settings (parsed once in create.c)
    char *sni;
    char *alpn;
    bool  verify;
    // runtime, shared across all lines
    SSL_CTX **threadlocal_ssl_contexts;   // one per worker
} tlsclient_tstate_t;

typedef struct tlsclient_lstate_s
{
    SSL           *ssl;
    BIO           *rbio;
    BIO           *wbio;
    buffer_queue_t bq;
    bool           handshake_completed;
    bool           resources_released;
} tlsclient_lstate_t;

Per-worker arrays (like threadlocal_ssl_contexts) live in tstate and are indexed only by an explicitly validated worker identity: lineGetWID(line) in a line callback, worker->wid in a worker-message callback, or a checked getCurrentWorkerWID() when the callback contract guarantees an event worker. Never index them with raw getWID(). Per-connection state lives in lstate.

For frequently allocated line states and objects with several private flags owned by the same worker, prefer adjacent bool flag : 1 fields, as in the SOCKS, Trojan/VLESS line states and HttpProxyServer session/header states. Group flags to avoid alignment holes. Keep ordinary booleans when callers need their addresses, including JSON output parameters, and preserve indexed arrays where useful. Packed flags share a storage unit: they are not independent atomic objects or a wire format. Compare both sizeof and the rounded line-state size before claiming an allocation saving; layouts depend on the target compiler/ABI.

Line-state types must require no more than kTunnelLineStateAlignment (32-byte) alignment. Use tunnelTryAlignLineStateSize() when composing embedded line-state regions, and include each rounded region in the tunnel's reservation before chaining. tunnelTryAlignStateSize() is for cache-aligned tunnel-wide state. Allocation sizes, embedded offsets, and zeroing lengths must use the same line-state alignment; clearing a larger region can overwrite the next slot.

`line_state.c`: Initialize and Destroy

Nodes that reserve zero line-state bytes need no placeholder lstate type or unused initialization/destruction helpers. Omit those declarations and source files, and keep any line-state size constant consistent with the constructor.

Line state is initialized in Init and destroyed exactly once. The destroy function must zero the aligned region so the line-free debug assertion (debugAssertZeroBuf, see Part 2) is satisfied:

void templateLinestateInitialize(template_lstate_t *ls)
{
    // set up this connection's state
}

void templateLinestateDestroy(template_lstate_t *ls)
{
    memoryZeroAligned32(ls, tunnelGetCorrectAlignedLineStateSize(sizeof(template_lstate_t)));
}

Rules (from Part 2, restated where you write the code):

Initialize in Init; later callbacks assume the state exists.
Initialize the final line-state object before attaching queues to an embedded buffer budget. Its address must remain stable; never copy populated state with self-references. Destroy attached queues and release active reservations, assert zero budget usage, then clear the line-state slot.
Destroy once; after LinestateDestroy() consider the state dead and zeroed.
Before permanent line bypass, call LinestateDestroy() and zero the entire aligned reservation, including padding. tunnelBypassLine() asserts this in Debug before clearing the bit; the bypassed tunnel will not receive Finish to perform cleanup later.
Never read a field after destroying the state.
Do not add an initialized boolean unless an explicitly documented line-role/callback contract requires state before ordinary Init.

APIServer and public operation boundaries

APIServer is a Layer-4 terminal node carrying four-byte unsigned big-endian lengths followed by UTF-8 JSON envelopes. It borrows normal lines, blocks splice, and creates no listener. Direct TCP, ordinary HttpServer body tunneling and TcpOverUdpServer's reconstructed stream provide its reliable input. Raw UDP is not an API transport. HTTP headers, method/path routing and per-operation HTTP status mapping do not belong to the API envelope layer.

Authenticate each envelope using APIServer's independent static operator token before target lookup. Resolve exact logical configuration/node names globally, then apply the exact operation allowlist. Node-only selection counts every declared top-level node with that name, including nodes without a registered API or permission. Never let permission filtering remove ambiguity, expose generated children or load a filename from a selector. All loaded configuration names must be nonempty and unique when an APIServer is present; configurations without an APIServer retain their existing rules. Preparation resolves each permission to an immutable registered descriptor; runtime admission reuses that descriptor after registry freeze. Missing exact permission is forbidden. read_only describes an operation and enforces neither access, rollback nor retry safety.

Keep lexical depth/value limits local to the small outer envelope. Reject unknown/duplicate keys, malformed UTF-8, trailing input and decoded NUL in keys, metadata and UTF-8 body strings. Strict canonical base64 carries arbitrary body bytes. Preserve decoded opaque bytes without parsing or reserializing the node's schema. Do not log credentials, envelopes, bodies or returned user data.

Reserve memory before parsing, copying, encoding and handler dispatch, including temporary overlaps. Bound input bytes, retained allocation charge, fragment and request entries, sessions, unfinished jobs and output. Measure the complete encoded response before emitting its prefix. Node-level failures remain dispatched results; an encoded result over the limit becomes a bounded gateway error. max_jobs=64 is an independent ceiling, not guaranteed capacity: at current defaults the shared 64 MiB domain fits roughly seven small jobs with their 8 MiB handler allowances before substantial session storage. Retained input/output and allocation geometry may reduce that count; do not shrink handler allowance to promise 64 jobs. Connections without a verified token also charge a separate synchronized max_unauthenticated_memory domain, including temporary parsing overlaps and error output. Its default is max(one line allowance, one quarter of total memory) and configuration leaves total headroom for one handler job. On successful credential verification release the entire anonymous charge once; ordinary line/domain charges remain until their owners settle. Every envelope still authenticates independently. Clear decoded tokens immediately after verification or schema refusal, with release-time clearing as a backstop.

One operation executes at a time per connection; queued requests and responses remain FIFO. As an exact-line tail, APIServer gates its reply producer on consumer Pause/Resume, while finite deadlines and hard limits remain active. Schedule bounded parsing/output continuations so buffered work does not require new input. After a turn reaches its work limit, a complete queued request with no active job or response is also runnable work and requires a continuation. Waiting for a job or paused output alone must not cause repeated immediate wakeups; their deadlines remain armed. The frame deadline starts at first-fragment arrival, including queue residence. Reuse a published timer with wtimerReset() after refreshing owner-loop time; preserve the earliest absolute deadline and suppress obsolete pending delivery. Reset refusal closes through ordinary cleanup without an add retry. The timer callback clears its slot before progress; quiescence cancels it. Malformed framing/envelope/schema and invalid authentication close after at most one bounded error. Gateway timeout and invalid_encoding also close the connection. Publishing a complete response ends that request's deadline and begins its independent output deadline. Recheck the original deadline after encoding and before publication; securely discard an unpublished late response and follow the timeout/closure path. A timeout does not roll back a committed operation. Once a prefix/body starts, expiry closes the line instead of injecting a second frame. A queued request expiring behind an earlier pending response closes without emitting an out-of-order error.

The handler's reserved body allowance is separate from the encoded envelope cap. Reserve at least 4096 body bytes even for a smaller envelope cap, so a bounded valid diagnostic result can become response_too_large at serialization. During preparation, working memory must fit that result's actual worker-pool allocation plus 8192 bytes of bounded parsing metadata after final padding is known.

AuthenticationServer keeps API-only create/update/password preparation, typed apply and handlers beside their operations. Their shared precommit ACK formatter runs before publication; typed core commit scratch checks and exact unknown-outcome semantics remain. Public JSON rendering emits ordinary spans in chunks, checks deadlines at entry, within every 256 emitted bytes and before successful completion, and retains per-write overflow/allowance checks. List rendering measures the empty envelope and fitting immutable prefix once, then writes that same measured prefix without repeating its sizing pass. Credential-bearing producer pages remain separate from redacted administrative views. Producer password and digest point lookups likewise use the bounded full-row copy while retaining collection ownership. Check the exact compact JSON size against the response payload allowance minus the frame header before copying arbitrary strings. Oversized rows return response-too-large; invalid stored encoding returns user-json-serialize-failed.

AuthenticationClient's registered status handler runs on worker 0 and copies owned scalar values before serialization. Pin active/baseline tables and preflight their immutable collection counts before allocating a detailed scan. Preserve control_mutex -> users_lock -> collection/user -> stats lock order; aggregate sampling describes an interval while traffic continues. Diagnostics never change admission, observations, baselines, batches or scheduling state.

AuthenticationServer administrative auditing uses a node-local event queue and checked file writer, separate from diagnostic logging. Emit committed events at the same state-locked revision boundary as each API/producer mutation, before reply delivery; report rejection only when that attempt did not commit. Copy bounded actor/gateway metadata and permitted field-name bits, never arbitrary body/value/credential strings. Queue admission holds only a short metadata mutex, never waits for free capacity and performs no allocation, rendering or file I/O. The independent writer serializes bounded JSONL records, checks append/flush and rotation, and latches file failure before dropping queued/subsequent records. Queue saturation or writer failure changes audit health, not account outcomes. Status uses an independently sampled, redacted audit object even when disabled. The node reference defines settings, retention, fields and the intentionally non-durable guarantee.

AuthenticationServer explicitly registers the read-only status, get-user and list-users operations plus writable update-user, create-user and change-user-password, each on the calling event worker. The strict read-operation parser must apply the 4096-byte, depth-two and 16-value bounds before building a JSON tree. IDs are canonical nonzero unsigned decimal strings; all numeric result leaves use decimal strings without floating-point conversion. Public projections copy only their explicit permitted fields and never use the credential-bearing persistence serializer or a whole-user copy with secrets removed afterward.

Acquire server state, collection, user configuration and user statistics in that order, using nonblocking read acquisition throughout. Contention discards partial copies and returns the node-level busy error. Never report an empty successful sample on acquisition failure or wait behind an existing save. Hold state through the bounded owned copy to keep configuration/revisions coherent, then release all locks before measuring/rendering JSON. No borrowed user pointer or stored string survives unlock. Status does constant-size work, detail uses one indexed ID lookup, and a page touches at most its requested 1..100 entries. Bound scans, temporary overlaps and actual pool allocation against deadlines and allowances; account result bodies have a further 256 KiB cap. Detail strings remain complete or produce a bounded refusal. Lists copy at most 256 complete UTF-8 bytes of the name and never scan email/notes; an invalid inspected name prefix produces an explicit null preview while traversal continues.

List cursors use the current collection order and are stateless position hints, not credentials or persistent snapshots. Bind each 64-character canonical cursor to the random process-local server instance, configuration revision and next position: 32 lowercase hexadecimal instance characters, 16 revision characters and 16 position characters. Check instance/revision/count under the state read lock before indexing. Any accepted configuration revision change invalidates prior cursors, including same-value patches. Removal/replacement paths must publish their configuration revision under the same state write lock. Statistics-only changes do not invalidate them; each page reports its current statistics revision and accounting-clock reference. Advance only past emitted records, return a smaller nonempty prefix when budgets require it, and never issue a partial-success cursor after timeout or contention.

Every successful server read publishes an owned edit_token sampled with its configuration revision: the existing 32 lowercase hexadecimal instance characters followed by 16 revision characters. The 48-character token is a server-wide optimistic edit precondition, not an authorization credential; pagination cursors keep their separate 64-character representation. Statistics-only mutations do not change the token, while any accepted configuration edit does, including unrelated-account and same-value edits. Recreating the server changes its identity.

update-user requires its own exact operation permission. Keep monitors read-only; producer credentials and protocol allow-user-write grant no gateway permission. Use public snake_case and u/d leaves, with canonical unsigned decimal strings, in this operation only. Require exactly nonzero id, canonical if_match, and a nonempty sparse patch. Reject protected fields, empty supplied containers, unknown/duplicate keys, wrong types and invalid encoding before any mutation. Bound the body to 32 KiB, lexical depth to four object levels and values to 64 before constructing its JSON tree. Preserve read-parser bounds unchanged.

Share the owned configuration-only preparation, semantic validation, preflight and nonfailing commit primitives with the existing blocking protocol/object callers. The API tries locks in server state write -> collection write -> target user configuration write order. Configuration patches must not acquire or write the statistics lock or replace whole user/time/statistics snapshots. Under state, compare instance and revision before resolving the account; mismatch is conflict without a fresh token. Ordinary contention returns busy without waiting. Acquire every required lock before erasing an index entry or changing a field. Complete uniqueness, overlap, capacity, memory and deadline checks before the first logical mutation; no refusal may leave half-applied state.

The API name-index path must use reserved/no-grow insertion and non-advancing erase; it must not reserve count + 1 and rehash the table for one edit. Bound inspection of the selected account's old name to 32 KiB plus terminator detection only when renaming, and refuse oversized names without changing them. Other selected fields must not hash/inspect that name for diagnostics. Preserve existing average-case hash semantics; cluster probing/shifting is not a count-independent worst-case latency guarantee. Reuse existing Allowed-IP tree nodes when possible; preflight exact checked new-growth geometry against 1 MiB and the remaining working allowance before allocation/copy. No second index or full-user scan is introduced. Fixed API diagnostics must not scan or log arbitrary stored names, notes, request bodies or credentials.

Charge parser storage, prepared values, scratch, index growth and the actual result-pool allocation including overlaps. Preflight the fixed 256-byte success body and reserve its buffer before mutation. Render the anticipated ID, next revision and token before the final precommit deadline check. Then commit the selected fields/index membership and increment configuration revision exactly once within the state-write transaction. After commit, do no fallible allocation, deadline-aware rendering, fresh snapshot or rollback: return that transaction's prepared acknowledgement. Private capacity reservation may survive a precommit refusal; logical account/index/revision/accounting state must not change.

Guard configuration-revision exhaustion before mutation in both API and producer AddNewUser/UpdateUser paths, and keep an invariant guard in the increment helper. UINT64_MAX-1 to UINT64_MAX remains valid; further mutations refuse rather than wrap. Preserve statistics, usage, first-use/creation observations, credentials, runtime reservations and producer checkpoints. Disabled/expired accounts remain editable by durable ID. API success acknowledges memory only, with no save or client-refresh guarantee. Existing save frequency and producer ordering remain.

A returned operation-level failure guarantees this edit did not commit. A lost reply, abandoned connection or late gateway timeout may follow a committed edit and therefore leaves its outcome unknown. Preserve generic dispatcher completion behavior; never infer rollback or add cancellation rollback. Callers reread and reconsider after conflict or unknown outcome; never automatically obtain a fresh token and replay the old patch. Request IDs are correlation labels rather than idempotency records. The compact acknowledgement, including maximum request-ID escaping, must fit the gateway's minimum 1024-byte encoded envelope.

create-user has an independent exact operation permission; update permission and producer credentials do not grant creation. Require a caller-supplied canonical nonzero decimal-string ID and nonempty exact UTF-8 password, with only an optional public config. Reuse the 18 strict configuration leaves and numeric rules without relaxing update-user's nonempty-patch requirement. Omitted or empty creation config preserves userCreate() defaults: enabled, 120000 ms statistics interval, empty strings and zero unlimited/unset limits/expiry. Supplied nested objects must remain nonempty. Bound creation bodies to 32 KiB, depth four and 64 lexical values including keys before JSON-tree allocation. Reject token fields, protected import fields, aliases, NUL and malformed UTF-8.

Prepare a fresh private owned user outside shared locks; credential derivation must not run under the API state/collection locks. Preserve 32-byte user_t alignment with matched aligned allocation/release, and never copy initialized native locks with struct assignment or memcpy. Derive ordinary credentials from the exact supplied password and use constructor-generated wall-clock creation time. Keep statistics, first use, runtime and client-local observations empty. Destroy/wipe owned password staging and unpublished users on every refusal and release preparation ownership after transfer. Public creation/read responses remain projections with no credentials.

The creation transaction tries server state write -> collection write, without acquiring any existing user's config/statistics lock. Reject exhausted revision under state; under collection prioritize existing ID over other duplicate classes. Share existing indexed integrity/preflight/reserved insertion with blocking add/load callers while preserving their duplicate precedence. Creation never upserts or treats an identical account as a successful replay. Fixed credentials_exist groups hash/UUID/WireGuard conflicts without revealing keys or the conflicting account; crypto/backend or native-init refusal is typed creation_failed, separate from malformed input.

Before any reserve, admit complete checked new extents for active pointers, monotonic stable slots, block pointers/new 64-user blocks, all seven hash maps' value/metadata arrays and optional Allowed-IP growth. Use real STC rounding and load-factor geometry. Charge private preparation, aligned allocation overhead, scratch, unpublished-slot/string copies and actual result-pool storage, including peak overlaps, against the supplied aggregate working allowance. Creation has no fixed 1 MiB total-growth cap. Keep name-map capacity sufficient for all active users, including unnamed accounts, so later bounded renames can insert without growth. Removed stable slots remain unavailable until collection clear.

When name-map growth is required, inspect admitted buckets and stored names with a cumulative byte-work budget equal to the supplied working allowance, including terminator detection and bounded deadline checks. Refuse before rehash if the scan cannot fit. Skip this full scan when name capacity is sufficient. Preserve average-case collision semantics and finite synchronous native rehash/crypto; check deadlines between reserves without promising interruptible operations or a strict response-time bound. Fixed diagnostics never scan/log stored names or credentials.

Reserve/prebuild the same 256-byte transaction ACK and actual pool charge before publication. Prepare an unpublished stable slot, including all native lock/string failure paths, and check deadlines immediately before publication. Reserved insertion must not allocate/rehash: publish all indexes, active pointer, active count and monotonic slot count once, then increment configuration revision once under state ownership. Impossible commit-stage inconsistencies are fatal invariants, not ordinary partial-success errors. Private reserved capacity may survive precommit refusal; membership/counts/revisions and published pointers must remain stable. After commit return the existing ACK without allocation, rendering, deadline refusal or rollback. Existing accounts, statistics revision and producer checkpoints remain unchanged.

Creation has no edit-token precondition: unrelated valid creations/edits do not invalidate the input. Success returns the exact committed revision/token and invalidates older tokens/cursors. No immediate save or telemetry update occurs; ordinary periodic/final persistence and client refresh remain separate. Keep producer AddNewUser synchronous save/rollback unchanged. An offline client cannot admit a previously unknown account until normal refresh, and existing pending accounting must survive that refresh.

A returned creation node failure publishes no account. Post-dispatch delivery failure is outcome unknown even if shutdown/disconnect abandons the reply. The caller retains the original ID/inputs and rereads that ID. Presence establishes existence, not creator or password ownership; absence permits an explicit same-ID retry decision while the first request may still run. Never automatically choose another ID, replace a user, reset credentials or treat id_exists as replay proof. IDs are caller-managed permanent identities unique in the current collection, without an all-time ledger or cross-crash exactly-once guarantee.

change-user-password requires its own exact permission; a password administrator normally receives only get-user and this operation. Require exactly canonical nonzero decimal-string id, canonical if_match and a nonempty exact UTF-8 password. Reject aliases, extra/duplicate fields, NUL, malformed encoding and nested values. Before JSON-tree allocation, bound the body to 32 KiB, root-only container depth one and 16 lexical values including keys. Check token mismatch under state before resolving the account. Disabled, expired and quota-exhausted accounts remain editable; a same-password assignment still advances revision once.

Share ordinary credential derivation and the lock-held credential transfer with the object layer. Prepare one owned password and fixed derived hashes/UUID/key material outside shared locks, deriving each primitive once. Preserve required alignment and paired allocation/release; never copy user objects or native locks. Charge parser/prepared/scratch/result storage and peak overlaps against the working allowance. Wipe owned request/decoded/prepared secrets on every result and securely clear replaced plaintext and temporary secret-equivalent material. Fixed diagnostics must not include bodies, credentials or arbitrary stored strings.

Try locks in server state write -> collection write -> target user configuration write order, with no statistics lock, I/O or worker wait. Check the four credential indexes for another account's keys, allowing the target's own identities. Plain to UUID, UUID to plain, UUID replacement and case-equivalent UUID assignments must update validity/membership correctly. A textual change need not revoke an unchanged protocol identity. Admit the selected old password using its authoritative cached owned length under the existing locks, with a 32 KiB maximum; an oversized imported password yields resource_limit unchanged. Keep bounded scanning of external new passwords, deadline checks before/after precommit, and secure cleanup proportional to the replaced allocation. Inspect no unrelated strings or full table. Preserve average-case hash semantics, without promising interruptible crypto or worst-case constant-time collision work.

The object implementation keeps storage/import/copy in users.c, hash indexes in users_index.c, Allowed-IP intervals in users_allowed_ip.c, transactions/preparations in users_update.c and users_prepared.c, public projections in users_public.c, and runtime accounting/migration in users_runtime.c. Private locked helpers/types stay in users_internal.h. The four fixed-byte credential maps share a typed private template; name, ID, pointer and interval indexes retain their distinct semantics. User state, credential preparation and copy staging share an aligned owned credential value; initialized native locks are always constructed separately.

Every AuthenticationServer account must fit a complete synchronization page, including future emitted numeric/statistics growth. The full-row serializer derives a fixed reserve using all five string keys and maximum scalar widths, including unsigned values encoded as decimal strings. Cached escaped-string charges supply the remaining row cost. Startup derives the one-row page reserve from maximum-width page metadata and subtracts that plus the nine-byte response frame header from the response payload limit; the outer envelope has separate geometry. The objects library receives only the resulting row-byte allowance.

Prepared create/configuration/password transactions check this prospective bound under their existing ownership locks before precommit or logical mutation. Configuration checks overlay selected prepared charges in constant work over five fields, without reading unchanged text or acquiring a statistics lock. Producer patches use the blocking export-limited transaction; administrative transactions retain their try-lock/deadline behavior. Generic convenience APIs explicitly delegate with SIZE_MAX; zero is a real zero-byte allowance. Typed too-large refusals preserve all account/index/accounting state. Keep the read-time exact bounded serializer as an independent defensive check.

User expiry helper now_ms belongs to the projected client's monotonic domain when a projection is valid, and otherwise to persisted expiry's wall-time domain. Accounting-clock installation and observation rebuild local expiry from retained first-use evidence; the manual projection setter does not erase that evidence. Runtime migration targets unpublished staging users, preserving new policy/timeinfo while transferring old reservations and accounting clocks, then rebuilding each projection before publication. See accounting ownership.

Missing old credential ownership, impossible index entry counts and missing reserved insertion capacity are internal corruption: emit a fixed fatal diagnostic and terminate directly. Ordinary duplicate and capacity refusals remain typed precommit results. Corrupt removal must never allocate or rebuild indexes before terminating.

All credential maps reserve for every active user, including users without UUIDs. Replacement admits final occupancy within existing capacity and uses non-advancing erase plus reserved/no-grow insertion, without reserve/rehash. Acquire all locks and finish uniqueness, capacity, budget and deadline checks before any logical mutation. Prebuild the existing 256-byte ACK with this transaction's next revision/token in already admitted pool storage. After a final deadline check, transfer credentials and publish all memberships as a nonfailing commit, then advance configuration revision once under state. Do not allocate, render, take a new snapshot or return a deadline refusal after commit. Refusals preserve all logical account/index state; irreversible invariant failures are fatal rather than recoverable partial changes.

Preserve user address/ID, count/order/slots, configuration/Allowed-IP indexes, creation/expiry/first-use, traffic/statistics revision, checkpoints and reservations. Use typed credentials_exist and password_change_failed results for collisions and backend failures. Guard revision exhaustion before mutation. API success is memory-only; periodic/final saving and normal client refresh remain separate. The ACK schema reuses the existing minimum-envelope proof. A returned node failure means this invocation did not commit; a lost reply leaves the outcome unknown. Reread for a current token, but do not treat public fields or an advanced revision as proof of the password. An explicit resubmission of the retained desired password can establish it at a new commit point. Never fetch a token and retry automatically, or add request records, replay-success shortcuts or crash recovery.

Primary-save telemetry is fixed-size process-local state under the server lock. Record an existing locked save once, at its primary failure/commit decision; loading and read queries do not create attempts. Use monotonic ages with explicit validity bits so zero is valid, preserve the previous success after a failure, and do not downgrade primary success for optional postcommit backup/sync warnings. This view changes neither persistence frequency nor its commit guarantee.

AuthenticationServer verbose and producer allow-stats-push, allow-user-pull and allow-user-write must be unique JSON Booleans in their own objects; absence defaults to false. Reject invalid types before runtime timers.

AuthenticationServer database loading and producer AddNewUser must validate the complete byte span before importing users: reject trailing non-whitespace, invalid UTF-8, raw/decoded NUL and duplicate decoded keys. Database reads retain their actual length and existing cJSON nesting policy; do not substitute API-envelope byte/value caps or change supported import layouts. Use configLexicalParseBoundedQuietWithCleanup() to wipe owned strings before the strict validator deletes a fully parsed but rejected tree; accepted trees remain the caller's responsibility. jsonWipeStrings() follows cJSON ownership and leaves reference values/children and constant keys alone. Partial trees discarded inside cJSON on syntax failure and unowned transport copies are outside this cleanup.

AuthenticationServer saving distinguishes a primary absent at open from other read failures. Confirmed absence publishes the valid in-memory table without rotating or requiring the existing recovery backup; subsequent saves resume ordinary rotation. Empty/read/close failures retain the ordinary save refusal. A failed producer addition may reclaim only its unpublished last storage slot through usersRollbackUnpublishedLastUserByIdentifier(). The caller must continuously hold external exclusive transaction ownership from insertion to rollback, expose no candidate pointer and roll back the most recent addition. The helper locks the collection, validates both last-active and last-allocated identity, removes all indexes and destroys the candidate before decrementing the slot count. Identity/storage mismatches are fatal invariants. Retain allocated capacity for retries; ordinary removal still invalidates pointers and never reuses their slots. Startup rollback must skip final database saving while releasing all created resources; normal process shutdown saves a loaded database. Weekly history slots subtract calendar days from tm_mday before local mktime, with tm_isdst=-1. Audit path checks preserve bounded configured/resolved paths and checked suffix formatting, and distinguish unsupported paths, failed parent resolution and actual canonical/file-identity collisions. Every failure refuses startup before writer creation.

Shared user Allowed-IP parsing requires complete inet_pton IPv4/IPv6 syntax; trim only surrounding whitespace, then preserve existing prefix checks, host-bit masking and overlap rules. IPv4-mapped IPv6 remains IPv6: ::ffff:10.0.0.1/128 is distinct from 10.0.0.1/32 for lookup and overlap; never silently convert it to IPv4. Imports reserve no more than 64 speculative records above the existing count before validating entries. A string-valued password or pass identifies a standalone record; object-valued entries with those names remain name-map users. The selected password still takes precedence over pass, and the users wrapper retains precedence. Failed append restores prior records, handles and indexes without requiring capacity to shrink.

See APIServer for gateway settings, the API protocol for wire envelopes, and the transport examples for HTTP/1.1 composition.

HTTP-Specific Rules

TlsClient registers ALPS only for an offered h2, with empty client application settings and codepoint 17613. BoringSSL serializes the ClientHello protocol list; 02 68 32 is the length-prefixed h2 name, never the settings value passed to SSL_add_application_settings(). HTTP/1.1 and custom ALPN offers have no ALPS registration. Received server settings remain inside TLS: TlsClient neither applies them to an HTTP engine nor forwards them to adjacent nodes. This shapes the TLS exchange without promising Chrome's application-layer behavior. See the TlsClient ALPS reference.

HttpProxyClient is a separate HTTP/1.1 node using HttpProxyCommon's pure parser. It has one borrowed normal line and no authentication service dependency. CONNECT optionally owns an internal DomainResolver tunnel before the client core: source -> origin resolver -> HttpProxyClient -> optional TlsClient -> proxy transport. The helper uses the same borrowed line and participates in chain indexing, padding, layer solving and splice capability. Branch callers enter through tunnelGetBranchEntry(). The parent destroys its helper through owned-child destruction; normal chain traversal already drives its shutdown hooks. The following transport independently selects/resolves the proxy; core Init snapshots the configured/dynamic origin for CONNECT authority or an absolute HTTP target. Its HTTP mode generates one request, permits only successful final responses, and delivers decoded body bytes. It does not share HttpClient's HTTP/2, WebSocket or split-connection machinery. Client callbacks keep request startup, upload framing, response decoding and directional cleanup in their corresponding files. Common helpers own strict settings, headers, bounded ordering drains and timer cleanup. Capability is advertised only for the documented direct body paths and checked parsing fallback; whole-chain splice gates remain authoritative.

HttpProxyServer exposes an HTTP/1.x forward-proxy interface, unlike the body transport interfaces of HttpClient and HttpServer. Its incoming normal line is borrowed; each outbound normal child is created and destroyed by the proxy. The incoming line stops at the proxy and only the child initializes the actual next chain entry. Child responses and pressure cross back to the associated client identity. Authentication selects exactly one of no-auth, an immutable local credential list, or AuthenticationClient tracked accounts. Each authenticated request reauthenticates before reuse. Reuse requires the requested authority and mode; local mode also requires the same complete credential pair, while tracked mode requires credential bytes and user ID/generation. Resolved destination context is mutable and must not be used as that authority key. Only tracked mode creates the internal UserController and adds a real user handle with credentials to the child. Local mode adds a credentials-only marker for routing, with no new account tracking or per-user limits. Both preserve inherited identities and avoid appending repeated markers to the borrowed client or reused child. Current TCP read EOF closes the full transport, including on IOCP; proxy persistence does not introduce half-close semantics.

CONNECT and fallback retain this same split between a borrowed client line and an owned child. HttpProxyServer remains the tail of one line and the head of the other, so it keeps mapped producer pressure and cannot permanently bypass itself. Opaque payload relay does not remove its association, deadline or child-cleanup responsibilities.

HttpProxyServer keeps authentication commitment, request reuse and branch selection in upstream/payload.c, with credential evaluation in common/auth.c returning explicit acceptance, denial, service-unavailability or internal-failure outcomes. Strict settings and local-user parsing live in common/settings.c; constructor allocation, callback wiring and rollback remain in instance/create.c. Response processing stays in downstream/payload.c. Both HTTP nodes put implementation helpers in internal.h, state in structure.h, and exports/custom callbacks in interface.h. Init, Est, Finish and receiver permission updates live in their corresponding callback files. common/input.c shares bounded admission and HTTP framing; common/flow.c coordinates exchange progress, raw relay and combined producer pressure; common/lifecycle.c owns session references and child creation/teardown. The shared pump serializes reentry and advances retained work from Payload, Resume, Est and timer callbacks.

The session embeds two hps_direction_state_t objects in directions, indexed by kHpsUpstream (client to selected child entry) and kHpsDownstream (child to client). Each groups its ordinary input/output/deferred/active buffers, body parser, optional trailer-owner pointer, header deadline, receive nesting count, and permission flags. Only an effective chunked body allocates a trailer owner, which holds the parsed context, saved raw header and its length together. Parsed trailer names borrow that saved header. Transfer-Encoding metadata on HEAD/304 responses does not allocate an owner. paused records consumer permission; read_paused records pressure sent to the producer. Thus upstream Pause/Resume changes downstream paused, and downstream Pause/Resume changes upstream paused.

Directions share the session's allocation and lifetime; an interior direction pointer grants no independent lifetime or permission after a callback. Never copy or reset a complete live direction object. Trailer cleanup detaches the owner pointer before wiping and freeing the header and owner, leaving the header deadline independent. Completion, canceled upload on an early final response, errors and terminal cleanup settle the owner. Acquisition happens before outward callbacks and follows the fatal heap-allocation policy. Exchange and terminal cleanup preserve the existing member-specific resets, including receive counters that unwind after reentrant close. Child close resets only upstream paused and downstream read_paused within the directions. Association, authentication, timer, pump and asymmetric exchange state remain in the session. Upstream producer pressure still considers total session retention; downstream response progress must remain possible while request storage is full.

Optional fallback-node-name selects a distinct configured L4 branch only for first-request authentication denial or service unavailability. Explicit lookup outcomes distinguish rejection from internal errors; parser, local-storage and post-authentication failures never select fallback. Successful authentication commits a monotonic connection latch before rewriting, local OPTIONS or outward callbacks. Returning to request parsing must not reset it. Fallback creates only one owned child through the resolved callable entry, bypasses this proxy's private UserController, inherits legitimate metadata, and never installs rejected credentials or the failed authority as child routing metadata. Its exact original wire replay includes credential bytes, followed by permanent opaque bidirectional relay. No local HTTP response or branch retry is allowed after selection. Connect and idle deadlines remain; HTTP header deadlines end at handoff. The node uses the existing worker child inventory and quiescence gates. Splice relay requires no older directional storage, no active pump/Init or outer Payload dispatch, and a live exact association. CONNECT also requires its success handoff; fallback does not require Est. Delivery uses the selected entry under the pump guard, so nested input materializes and waits in FIFO order. Handoff consumes the wrapper even if a callback closes the session. All retained input, output, deferred and active incoming buffers remain ordinary; HTTP bodies stay materialized throughout an exchange. After authentication selects a normal HTTP exchange, HttpProxyServer requests ordinary upstream reads on the borrowed client line and ordinary downstream reads on each owned HTTP child before its Init. CONNECT and fallback do not request these preferences. The two directions belong to different lines, so the server uses the directional helpers separately. An incoming preference from an earlier HTTP exchange remains set if that connection later enters CONNECT. These requests do not replace handling for splice input already delivered or arriving through another line association. All HTTP activity and deadline timestamps use wloopNowMonotonicMS() on the client line's owner loop, with cached-loop freshness and no wall-clock adjustment.

Pure authority, header syntax and streaming body parsing live in tunnels/Internals/HttpProxyCommon. HttpProxyServer retains its request rewriting and Basic credential decoding in its own common/parser.c; its parser entry preserves HTTP/1.0 admission. The common HTTP/1.1 response entry distinguishes ordinary, HEAD and CONNECT contexts. Successful CONNECT validates generic header syntax but ignores Content-Length and Transfer-Encoding body framing, including conflicting fields; ordinary responses retain strict framing validation.

HttpProxyServer uses the shared ASCII/hex helpers and wwBase64DecodeCanonical() from ww for Basic credentials. The canonical decoder bounds output capacity, requires standard Base64 padding where needed, rejects nonzero padding bits and writes no NUL terminator. The proxy reserves that terminator itself and retains its own credential rules: no control bytes, the first colon as separator, and 1–255 bytes each for username and password. wwBase64Decode() does not enforce canonical padding bits and must not replace the canonical decoder for this path.

HttpProxyServer treats max-pending-bytes (P) as its working pending-data budget. Four working buffers retain at most 4 * P allocation bytes. Each direction may additionally hold one already-delivered remainder of at most D = 2 MiB logical bytes. This node-owned headroom is independent of pool sizes, RAM profile and splice settings. Total retained logical bytes are bounded by P + 2 * D. A single delivery is limited to P + D; only its immediately drainable prefix may exceed the per-direction remainder allowance. Each remainder's payload capacity is at most max(small_capacity, 2 * D) plus cache-line rounding; its full padding, sbuf header, and aligned-allocation overhead are additional. Remainder admission copies into ordinary best-fit storage (pooled or dedicated) when needed to satisfy this bound. Working slice room excludes deferred bytes so moving bytes into working storage never double-counts them. Remainders participate in pressure, FIFO pumping, and every error/timeout/close cleanup path. Blocked pipelined requests leave response progress available. Raw fallback and CONNECT relay do not copy their deferred bytes back into working storage to forward them. One paused direction must not consume the capacity required to drain the other. Their owned child's Finish settles that child immediately and drains already accepted replies under current client permission before finishing the borrowed client.

HalfDuplexServer and HttpServer split derive waiting budgets from scale = max(1, ceil(basis / 32768)), where basis = bufferpoolGetWaitingBudgetBasis() is 64 KiB in S1/S2 and 1 MiB in higher profiles, independently of allocation sizes and the splice limit. HalfDuplexServer closes at or above 131070 * scale, including initial upload publication; HttpServer split closes above 131070 * scale unless its explicit buffering-limit bypass is enabled. Handshake/header validation and available-peer pairing precede waiting checks. The 32 KiB divisor is a fixed budget reference. Wire and HTTP framing limits remain independent.

ReverseServer uses fixed per-half application FIFO limits of 2 MiB and 1,024 buffers, shared by waiting and startup retention and independent of pool profile. Its activation prefix has a separate protocol bound; see Part 3.

If you work on HttpClient or HttpServer, these source-backed constraints apply:

HTTP/1.x header limits count bytes through the first complete CRLF CRLF, including the terminator. Body bytes received in the same buffer do not count against that limit. Without a terminator, the buffered header prefix remains subject to the limit. This applies to request, response, and split headers.
WaterWall targets HTTP/1.x and single-stream HTTP/2 only. Do not add HTTP/3. Do not support more than one logical HTTP/2 stream. If a peer opens extra streams, reject or ignore them conservatively per the existing design.
HttpClient and HttpServer share the ordinary-buffer HTTP/2 DATA-provider lifecycle in tunnels/Internals/HttpCommon. Each queue stays at a stable address on its buffer pool's owner worker. Append transfers payload ownership; read EOF does not release the active item. Only its matching DATA frame-send callback retires it, or terminal cleanup releases it after deleting the nghttp2 session. DATA_EXIST retains the same front item; another submission error releases only the rejected item and leaves the rest for terminal cleanup. Nodes retain stream selection, HEADERS readiness, empty-input policy, END_STREAM decisions and wire output. Keep both line-survival and state/direction checks after wire callbacks.
HttpClient framing: headers remain separate from body input. HTTP/1 body chunks contain at most 16,372 payload bytes and publish prefix, body and final CRLF in one buffer. Empty input is not a terminal chunk. Reentrant body input must remain behind all pieces of older input. Nested admission uses the existing queue with limits of eight buffers, 1 MiB payload and 2 MiB allocation charge; refusal closes the line. Finish stops output and releases every untransferred piece. HTTP/2 DATA must wait until nghttp2 has serialized this request's HEADERS, including when a callback admits payload during startup.
HttpClient HTTP/2 profile: direct startup combines preface, initial SETTINGS and the connection WINDOW_UPDATE before separate request HEADERS. The ordinary SETTINGS profile is Chrome 154's 64 KiB header table, disabled push, 6 MiB stream window and 256 KiB header-list limit; the connection window is 15 MiB and the default 16 KiB receive-frame limit remains. WebSocket additionally advertises extended CONNECT support. Share this SETTINGS definition with h2c upgrade. Outgoing DATA payloads are capped at 16,375 bytes so the nine-byte frame header fits within one 16 KiB TLS plaintext record. Do not confuse advertised receive limits with outbound DATA sizing or infer browser pooling from this profile.
h2c upgrade (nghttp2_session_upgrade2()): stream 1 is the original upgraded request. The client must not submit a second synthetic request on stream 1; the server must not auto-generate a fake synthetic response on stream 1.
h2c with a request body: upgrade-with-body on the original HTTP/1.1 request is not safely supported in the single-stream design. The client should reject it; the server should ignore or reject an upgrade whose HTTP/1.1 request carries a body.
Clean finish: an upstream Finish for the request side may require final HTTP bytes first. Send them while holding a line reference, then destroy local state, then forward the real Finish — the general final-bytes pattern from Part 2. Do not keep HTTP line state alive after a clean finish unless you intentionally held a line reference for that purpose.
HttpClient split teardown: retain the borrowed main line and both owned transport lines before final-byte or close callbacks. A callback on one transport can close its sibling and clear main state while the borrowed main line remains alive. After callbacks, check logical life and the existing main split role before cleaning or notifying it again. Honor the main line's prev_finished flag even when closure arrives through the other transport.
Custom raw upgrade: a successful non-WebSocket, non-h2c HTTP/1.1 upgrade makes the single-line client and server transparent after the opening header, coalesced input and retained output settle. Serialize reentrant input behind older bytes during this transition. Then destroy and zero the aligned line state before permanent bypass; suspended startup/parser callbacks must not access retired state. HTTP body framing, HTTP/2, WebSocket and split modes stay active for their ongoing protocol or line-ownership duties. Pause/Resume already relay transparently on ordinary same-line paths; split paths retain their exact line mapping and Finish guards.
Pairing IDs (HttpClient split mode and HalfDuplexClient): give every logical pair a fresh 128-bit identifier from two sequenced fastRand64() values, then copy that same identifier to both child transport lines. Never derive pairing IDs from counters or non-cryptographic output. The ID is internal routing metadata, not authentication or transport protection; chains that need those properties use dedicated security nodes.
HalfDuplex rendezvous publication: HalfDuplexServer performs the opposite-role lookup and, when no peer exists, publication of the current role as one lock-linearized transaction across both pending maps. This keeps simultaneous upload and download arrivals from both observing a miss. A peer owned by another WID is identified while the maps are locked, but the current decision copies only that WID, not the borrowed peer line-state pointer, and the current line is piped only after releasing the lock. No worker handoff or tunnel callback runs inside the pending-map transaction. Final child destroy requires both borrowed pending maps to be empty and never reclaims their line state or buffered payload on behalf of the real line owners.

The Implementation Workflow

Use this order for any new feature or fix. Inspect the repository first; ask a focused question only when a consequential requirement, compatibility choice, or scope boundary cannot be resolved from code and documentation. Make and record reasonable assumptions for low-risk implementation details instead of delegating them back to the maintainer.

Draw the chain flow. Mark the Up-end and Down-end, and label which callback direction owns the transformation.
Identify the line owner. Which tunnel creates and destroys the line?
Read the existing tunnel and its immediate neighbors.
Read the contracts: ww/net/line.h, ww/net/tunnel.h, and ww/net/chain.c.
If framing/prepending is involved, read ww/bufio/shiftbuffer.h and buffer_pool.h, plus nearby tunnels' node.c padding.
Compare relevant mature tunnel patterns, but verify them against the contract, callers, and neighbors rather than copying them blindly. Check existing containers and helpers before adding a private data structure, buffer utility, or timer-based idle scheduler.
Implement the smallest complete root-cause fix that preserves composition. Do not minimize the diff by retaining a known defect, duplicating unsafe logic, or patching only a symptom.
Decide whether focused regression coverage is warranted. It normally is for new behavior, a reproducible regression-prone bug, a meaningful edge case, or a contract change—especially when a test can fail on reversed direction. Reuse existing direct coverage when sufficient, and do not manufacture behavior tests for mechanical or documentation-only work.
Validate with the preset build and the relevant tests (see Part 6).

Avoid speculative abstractions

A new generic layer is rarely the fix here. Preserve the established lifecycle contract and prefer a direct complete fix. If one tunnel appears to require a new lifecycle concept, stop and verify the need against the core contract and every affected caller; do not use a reference tunnel as proof that either the old or new shape is correct.

Continue to Part 6: Building, Testing & Reviewing.

Attaching a tunnel to lwIP

initTcpIpStack() is an idempotent construction-time shared-bootstrap facade; it creates no thread. Global startup initializes shared storage before spawning workers. The runtime lazily constructs an engine on its ordinary owner worker. The standalone engine constructor retains its nullable contract: a failed construction leaves the slot empty and the affected packet/flow refused. A tunnel constructor must not create foreign engines or publish partial netifs.

PTC and CTP attach separate netifs to the owner's shared engine. Their onWorkerStop hooks detach local PCBs, listeners, drains, timers and netifs. Main-thread onStop checks completed worker barriers and frees shared node services; it must not traverse worker stack objects. Nodes never destroy engines. Runtime teardown releases each engine after detached messages settle and before the saved event loop and owner pools are destroyed.

TunDevice and PTC advertise trusted packet-checksum capability only in their node templates. The active bit starts clear and is published by TunDevice after final adjacency and device framing are known. Constructors and PTC startup must not cache an inactive decision before that publication. Public node/tunnel layouts remain unchanged; mode never propagates through a helper node.

TunDevice also advertises the static kNodeFlagPacketPayloadEnqueueOnly flag. Its packet payload callback must preserve Part 2's non-reentry and finite enqueue contract across Linux, Darwin and Windows. No public node/tunnel layout changes are required. PTC tests exact reciprocal adjacency, L3 sink capability and owner packet-line identity; it never infers this promise from type names or checksum trust. Removing the promise requires returning that edge to deferred dispatch.

Directory Layout​

What Each File Owns​

Caller preconditions and failure handling​

Avoiding redundant checks and state​

Heap allocation failure​

Typed internal capabilities versus runtime messages​

Address-context metadata ownership​

node.c: Declaring the Node​

Layer Groups & Strict Topology Validation​

Bridge and reverse-node adjacency​

Node Flags​

External-node ABI​

create.c: Building the Instance​

Inserting an internal helper before a tunnel​

The lifecycle-v2 shutdown stages​

structure.h: tstate and lstate​

line_state.c: Initialize and Destroy​

APIServer and public operation boundaries​

HTTP-Specific Rules​

The Implementation Workflow​

Attaching a tunnel to lwIP​