Part 2: Lines, Callbacks & Lifetime Safety

This part covers the contracts that cause the most crashes when they are broken: the per-connection callback lifecycle, who owns a line, and how to survive the re-entrant callbacks that can destroy a line out from under you.

If you read only one part of this guide before touching connection flow, read this one. Prerequisite: Part 1 (objects, directions, lifecycle).

The Six Flow Callbacks

Every connection is driven by six events, each in two directions (upstream toward next, downstream toward prev):

Event	Meaning	Direction notes
Init	A line is starting; initialize this tunnel's per-line state.	Flows upstream from the creating adapter.
Est	The far side is actually established (e.g. the remote socket connected).	Usually flows downstream, back toward the originator.
Payload	A chunk of data (sbuf_t) to transform and forward.	Either direction.
Pause	Backpressure: stop sending in this direction for now.	Either direction.
Resume	Backpressure released: you may send again.	Either direction.
Finish	This direction is closing.	Directional and destructive. See below.

A tunnel implements only the handlers it cares about and leaves the rest at the framework default (a pass-through to the neighbor). For example, an encryptor overrides Payload in both directions but can leave Pause/Resume at default.

How a Line Starts

A normal connection begins with Init, originated by an adapter:

1. An adapter (e.g. TcpListener) accepts a socket and creates a line_t.
2. It calls tunnelNextUpStreamInit().
3. That invokes the next tunnel's fnInitU, which initializes its own line state and forwards Init further upstream.
4. Each middle tunnel initializes its per-line state in turn.
5. The final adapter (e.g. TcpConnector) starts the real network operation.

Init is not isolated

A tunnel's Init can synchronously trigger more callbacks before it returns — Payload, Est, Pause, Resume, or even Finish — because the downstream tunnel it called into may immediately push data back or close. This is the same re-entrancy hazard described below; it simply starts at line birth.

Rules for Init:

• Initialize this tunnel's line state in Init. Later callbacks are allowed to assume that state already exists.
• Do not add an initialized boolean to guard later callbacks. In the normal WaterWall lifecycle Init always runs first for a tunnel; an initialized flag is almost always a band-aid over unsafe control flow. Add one only if the source clearly proves it is required.
• If your Init calls a re-entrant forwarding helper and then still needs to touch the line afterward, protect it (see Line Locking). If your Init just forwards and returns, you do not need to lock.

Line Lifetime: refcount and alive

A line_t carries an atomic refc and a boolean alive (see ww/net/line.h). These are two different ideas, and conflating them causes bugs:

• lineLock(line) increments refc. It guarantees the memory will not be freed while you hold the reference.
• lineUnlock(line) decrements refc, freeing the line when it reaches zero.
• lineDestroy(line) sets alive = false and drops the creator's reference.
• lineIsAlive(line) reports whether the line is still logically open.

The crucial distinction:

Holding a lock keeps the memory valid. It does not mean the line is still logically alive. After a re-entrant callback, a locked line may be alive == false. Always re-check lineIsAlive() before continuing.

Logical death is immediate; physical reclamation is not. lineDestroy() clears alive and drops one reference. If anyone else still holds a lineLock() the allocation stays readable until they unlock — which is exactly what makes the owner postcondition below both checkable and cheap:

lineLock(l);                 /* keep the allocation present for the assertion */
owner->fnFinD(owner, l);
assert(! lineIsAlive(l));    /* logical death is immediate */
lineUnlock(l);               /* physical free may occur right here */

A logically dead line must never receive another flow callback, and by the time it is reclaimed every tunnel's line state must already be zeroed.

Who May Destroy a Line

Only the tunnel that created a line may call lineDestroy() on it — lineCreate() or lineCreateForWorker() for that exact line_t instance.

Ownership belongs to the line, not to the node type. Do not infer it from a name such as "Client", "Server", "Listener", or "Connector":

• TcpListener calls lineCreate() for every accepted connection and is that line's owner.
• TcpConnector owns a socket, not a line. It calls neither lineCreate() nor lineDestroy(); it consumes a line created further up the chain. Owning an OS resource is not owning a line_t.
• A tunnel routinely mixes roles: MuxClient owns its parent/carrier line while borrowing every child line; HttpClient owns the upload/download transport lines it splits a borrowed main line into; Socks5Server, TrojanServer, and VlessServer own UDP remote lines while borrowing the client line. Decide per line, in the same handler, before cleaning anything up.

tests/line_ownership_policy_test.py records the classification of every production creation site and fails when a new one appears unclassified.

Every other tunnel must propagate Finish and let the owner destroy the line. Calling lineDestroy() on a line you did not create is a serious bug.

There is also a debug invariant to respect: when a line is finally freed, lineUnRefInternal() asserts that every tunnel's line state is already zeroed (debugAssertZeroBuf). That is why each tunnel must destroy its own line state before the real line-closing Finish propagates — unless it is intentionally holding a positive reference across the close.

Protecting a Line Across Re-entrant Callbacks

These inter-tunnel calls can indirectly close the line before they return. Treat every one of them as dangerous:

tunnelNextUpStreamInit       tunnelNextUpStreamPayload
tunnelPrevDownStreamInit     tunnelPrevDownStreamPayload
tunnelNextUpStreamEst        tunnelNextUpStreamPause      tunnelNextUpStreamResume
tunnelPrevDownStreamEst      tunnelPrevDownStreamPause    tunnelPrevDownStreamResume

After any of these returns:

• the line may already be dead,
• this tunnel's line state may already be destroyed and zeroed,
• any code that touches line or ls is unsafe unless you protected the line.

Never reason: "the call returned, so my line state is still valid."

Method A — withLineLocked() (preferred)

withLineLocked() and withLineLockedWithBuf() hold a temporary reference across the callback and tell you whether the line survived. Their actual implementation (see ww/net/line.h) is:

static inline bool withLineLocked(line_t *const line, LineTaskFnNoBuf task, tunnel_t *t)
{
    lineLock(line);
    task(t, line);
    if (! lineIsAlive(line))
    {
        lineUnlock(line);
        return false;   // line died during the callback
    }
    lineUnlock(line);
    return true;        // line still alive; safe to continue
}

Use it whenever you must keep working with the line after a re-entrant call:

if (! withLineLocked(line, tunnelNextUpStreamInit, t))
{
    return;   // line died; do not touch line or ls
}

// Still alive here. Re-read state if you need it.
my_lstate_t *ls = lineGetState(line, t);

For payload forwarding, use the buffered variant:

if (! withLineLockedWithBuf(line, tunnelNextUpStreamPayload, t, buf))
{
    return;
}

When a helper returns false:

• do not touch line,
• do not touch this tunnel's line state,
• do not call your LinestateDestroy() "just to clean up" — the close path that killed the line already did that,
• if you still own a buffer you did not hand off, recycle it without touching the dead line state.

When you do not need the wrapper

If you call a forwarding helper and then immediately return with no further work on the line, wrapping it in withLineLocked() buys you nothing — the boolean result would not change your behavior. This is common for plain Init and Finish pass-throughs.

Method B — Manual lineLock() / lineUnlock()

Use manual locking when you need explicit control across several calls:

lineLock(line);

// one or more potentially re-entrant callbacks ...

if (! lineIsAlive(line))
{
    lineUnlock(line);
    return;
}

// still alive: safe to continue
lineUnlock(line);

Rules: always unlock what you lock; never read tunnel state after destroying it; if a helper both destroys your line state and propagates Finish, the caller must return immediately afterward.

Finish Semantics

Finish is directional and destructive, and it has one property that makes it special among all the callbacks:

Finish is guaranteed non-re-entrant

All other flow callbacks may be re-entrant. Finish is the exception. If Tunnel A sends Finish to Tunnel B for a line, Tunnel B must not call any callback (including Finish) back in that same direction for that line_t.

Concretely:

• A tunnel that receives upstream Finish must not send tunnelPrevDownStream* (Init/Payload/Est/Pause/Resume/Finish) back toward the sender for that line.
• A tunnel that receives downstream Finish must not send tunnelNextUpStream* back toward the sender for that line.

This is called reflection, and it is a frequent crash source precisely because tunnels destroy their line state before propagating Finish — reflecting a callback calls into a tunnel whose state for that line is already gone.

Turn the guarantee around and it becomes a simple, load-bearing rule: because Finish is non-re-entrant, you are promised you will not be called back on the finished side — so you must extend the same promise and send nothing back. The instant you receive Finish from a direction, that direction is closed for that line. No Payload, no Est, no Pause/Resume, no second Finish, for any reason, on any later event for that line_t.

Also remember the destruction direction: a downstream Finish that reaches the adapter which created the line will lead to lineDestroy(). So after you propagate a downstream Finish, assume the line can be gone.

The Simplest Clean Finish

A middle tunnel with nothing to flush just destroys its own line state and forwards. This is EncryptionClient's upstream finish, verbatim:

void encryptionclientTunnelUpStreamFinish(tunnel_t *t, line_t *l)
{
    encryptionclient_lstate_t *ls = lineGetState(l, t);
    encryptionclientLinestateDestroy(ls);   // destroy local state first
    tunnelNextUpStreamFinish(t, l);          // then propagate
}

Order matters: destroy local state before propagating the real close. Do not read any field of ls after LinestateDestroy(ls) — assume it is zeroed.

Closing Both Directions From the Middle

When a middle tunnel itself decides to tear down the connection (a protocol error, a limit hit), it is responsible for closing both sides. The safe ordering is:

1. Destroy this tunnel's own line state.
2. Send the upstream Finish first. (Finish is non-re-entrant, so the line is still alive after this.)
3. Send the downstream Finish second. (This one can lead to the line dying.)
4. Return immediately.

my_lstate_t *ls = lineGetState(l, t);
myLinestateDestroy(ls);          // 1. local state gone

tunnelNextUpStreamFinish(t, l);  // 2. close forward (line still alive)
tunnelPrevDownStreamFinish(t, l); // 3. close backward (line may die now)
return;                          // 4. do not touch l or ls again

Adapters are simpler: sitting at one end of the chain, they only ever send Finish once, toward the opposite direction.

Lines you own are different again, and they carry a stronger, checkable obligation — the next section.

The Owner Finish Postcondition

When a tunnel receives Finish for a normal line it created, that handler must not return while the line is still logically alive:

/* on return from the owner's Finish handler for a normal line */
assert(! lineIsAlive(line));

The postcondition is about logical death only. Outstanding lineLock() references may keep the allocation present until suspended or re-entrant frames unwind; the memory does not have to be freed before the handler returns.

Before returning, an owner must:

1. Detach every producer that can still target the line — io callbacks, timers, idle-table items, lookup maps, queues, scheduled tasks.
2. Complete whatever cleanup is legal for the direction that sent Finish.
3. Destroy its own line state exactly once.
4. Propagate Finish only where the topology requires, and only away from the sender — the no-reflection rule is unchanged by ownership.
5. Call lineDestroy(line) unless a nested path already did.
6. Return immediately, touching no destroyed state.

A creator such as MuxClient does exactly that. The excerpt below is muxclientTunnelDownStreamFinish() — the fnFinD slot, so next is the sender. The parent's children hang off the prev side, and prev is therefore the only direction anything may travel:

/* fnFinD on the parent line MuxClient created; next finished us. */
while (child_ls)
{
    /* ... */
    muxclientLinestateDestroy(child_ls);
    tunnelPrevDownStreamFinish(t, child_l);   // away from the sender
    /* ... */
}

muxclientLinestateDestroy(parent_ls);
lineDestroy(parent_l);                        // only the owner may do this

The general shape, with the copy-to-stack step that keeps the code safe after LinestateDestroy(). Read the direction off the callback slot the handler is installed in — this one is fnFinU, so prev sent it and next is the only side that may be told:

void ownerUpStreamFinish(tunnel_t *t, line_t *l)
{
    owner_lstate_t *ls = lineGetState(l, t);

    ownerDetachResources(t, l, ls);                  /* 1 */

    bool propagate_away = ls->next_init_sent;        /* copy before destroying */
    ownerLinestateDestroy(ls);                       /* 3 */

    if (propagate_away)
    {
        tunnelNextUpStreamFinish(t, l);              /* 4 */
    }
    if (lineIsAlive(l))
    {
        lineDestroy(l);                              /* 5 */
    }
    return;                                          /* 6 */
}

The fnFinD mirror is the same function with tunnelPrevDownStreamFinish() in step 4. An endpoint owner — a chain head such as TcpListener or UdpStatelessSocket — has no prev at all, so its fnFinD propagates nothing and step 4 disappears. The postcondition is unchanged either way.

Why it matters. Without it, this sequence corrupts memory: a middle tunnel forwards a re-entrant callback; a nested Finish destroys that middle tunnel's per-line state; the line owner receives the propagated Finish but leaves the line alive; the outer callback then reads lineIsAlive(line) == true and resumes against state that no longer exists. The postcondition makes lineIsAlive() an honest post-callback signal.

Borrowed lines are the mirror image. Destroy only your own per-line state, propagate the same directional Finish away from the sender when the topology requires it, and never call lineDestroy() — the real owner's destroy must stay valid. Never reflect Finish, payload, Est, Pause, or Resume back toward the side that finished you.

A middle tunnel that also owns the line still follows the both-directions ordering above for its local close, and then the owner postcondition applies on top: the line must be dead before that close frame returns.

requestProgramShutdown() does not satisfy the postcondition. It schedules global teardown on worker 0; the current worker callback returns and unwinds through every suspended frame first. An owner whose Finish reaches a terminal failure must still detach, destroy its state, and mark the line dead — and then request the shutdown.

Deferred OS cleanup is not an excuse to stay alive. If a resource needs asynchronous teardown, detach it from the line first and give the deferred cleanup object its own copy of whatever it needs. A live line is not a cleanup token.

Packet lines are excluded. They are never subject to the owned-normal-line postcondition and must never be destroyed at runtime. A Finish on one is not automatically a contract violation: a packet-lifecycle anchor may reject it, a transparent middle transform forwards it in the same direction, and an intentional terminal absorber may absorb it when there is nothing onward to notify or release. The exact handler role and direction decide which category applies — see Part 4.

Most Tunnels Need No Directional-Finish State

A direct consequence of non-re-entrancy: do not give a tunnel prev_finished / next_finished (or can_upstream / can_downstream) flags by default. A tunnel that simply forwards Finish is guaranteed never to be called back on the finished side, so it has nothing to remember. A speculative "did this side finish?" boolean is almost always a control-flow problem being patched instead of fixed — the same anti-pattern as an initialized flag.

There is exactly one situation that justifies directional-close state: the tunnel must send final bytes before it closes (the next subsection). That send re-enters the adapter, which can emit Pause/Resume, and the flag is what lets your pause/resume handlers drop anything that would reflect toward the already-finished side. The flag is bookkeeping for that hazard and nothing else.

Real tunnels carry exactly this state and no more. The names differ; the role is identical:

Tunnel	Directional-close state	Why it needs it
TlsServer	upstream_finished, downstream_finishing	Sends TLS alert / close and resolves fallback before closing
TcpOverUdpClient	can_downstream	Must finish its reliable-stream teardown before closing
TcpOverUdpServer	can_upstream	The same, in the opposite direction
HttpClient, HttpServer, Socks5Server	prev_finished, next_finished	Send final protocol bytes before transport close

If your tunnel does not send final bytes during Finish, it should not carry any of these flags. Destroy local state and forward — that is the whole job.

Final Protocol Bytes Before Closing

Some protocols must emit closing bytes before the transport closes: TLS close_notify, an HTTP final chunk, HTTP/2 END_STREAM, a WebSocket close frame, a KCP close frame. Sending those bytes during Finish opens a dangerous re-entrant window.

The hazard, step by step:

1. Tunnel receives Finish from side A.
2. It sends final protocol bytes toward side B.
3. Side B's adapter blocks while writing and emits Pause (or later Resume).
4. That Pause/Resume travels back through the tunnel...
5. ...and if the tunnel reflects it toward side A — whose state was already
   destroyed by the original Finish — it calls into freed/zeroed state. Crash.

The required safe shape:

1. lineLock(line) — you are about to send bytes re-entrantly.
2. Mark the sender side finished first, using the tunnel's existing directional state (prev_finished, next_finished, can_downstream = false, can_upstream = false, etc.). Your Pause/Resume handlers must check this and refuse to reflect toward a finished side.
3. Send the final protocol bytes toward the other side.
4. Destroy this tunnel's local line state.
5. Propagate the real WaterWall Finish.
6. lineUnlock(line).
7. Return immediately. Do not touch local line state afterward.

Direction examples:

• Handling upstream Finish, sending final bytes upstream to next: any downstream Pause/Resume caused by that send must not be forwarded back to prev.
• Handling downstream Finish, sending final bytes downstream to prev: any upstream Pause/Resume caused by that send must not be forwarded back to next.

Do not invent a new initialized flag for this. Reuse the tunnel's existing directional finish/close state, or — if the tunnel genuinely lacks one and this is the final-bytes case — add an explicit directional close flag. As stressed above, this is the only place such a flag belongs.

Adapters Flush On Their Own

Adapters such as TcpConnector and TcpListener may flush queued bytes before they actually close the socket after receiving Finish. So a middle tunnel should: send its final protocol bytes, destroy local state, forward Finish — and then trust the adapter to flush. Do not keep protocol state alive just to "wait for the socket to drain."

Est, Pause, and Resume

• Est signals that the downstream side is genuinely established. Do not emit it early. A tunnel that needs to know when its line is up can also check lineIsEstablished() / lineMarkEstablished() (see ww/net/line.h).
• Pause / Resume are backpressure signals. Emit them only when semantically correct and directionally consistent, and never reflect them back toward a finished direction (see above). Do not emit them casually; spurious pause/resume can deadlock a stream.

Worked Example: Tracing a Payload

Consider the chain:

TcpListener -> ObfuscatorClient -> TlsClient -> TcpConnector
   (head)         (middle)          (middle)      (tail)

Upstream (client to remote):

client bytes arrive at TcpListener's socket
  TcpListener: tunnelNextUpStreamPayload(t, line, buf)
    -> ObfuscatorClient.fnPayloadU: scramble bytes, forward upstream
      -> TlsClient.fnPayloadU: encrypt via BoringSSL, forward upstream
        -> TcpConnector.fnPayloadU: write ciphertext to the remote socket

Downstream (remote to client):

remote bytes arrive at TcpConnector's socket
  TcpConnector: tunnelPrevDownStreamPayload(t, line, buf)
    -> TlsClient.fnPayloadD: decrypt, forward downstream
      -> ObfuscatorClient.fnPayloadD: unscramble, forward downstream
        -> TcpListener.fnPayloadD: write plaintext to the accepted client

Notice that the same TlsClient instance handles encrypt on fnPayloadU and decrypt on fnPayloadD. The direction, not the tunnel, decides the transformation. If TlsClient's upstream payload write blocks at TcpConnector, backpressure flows back as a Pause in the downstream direction — and must not be reflected past a side that has already finished.

Checklist for This Part

Before you call a flow change done:

• Does Init initialize this tunnel's line state before any callback can use it?
• Are upstream callbacks forwarded only with tunnelNextUpStream*, and downstream only with tunnelPrevDownStream*?
• Does every re-entrant call either return immediately or protect the line with withLineLocked() / manual lock + lineIsAlive()?
• On a false from withLineLocked(), do you avoid touching line, ls, and LinestateDestroy()?
• Does Finish destroy local state before propagating the real close?
• When closing from the middle, do you finish upstream first, then downstream, then return?
• If you send final protocol bytes during Finish, do you mark the sender side finished before the send and block reflected Pause/Resume?
• Does only the line owner ever call lineDestroy()?
• Did you classify each line as owned normal, borrowed normal, or packet before editing its Finish path?
• Does every owner Finish path for a normal line return with ! lineIsAlive(line) — including the error, timeout, and terminal-verdict branches?
• Are io callbacks, timers, idle items, and map entries detached before the owner's state and line are destroyed?
• Does a borrowed-line path avoid lineDestroy() entirely?
• Is requestProgramShutdown() used in addition to closing an owned line, never instead of it?
• Did you avoid adding an initialized flag the source does not require?

---

Continue to Part 3: Buffers, Padding & Shift Buffers.