AuthenticationClient

AuthenticationClient is the frontend side of the WaterWall authentication database protocol. It creates and owns one internal control line on worker 0, sends that line upstream to its next node, authenticates with an AuthenticationServer, pulls the authoritative users table, and periodically keeps the session alive and pushes local traffic counters.

AuthenticationClient is not a data pass-through tunnel. Traffic-serving nodes use its internal API to read users and account local usage against the pulled users table.

Typical placement:

AuthenticationClient -> ... -> AuthenticationServer

For a remote server, the client-side chain commonly looks like:

AuthenticationClient -> TcpConnector

The connector path must reach a server-side chain that ends in AuthenticationServer.

What It Does

• Creates one long-lived internal control line on worker 0.
• Sends Authenticate to AuthenticationServer using configured client credentials.
• Stores the returned 64-byte session token.
• Pulls the full users table with GetAllUsers after authentication.
• Keeps a local, lock-protected users table for other tunnels.
• Preserves unsynced local traffic counters across full table refreshes.
• Sends periodic authenticated Ping requests.
• Sends periodic PushUserStats requests with traffic-counter hints.
• Sends periodic or revision-driven GetAllUsers requests.
• Reconnects and reauthenticates if the downstream transport closes.
• Exposes user lookup, JSON copy-out, traffic accounting, and admission helpers through AuthenticationClient/interface.h.

Because it creates the control line, AuthenticationClient is also responsible for destroying that line.

Configuration Example

Minimal in-process authentication chain:

{
  "name": "auth-client",
  "type": "AuthenticationClient",
  "settings": {
    "name": "edge-1",
    "secret": "long-random-secret"
  },
  "next": "auth-db"
}

Remote authentication server chain:

{
  "name": "auth-client",
  "type": "AuthenticationClient",
  "settings": {
    "name": "edge-1",
    "secret": "long-random-secret",
    "ping-interval-ms": 60000,
    "pull-interval-ms": 300000,
    "push-interval-ms": 300000,
    "reconnect-interval-ms": 5000,
    "request-timeout-ms": 120000,
    "max-pending-requests": 64,
    "verbose": false
  },
  "next": "auth-server-connector"
}

{
  "name": "auth-server-connector",
  "type": "TcpConnector",
  "settings": {
    "address": "127.0.0.1",
    "port": 9000,
    "nodelay": true
  }
}

The settings.name and settings.secret pair must match one AuthenticationServer settings.auth-clients entry:

{
  "name": "edge-1",
  "secret": "long-random-secret",
  "allow-user-pull": true,
  "allow-stats-push": true,
  "allow-user-write": false
}

Required Fields

Top-level fields:

Field	Type	Description
name	string	User-chosen node name. Other nodes use this value to reference the auth client.
type	string	Must be exactly "AuthenticationClient".
next	string	Required. The next node must lead to an AuthenticationServer.
settings	object	Authentication credentials and timer settings.

Required settings fields:

Field	Type	Description
name	non-empty string	Control-plane client name sent to AuthenticationServer.
secret	non-empty string	Control-plane client secret sent with settings.name.

The control-plane name and secret are not traffic-user credentials.

Optional Settings

Setting	Type	Default	Description
ping-interval-ms	zero or positive integer	60000	Sends authenticated Ping; also retries Authenticate while connected but unauthenticated.
pull-interval-ms	zero or positive integer	300000	Periodic GetAllUsers cadence. 0 disables timer-driven pulls.
push-interval-ms	zero or positive integer	300000	Periodic PushUserStats cadence. 0 disables periodic pushes.
reconnect-interval-ms	zero or positive integer	5000	Delay before reconnecting after the control transport closes.
request-timeout-ms	zero or positive integer	120000	Reconnects if a pending request waits longer than this. 0 disables request timeout checks.
max-pending-requests	positive integer	64	Maximum number of in-flight protocol requests.
verbose	boolean	false	Enables focused control-line, timer, request, and response debug logs.

pull-interval-ms only disables timer-driven pulls. The client still pulls once after successful authentication, and it pulls again when the server response says the local copy needs refresh.

push-interval-ms only disables periodic pushes. Other tunnels can still ask for a push through the internal API.

Using It From Other Nodes

AuthenticationClient is usually referenced by node name. For example, Socks5Server can use an existing AuthenticationClient like this:

{
  "name": "socks-server",
  "type": "Socks5Server",
  "settings": {
    "auth-client-node-name": "auth-client",
    "connect": true,
    "udp": false
  },
  "next": "outbound-tcp"
}

AuthenticationClient is not placed in the Socks5Server data path. It keeps its own control connection and exposes the pulled user table through its internal API.

Communication Model

The client uses one long-lived control line, not one line per request.

On startup, worker 0 creates a single internal line_t and sends upstream init to the next tunnel. In a chain such as AuthenticationClient -> TcpConnector, that one WaterWall line becomes one outbound transport connection to the server-side chain.

All authentication, ping, pull, and push requests are framed over the same line. Responses are matched to requests by correlation ID.

If the transport closes, downstream finish closes and destroys the owned control line, clears the active token, clears pending request state, and schedules reconnect unless the tunnel is stopping. The next connection starts with a new Authenticate request; the old token is not reused.

While disconnected or unauthenticated, new protocol requests are not sent. The local users table remains available as a cache, but synchronization resumes only after reconnect, transport est, and successful authentication.

Startup Flow

On onStart, the tunnel queues startup work to worker 0.

Worker 0 then:

1. Marks the client as started.
2. Arms the ping timer when enabled.
3. Starts the sync timer when pull or push is enabled.
4. Creates the owned control line.
5. Initializes this tunnel's per-line state.
6. Sends tunnelNextUpStreamInit().

When the next side establishes the transport and sends downstream est, the client marks the control line connected and sends Authenticate with a zero session token.

When Authenticate returns a session response, the 64-byte token becomes the active token for later requests, and the client immediately sends GetAllUsers.

Timers

All protocol timers live on worker 0, the same worker as the owned control line.

Timer	Behavior
ping_timer	Sends Ping while authenticated and retries Authenticate while connected but unauthenticated.
sync_timer	Drives user-table pull and stats push using the shortest enabled pull/push interval.
reconnect_timer	Opens a new control line after downstream transport close.

Each sync tick attempts PushUserStats first when due, then sends GetAllUsers when revisions differ or when the pull cadence is due and no newer equal-revision response has already confirmed the local table is current.

Timers are deleted from onWorkerStop for worker 0, because timer deletion must happen on the same worker event loop that owns the timer.

Local Users Table

The client keeps one active local users table protected by a tunnel-level wrwlock_t.

GetAllUsers replacement is atomic from the point of view of other tunnels:

1. Response JSON is parsed into a fresh users_t.
2. The fresh table is validated, including rejecting overlapping wireguard-allowed-ips ranges.
3. Unsynced local traffic deltas and process-local runtime counters are carried forward by durable user id when present, with SHA-256 as a legacy fallback.
4. Each user's local expiry deadline is projected from server-owned time fields and response server-time-ms.
5. The active table pointer is swapped under the write side of the rwlock.
6. The generation counter is incremented.
7. The old table is destroyed after the swap.

Readers hold the read side of the same rwlock while copying JSON, copying stats, or updating traffic by SHA-256. This prevents a pointer inside the current users_t from becoming invalid during an operation.

The local table is a cache plus local counter accumulation. The server remains the source of truth for full user configuration, including optional wireguard-allowed-ips, and for first-usage-at-ms.

Internal API

Other tunnels include:

#include "AuthenticationClient/interface.h"

This internal WaterWall API is separate from the external api.c tunnel API.

Other tunnels must not receive or cache raw user_t * pointers from AuthenticationClient. Instead they use user_handle_t, which is a value identifier containing:

sha256(password)
users_generation
user_id

user_id is copied from the synced user record and may be 0 for legacy users. When it is present, live enforcement helpers resolve by durable user id, with SHA-256 as a fallback.

Common state helpers:

authenticationclientGetState(t)
authenticationclientIsReady(t)
authenticationclientUsersGeneration(t)

Lookup helpers:

authenticationclientGetUserByPassword(t, password, &handle)
authenticationclientGetUserByPasswordWithResult(t, password, &handle)
authenticationclientGetUserBySHA224(t, sha224, &handle)
authenticationclientGetUserBySHA256(t, sha256, &handle)
authenticationclientGetUserByUUID(t, uuid, &handle)
authenticationclientGetUserByWireGuardPublicKey(t, publickey, &handle)
authenticationclientGetUserByPasswordWithProfile(t, password, &handle, &profile)
authenticationclientGetUserBySHA224WithProfile(t, sha224, &handle, &profile)
authenticationclientGetUserByUUIDWithProfile(t, uuid, &handle, &profile)
authenticationclientGetUserByWireGuardPublicKeyWithProfile(t, publickey, &handle, &profile)
authenticationclientUserProfileClear(&profile)

Copy-out and stats helpers:

authenticationclientUserHandleIsLive(t, &handle)
authenticationclientUserToJson(t, &handle)
authenticationclientUsersToJson(t)
authenticationclientUserGetStats(t, &handle, &stats)
authenticationclientUserAddTraffic(t, &handle, upload, download)

Live enforcement helpers:

authenticationclientUserTryAdmitConnection(t, &handle, &ip_key, now_ms)
authenticationclientUserReleaseConnection(t, &handle, &ip_key)
authenticationclientUserAccountTraffic(t, &handle, upload, download, now_ms)
authenticationclientUserShouldClose(t, &handle, now_ms)

Manual sync helpers:

authenticationclientRequestPull(t)
authenticationclientRequestPush(t)

authenticationclientUserToJson() and authenticationclientUsersToJson() return new cJSON objects owned by the caller. Stats are copied into caller storage. Traffic-serving nodes should never mutate a user_t directly.

The now_ms arguments for live enforcement helpers are in the client's local monotonic clock domain. Server expiry fields are projected into that domain when GetAllUsers is installed.

Client States

authenticationclientGetState() returns:

State	Meaning
kAuthenticationClientStateStopped	The client is stopped or unavailable.
kAuthenticationClientStateConnecting	The client has started but has no authenticated control session yet.
kAuthenticationClientStateAuthenticating	The transport is connected or token is present, but users are not fully loaded yet.
kAuthenticationClientStateReady	The client is authenticated and has installed a users table.

authenticationclientIsReady() is true only in kAuthenticationClientStateReady.

Password lookup with result can return:

Result	Meaning
ok	Lookup succeeded.
invalid password lookup	Bad arguments, empty password, or missing output handle.
password hash failed	SHA-256 calculation failed.
users table unavailable	The local users table is not loaded.
user not found	No user matches the password.
password mismatch	Hash matched but plaintext password check failed.
user disabled	The matching user is disabled.
user expired	The matching user is expired in the local client-time view.
user limit reached	The matching user has reached a configured limit.

Wire Protocol

The protocol matches AuthenticationServer.

Request message:

u32 body_len
u8[64] session_token
request_frame

Request frame:

u8 request_type
u32 correlation_id
u32 request_data_len
u8[request_data_len] request_data

Response message:

u32 body_len
u64 config_revision
u64 stats_revision
response_frame...

Response frame:

u8 response_type
u32 correlation_id
u32 response_data_len
u8[response_data_len] response_data

All integer fields are big-endian. The client sends one request frame per message. The parser accepts any number of response frames in one response message and matches each frame by correlation ID.

The response revisions are remembered as the latest server revisions and compared with local users-table revisions before timer-driven pulls.

Stats Push

PushUserStats is a partial hint update. The client sends only the fields the server needs:

{
  "users": [
    {
      "password": "user-password",
      "stats": {
        "traffic": {
          "up": "12345",
          "down": "67890"
        }
      }
    }
  ]
}

Users without changed traffic counters are skipped for that push.

The server compares these counters against the session baseline created by GetAllUsers and applies only deltas. The client does not send time.first-usage-at-ms; when the first non-zero traffic delta arrives for a user whose authoritative first-usage time is still missing, the server stamps that field using its own clock.

authenticationclientUserAccountTraffic() also marks a runtime-only per-user flag on the first non-zero local traffic accounting while first usage is still missing. The client queues a worker-0 PushUserStats attempt, defers one follow-up push if a stats push is already in flight, and coalesces later payload calls until the pushed usage is followed by a fresh GetAllUsers view or the send attempt fails.

If the server says needs-pull: true, the client requests GetAllUsers again to refresh the local cache and server-side session baseline.

Lifecycle and Direction

AuthenticationClient is a chain-head control node. It sends requests upstream:

tunnelNextUpStreamInit()
tunnelNextUpStreamPayload()
tunnelNextUpStreamFinish()

It receives server responses through downstream callbacks:

DownStreamEst
DownStreamPayload
DownStreamPause
DownStreamResume
DownStreamFinish

Pause and resume are not reflected to a previous tunnel because AuthenticationClient has no previous data producer. They only track whether new protocol requests may be sent immediately.

The control line's line state contains the downstream response read stream. It is initialized before tunnelNextUpStreamInit() and destroyed before the client propagates upstream finish or destroys the owned line.

On downstream finish, the client:

1. Removes the active control-line pointer under the control mutex.
2. Clears session token, authenticated state, in-flight flags, and pending requests.
3. Destroys this tunnel's line state.
4. Destroys the owned control line.
5. Schedules reconnect unless the tunnel is stopping.

When the client closes the control line itself, such as during worker-0 shutdown or malformed response handling, it destroys local line state first, sends tunnelNextUpStreamFinish(), and then destroys the owned line if it is still alive.

Notes and Caveats

• AuthenticationClient requires a next node.
• It is a control-plane node, not a data pass-through node.
• It owns and destroys only the internal control line it created.
• It does not use packet lines and is not a packet tunnel.
• The request data, response payload, and outer message payload limits are 16 MiB.
• Secrets, session tokens, passwords, and raw users JSON are intentionally not printed by verbose logging.
• The module name UpdateUserTraficStatsDiff intentionally follows the current source spelling.

User Database Performance

• Plaintext-password lookups derive SHA-256 once and resolve through the SHA-256 hash index with no fallback scan, so hits and misses are average O(1). The candidate is then confirmed with an exact plaintext comparison, so a hypothetical SHA-256 collision fails closed.
• The local sync baseline is duplicated with the native usersCopy() deep copy rather than a JSON round trip. Only the received GetAllUsers network response still requires one JSON parse into the active table.