# SignalR Binary Protocol — VarUInt Frame Marker

Implementation plan for replacing the fixed-width `[201][UINT16 size][data]` chunk-data
frame with a unified VarUInt-based marker scheme. Single-grammar dispatch: the first
VarUInt encodes either a sentinel marker (1-4) or the chunk-data body length (≥5).

> Related: `SIGNALR_BINARY_PROTOCOL_ISSUES.md` (B7K9 cap, Z2X9 cancel-recovery),
> `../../../../AyCode.Core/AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_ISSUES.md`
> (streaming I/O layer issues).

## Motivation

Three coordinated wins from a single wire-format change:

1. **Wire-size reduction across all typical chunk-size tiers** — `-1 byte / chunk`
   on the default 4 KB config, `-2 byte / chunk` on small (<128 B) chunks.
2. **65 KB single-value cap removed** (`ACCORE-BIN-I-B7K9`) — the UINT16 wire-field
   constraint is gone; large string/byte[] property values fit a single chunk via
   wider VarUInt encoding (up to ~4 GB / chunk).
3. **CHUNK_ABORT becomes a real wire marker** (`ACCORE-BIN-I-Z2X9`) — the
   currently transport-layer-only abort signal moves to the wire format, enabling
   in-band cancel/timeout recovery without TCP-disconnect ceremony.

The hot-path frame decode collapses to a single VarUInt-read plus one branch
(`first >= 5 → CHUNK_DATA, body length = first`), the dominant case on any
typical workload.

## Wire-format scheme

```
VarUInt(1)   → CHUNK_START   (followed by the standard SignalR length-prefixed message body)
VarUInt(2)   → CHUNK_END     (no body — end-of-message)
VarUInt(3)   → CHUNK_ABORT   (no body — sender-side cancel; receiver discards in-progress message)
VarUInt(4)   → CHUNK_DATA    (followed by a separate VarUInt body length, 0..4 byte body)
VarUInt(N≥5) → CHUNK_DATA    (body length = N, body bytes follow)
```

Notes:
- `VarUInt(0)` is **never valid on the wire**. A decoded 0 signals stream corruption
  → defensive `throw InvalidDataException`. One `cmp` instruction in the reader.
- `VarUInt(4)` is the explicit small-body CHUNK_DATA marker — covers `body length ∈ [0..4]`
  which would otherwise collide with the sentinel range. Adding 1 byte for marker
  but enabling 0..4-byte bodies (keepalive / flush / tiny payload) is still a net
  size win vs the legacy 3-byte header.
- The marker space (1-4) and the body-length space (≥5) share the same VarUInt
  grammar — the reader does not need to branch between "marker-byte then length"
  vs "length-byte"; one read, one switch.

### CHUNK_START semantics (unchanged from current)

After `VarUInt(1)`, the bytes that follow are the standard SignalR length-prefixed
message envelope: `[INT32 LE payloadLength][payload bytes]`. The payload itself is
the InvocationMessage / CompletionMessage / etc. encoded per the existing rules
(streamed-arg-placeholder marker `INT32 -1`, args, headers). No change to that
inner structure.

### Outer SignalR transport prefix (unchanged)

The transport-level `[INT32 LE payloadLength]` that Kestrel / WebSocket pipes
expect at the head of every hub-protocol message **stays** — this is a SignalR
protocol contract, not part of the AcBinary chunk framing. The VarUInt scheme
described here lives **inside** that payload.

## Size matrix — old vs new

| Body length | Legacy (`[201][UINT16 size]`) | New | Δ |
|---|---|---|---|
| 0..4 | 3 byte | 2 byte (marker `4` + VarUInt 0..4) | **−1** |
| 5..127 | 3 byte | 1 byte (VarUInt 1-byte tier) | **−2** |
| 128..16 383 | 3 byte | 2 byte (VarUInt 2-byte tier) | **−1** |
| 16 384..65 535 | 3 byte | 3 byte (VarUInt 3-byte tier) | same |
| 65 536..2 097 151 | **not representable** | 3 byte | new capability |
| 2 097 152..2³² − 1 | **not representable** | 4-5 byte | new capability |

Realistic chunk-size configurations:

| Chunk-size | Body length tier | New header bytes | Δ vs legacy 3-byte |
|---|---|---|---|
| Tiny 64 B | 1-byte VarUInt | 1 | **−2** |
| Default 4 KB | 2-byte VarUInt | 2 | **−1** |
| Large 32 KB | 2-byte VarUInt | 2 | **−1** |
| Maximum 64 KB (legacy cap) | 3-byte VarUInt | 3 | same |
| > 64 KB (new tier) | 3-5 byte VarUInt | 3-5 | enables large single-chunk values |

The default 4 KB config — the typical SignalR deployment — saves 1 byte per chunk.
A 275 KB payload at 4 KB chunks ≈ 68 chunks → **−68 byte / message** wire overhead
reduction. Per-message marginal, but cumulative on high-throughput streams.

## Reader hot path

```csharp
var first = context.ReadVarUInt();

if (first >= 5)
{
    // CHUNK_DATA fast path — body length = first
    ProcessChunkData(bodyLength: (int)first);
    return;
}

// Rare branches — sentinel markers + small-body CHUNK_DATA
switch (first)
{
    case 1: HandleChunkStart(); break;
    case 2: HandleChunkEnd(); break;
    case 3: HandleChunkAbort(); break;
    case 4:
        var length = context.ReadVarUInt();
        ProcessChunkData(bodyLength: (int)length);
        break;
    default:
        // first == 0 — corruption signal
        throw new InvalidDataException("Invalid AcBinary chunk frame marker (0).");
}
```

The `first >= 5` branch is the dominant hot path (>99% on typical workloads,
because chunk sizes are configured to ≥ 64 B and almost always exceed 5 bytes
of body content). One branch + one VarUInt read — minimal frame-decode cost.

VarUInt-decode cost vs the legacy `[201][UINT16]` fixed read: marginally higher
on small-tier (1-2 byte VarUInts), comparable on 2-3 byte VarUInts. The wire-size
savings on the network compensate for any single-digit-nanosecond decode-time
difference. Benchmark validation recommended at impl time but not expected to
gate the change.

## Cross-cutting issue resolutions

The wire-format change closes two pre-existing open issues without separate fixes.

### `ACCORE-BIN-I-B7K9` — 65 KB single-value cap removed

The legacy `[201][UINT16 size]` field caps any single chunk's data payload at
65 535 bytes. The issue documents the workaround direction (producer-side
"transparent split-on-commit" with `ArrayPool`-rented owned buffer) as a complex
refactor. The new VarUInt scheme makes that **unnecessary** — a single chunk
can carry up to ~4 GB of body, so any practical large string / byte[] property
fits without splitting.

The proposed B7K9 producer-side refactor is therefore **scope-eliminated** by
this change. The current temporary sanity guard
(`BinarySerializationContext.EnsureCapacity` → `GrowAndValidate` →
`ThrowGrowFailedToSatisfy`) can stay as a defensive check, but the underlying
cap it surfaces no longer exists.

### `ACCORE-BIN-I-Z2X9` — Built-in cancel/timeout recovery

The current wire-format has no marker for "abort current message, restart fresh".
The issue documents that:
> If a sender starts writing a multi-chunk message and then aborts (no [202]
> follows), the receiver's framing state machine is stuck in `AwaitingData` ...
> When the next message's [201] header byte arrives, the state machine
> interprets it as data — silent corruption follows.

The new `VarUInt(3) → CHUNK_ABORT` marker provides the in-band abort signal:

- Sender mid-message → emits `VarUInt(3)` → receiver resets its framing state to
  `AwaitingHeader`, discards the in-progress message buffer, fires the abort
  callback (`AyCodeBinaryHubProtocol.OnChunkAbort` already exists for this).
- No transport-disconnect needed. The connection stays open for the next message.

This resolves Z2X9 without a separate fix and removes the
"transport-layer-recovery is the contract" workaround from the receiver.

## Implementation scope

| File | Change |
|---|---|
| `AyCode.Services/SignalRs/AcBinaryHubProtocol.cs` | Constants: replace `MsgAsyncChunkStart=200` / `MsgAsyncChunkEnd=202` byte constants with VarUInt-encoded sentinels (1, 2). Add `MsgAsyncChunkAbort=3`. Add `MsgAsyncChunkDataSmall=4`. `WriteMessageChunked` — emit start/end via VarUInt. `TryParseChunkData` / `TryParseMessage` — VarUInt-based dispatch. |
| `AyCode.Core/Serializers/Binaries/AsyncPipeWriterOutput.cs` | CHUNK_DATA frame emit: replace `WriteByte(201) + WriteUInt16LittleEndian(size)` with `WriteVarUInt(size)` for the `≥5` fast path, `WriteVarUInt(4) + WriteVarUInt(size)` for the small-body path. `[202]` end-marker → `WriteVarUInt(2)`. Add `WriteVarUInt(3)` ABORT-emit path. |
| `AyCode.Core/Serializers/Binaries/AsyncPipeReaderInput.cs` | `Feed` framing state machine: replace `[201/202]` byte-marker dispatch with VarUInt-decode + sentinel/length branch. Defensive `VarUInt(0) → throw`. ABORT-marker (`VarUInt(3)`) → reset state + fire abort callback. |
| `SIGNALR_BINARY_PROTOCOL/README.md` | Wire-format spec section update — replace `[201][UINT16][data]...[202]` documentation with the new VarUInt scheme. Update the "BinaryProtocolMode" / "AsyncSegment" section. |
| `BINARY_FORMAT.md` | **Not affected** — this is SignalR-protocol-layer framing, not AcBinary inner format. |
| Tests | Round-trip tests across all body-length tiers (0..4, 5..127, 128..16383, 16384..65535, 65536+ new tier). Corruption-detection test (wire-byte 0 → throw). ABORT-flow integration test (sender abort → receiver state reset). |

Estimated effort: ~1-2 days for the wire-format change + state machine update,
plus test coverage. Low blast radius — all changes localized to three files +
wire-format spec doc.

## Defensive validation

### VarUInt(0) on the wire — corruption

VarUInt encoding never produces a 0-valued first byte for a non-zero number,
and the protocol never legitimately emits the value 0 (no sentinel maps to it,
no chunk has negative-length body). A decoded 0 indicates either a corrupted
stream or a sender-side encoder bug. Reader response:

```csharp
if (first == 0)
    throw new InvalidDataException(
        "AcBinary SignalR frame: invalid VarUInt(0) marker — stream corruption suspected.");
```

One `cmp` in the hot path's else-branch (the rare-marker switch's `default`).

### Robustness vs corruption mid-stream

Same property as the legacy `[201/202]` byte-marker format: once the framing
state machine loses sync, it cannot self-recover from a corrupted byte position
without external resynchronization (e.g. transport disconnect + reconnect). The
new scheme does not regress this — both formats rely on the same
"transport-layer-recovery is the contract" semantics, except the ABORT-marker
now provides an explicit in-band fast-path for known-sender-side aborts (no
disconnect needed for that case).

## Open considerations

- **VarUInt encoding choice** — the existing AcBinary `WriteVarUInt` /
  `ReadVarUInt` implementations are used. No new encoding helpers.
- **Performance regression check** — BDN benchmark comparing legacy-byte-marker
  vs VarUInt-marker decode cost on a representative chunk-stream. Expected: ≤2%
  decode-time regression, offset by wire-size savings on real network.
- **ABORT-callback semantics** — `AyCodeBinaryHubProtocol.OnChunkAbort` already
  exists for the transport-disconnect path; the in-band marker reuses the same
  callback, no consumer-side change needed.

## References

- `SIGNALR_BINARY_PROTOCOL_ISSUES.md#accore-bin-i-b7k9` — 65 KB cap (resolved by this)
- `../../../../AyCode.Core/AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_ISSUES.md#accore-bin-i-z2x9` — cancel recovery (resolved by this)
- `SIGNALR_BINARY_PROTOCOL_ISSUES.md#accore-bin-i-m9p3` — chunk multiplexing (separate concern, not addressed here)
- `README.md` — current wire-format spec (to be updated on landing)
