AyCode.Core/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md

SignalR Binary Protocol

AcBinaryHubProtocol (unsealed base) — custom IHubProtocol (name: "acbinary") replacing SignalR JSON+Base64 with AcBinarySerializer. General binary framing only — no consumer-specific logic.

AyCodeBinaryHubProtocol (derived) — project-specific consumer logic: SignalParams capture via OnArgumentRead, IsRawBytesData path, SignalDataType type resolution in ReadSingleArgument override.

Architecture (tag system, dispatch, request/response): SIGNALR.md Output writers (cached chunk, buffer states, chunk sizing): AyCode.Core/docs/BINARY_WRITERS.md

Wire Format

[INT32 LE payload length] [1 byte message type] [message-specific fields]

Type	Byte	Fields
Invocation	1	nullable invocationId, target, arguments, streamIds, headers
StreamItem	2	invocationId, argument, headers
Completion	3	invocationId, nullable error, hasResult, optional result, headers
StreamInvocation	4	invocationId, target, arguments, streamIds, headers
CancelInvocation	5	invocationId, headers
Ping	6	(empty)
Close	7	nullable error, allowReconnect
Ack	8	sequenceId (INT64)
Sequence	9	sequenceId (INT64)

Argument framing: [INT32 LE arg length] [serialized bytes] — deferred deserialization (target parsed first → IInvocationBinder resolves types).

Strings: [VarUInt byte length] [UTF-8]. Nullable: 0=null, 1=value follows.

Headers: [VarUInt count] [key-value pairs...]. Count=0 → no headers.

Zero-Copy Write Pipeline

Writes directly to SignalR pipe's IBufferWriter<byte>, no intermediate byte[].

WriteMessage(HubMessage, IBufferWriter<byte> output)
├─ Reserve 4-byte outer length prefix
├─ BWO = BufferWriterBinaryOutput(output)  [standalone mode]
│  ├─ WriteByte(messageType)
│  ├─ WriteStringUtf8(invocationId, target)
│  ├─ WriteVarUInt(argCount)
│  ├─ Per argument:
│  │  ├─ byte[] (AcBinary) → raw length + bytes (no tag, no VarUInt)
│  │  ├─ byte[] (other)    → tag (0x44) + raw bytes (no VarUInt — argLength implies size)
│  │  └─ object → FlushAndReset() → reserve INT32 arg prefix
│  │     → AcBinarySerializer.Serialize(value, output) → patch prefix
│  ├─ WriteStringArray(streamIds)
│  └─ WriteHeaders(headers)
├─ Patch outer length = BWO.Position + externalBytes
└─ BWO.Flush()

byte[] Write: isAcBinary Detection

When argument is byte[], the protocol checks if it's already AcBinary-serialized data:

var isAcBinary = byteArray.Length >= 2
    && byteArray[0] == AcBinarySerializerOptions.FormatVersion  // 1
    && (byteArray[1] & 0xF0) == BinaryTypeCode.HeaderFlagsBase; // 0x90

• isAcBinary = true: [argLength=N][raw AcBinary bytes] — no tag, reader deserializes directly
• isAcBinary = false: [argLength=1+N][0x44 tag][raw bytes] — tag for type detection, no VarUInt (argLength implies size)

Dual BWO Pattern

Protocol and serializer each create own BufferWriterBinaryOutput on the same IBufferWriter. Sequential, never concurrent:

1. Protocol BWO (standalone): framing — message type, strings, headers. Cached chunk, zero dispatch.
2. FlushAndReset(): commits bytes to pipe, invalidates chunk.
3. Serializer BWO (context mode): Serialize() creates internal BWO, acquires fresh chunk, writes, flushes.
4. Protocol BWO re-acquires chunk on next write (via Grow).

Cost: one extra GetMemory per argument (nanoseconds). Benefit: zero-copy end-to-end, no intermediate byte[], no wrapper class.

Why two BWOs: serializer writes must live on BinarySerializationContext (sealed class) for JIT optimization — context owns its own BWO. See AyCode.Core/docs/BINARY_WRITERS.md § "Why Writes Are on the Context".

Length Prefix Patching

var lengthSpan = output.GetSpan(4);
output.Advance(4);
// ... write payload ...
Unsafe.WriteUnaligned(ref lengthSpan[0], payloadLength);

Safe for PipeWriter — segments writable until FlushAsync.

GetMessageBytes caveat: ArrayBufferWriter initial capacity must include LengthPrefixSize to prevent resize after prefix reservation (stale span).

Read: Argument Deserialization

Base AcBinaryHubProtocol.ReadSingleArgument reads [INT32 argLength] [argBytes] from the pipe's ReadOnlySequence via SequenceReader<byte>:

ReadSingleArgument(SequenceReader, targetType):
  Read INT32 argLength
  if argLength == 0 → return null
  if argLength == 1 && first byte == 0 → return null  (null marker)

  argSlice = UnreadSequence.Slice(0, argLength)  — zero-copy reference
  Advance(argLength)

  1. byte[] fast-path:
     if first byte == BinaryTypeCode.ByteArray (0x44):
       return argSlice.Slice(1) as byte[]  — skip tag, rest is raw payload
       No VarUInt — argLength implies size

  2. Default: DeserializeFromSequence(argSlice, targetType, options)

AyCodeBinaryHubProtocol.ReadSingleArgument overrides with consumer-specific paths:

  (same argLength/null/slice logic as base)

  1. byte[] fast-path (same as base)

  2. IsRawBytesData path:
     if _currentSignalParams.IsRawBytesData == true:
       return SequenceToByteArray(argSlice)  — entire arg as raw byte[]
       Consumer (DataSource.PopulateMerge) handles deserialization

  3. Typed deserialization:
     if targetType == object && SignalDataType != null:
       resolve Type from SignalDataType (AssemblyQualifiedName)
     DeserializeFromSequence(argSlice, resolvedType, options)

SignalParams Capture

Base AcBinaryHubProtocol.ReadArguments calls OnArgumentRead(value, index) after each argument. AyCodeBinaryHubProtocol overrides this hook to capture SignalParams (arg[2]) for type-aware deserialization of subsequent args (arg[3] = data). Thread-safe: SignalR processes messages sequentially per connection.

SequenceToByteArray

Zero-copy when possible: if single-segment and backing array matches exactly → return the array directly. Otherwise ReadOnlySequence.ToArray().

SequenceBinaryInput (Multi-Segment Deserialization)

struct SequenceBinaryInput : IBinaryInputBase — reads from ReadOnlySequence<byte> without linearizing. Lazy iteration via ReadOnlySequence.TryGet — zero constructor allocation, no pre-extracted segment array.

The context's _buffer always points directly to the current segment's backing byte[] (zero-copy). Cross-boundary reads (value straddling segment boundary) copy only the affected bytes into a small ArrayPool-rented scratch buffer. After the scratch read, _afterCrossBoundary flag restores the context to the next segment's backing array.

Typical overhead for 225KB payload with 4096-byte segments: ~224.5KB zero-copy, ~500 bytes scratch copy at ~55 boundaries. The scratch buffer is rented once (lazy, on first boundary) and reused across all boundaries. Release() returns it to ArrayPool after deserialization.

Known issues: AyCode.Core/docs/BINARY_ISSUES.md

Configuration

Hub protocol settings are controlled via AcBinaryHubProtocolOptions (mutable class). Pass directly to the protocol constructor, or configure via DI in Program.cs with services.Configure<AcBinaryHubProtocolOptions>(opts => …).

Property	Default	Purpose
SerializerOptions	AcBinarySerializerOptions.Default	Binary serializer options (also usable standalone via ToBinary/BinaryTo).
ProtocolMode	Bytes	Wire format and pipeline strategy — see BinaryProtocolMode below.
BufferSize	4096	Per-chunk size. 4 KB aligns with Kestrel's slab. Max 65535 (UINT16).
WaitForFlush	true	AsyncSegment flush strategy — see trade-off below.
FlushTimeout	10 s	Per-flush wait limit. Timeout.InfiniteTimeSpan = disabled.
Name	"acbinary"	SignalR handshake protocol name. Client and server must match.
Logger	null	Optional ILogger; injected from DI when registered.

Inner AcBinarySerializerOptions defaults relevant for SignalR: UseGeneratedCode=true (hybrid source-gen + reflection), UseStringInterning=All, InitialBufferCapacity=16384.

WaitForFlush (AsyncSegment-only)

Value	Pro	Con
true (default)	Max pipeline parallelism + guaranteed end-to-end zero-copy on send.	Slow consumer propagates back as server-thread blocking (bounded by FlushTimeout).
false	Adaptive — fire-and-forget per chunk, blocks only when ~60 KB memory threshold is hit.	Under heavy backpressure a chunk may fall back to an owned (copied) buffer, losing zero-copy for that chunk.

FlushTimeout rationale (10 s default)

• AsyncSegment chunks are ≤ 65 KB (UINT16). Even GPRS-class links (~60 Kbit/s) transfer 65 KB in ~9 s — so any flush exceeding 10 s indicates a genuinely stuck consumer.
• Pro: fast failure detection; server thread never blocks indefinitely.
• Con: an unusually slow but otherwise healthy consumer will be disconnected — tune up for satellite / throttled links.
• Complementary to SignalR's connection-level timeouts (ClientTimeoutInterval, KeepAliveInterval). Set FlushTimeout < ClientTimeoutInterval so this per-operation guard fires first.
• Set to Timeout.InfiniteTimeSpan to fully disable (legacy behavior).

BinaryProtocolMode

enum BinaryProtocolMode — constructor parameter for AcBinaryHubProtocol, selects serialization + transport strategy:

Value	Serialize	Deserialize (non-WASM)	Pro / Con
Bytes (default)	ArrayBinaryOutput → byte[] → write to pipe as raw blob	ArrayBinaryInput (single contiguous buffer via MemoryMarshal.TryGetArray zero-copy / pool-rent).	Pro: simplest, fastest per-call, WASM-safe on both sides. Con: no zero-copy write, no pipeline overlap.
Segment	BufferWriterBinaryOutput → directly to PipeWriter, chunk-by-chunk, single Flush at end	Same as Bytes (unified ArrayBinaryInput receive path — _protocolMode affects send only).	Pro: zero-copy write, WASM-safe. Con: no pipeline overlap — receiver must wait for full payload before deser starts.
AsyncSegment	AsyncPipeWriterOutput → self-describing chunked framing [201][UINT16 size][data] per chunk, per-chunk FlushAsync with timeout-bounded sync-await	SegmentBufferReader (growing contiguous byte[]) + SegmentBufferReaderInput; background Task.Run deserializes while chunks arrive. WASM: synchronous deser on CHUNK_END.	Pro: zero-copy write + pipeline parallelism (ser / network / deser overlap). Con: send-side not WASM-compatible (see below); slow consumer propagates as server-thread blocking (bounded by FlushTimeout). Max chunk: 65535 bytes.

In AsyncSegment mode, WriteMessage dispatches to WriteMessageChunked which sends: (1) CHUNK_START — standard SignalR framing [INT32 len][200][original message with INT32 -1 for streamed arg], (2) N x CHUNK_DATA — [201][UINT16 size][data] per chunk (zero-copy via PipeWriter.Advance with 3-byte header reservation), (3) CHUNK_END — [202] (1 byte). The receiver's TryParseChunkData accumulates into a SegmentBufferReader; on non-WASM platforms a background Task.Run deserializes in parallel via SegmentBufferReaderInput, on WASM the deserializer runs synchronously on CHUNK_END over the already-buffered data.

In Bytes and Segment mode, the standard WriteMessage path is used.

WebAssembly compatibility

The send and receive paths handle WASM (OperatingSystem.IsBrowser()) asymmetrically — send is strictly bound to _protocolMode, receive adapts to the wire format and falls back to a synchronous path only when the platform cannot support the optimal strategy.

• Send path: AsyncSegment is not supported on WebAssembly. The AcBinaryHubProtocolOptions.Validate() method throws PlatformNotSupportedException if IsBrowser && ProtocolMode == AsyncSegment (the AsyncPipeWriterOutput.SyncAwaitFlush sync-over-async pattern would block the single UI thread). WASM clients must use Bytes or Segment. (Note: this guard is currently commented out in Validate() to enable hybrid Windows-app + WASM testing against a single protocol instance. Will be re-enabled once the options are fully wired through Program.cs.)
• Receive path: works on WASM with any server-side mode (including AsyncSegment → chunked wire). TryParseChunkData detects the platform at runtime:
  • Non-browser: first CHUNK_DATA spawns a background Task.Run over a SegmentBufferReader (pipeline parallelism — serialize / network / deserialize overlap). CHUNK_END awaits the task's result.
  • Browser: the background task is skipped. Chunks accumulate in SegmentBufferReader; on CHUNK_END the buffer is Complete()d and the deserializer runs synchronously on the current thread. SegmentBufferReaderInput.TryAdvanceSegment sees _completed=true and never calls ManualResetEventSlim.Wait() (which throws PlatformNotSupportedException on WASM).

Consequence: a mixed topology (desktop server in AsyncSegment, WASM client in Bytes) works without any negotiation or protocol-name variation — the client converts the incoming chunked wire to its own synchronous processing model.

Source: AyCode.Services/SignalRs/AcBinaryHubProtocol.cs (base), AyCode.Services/SignalRs/AyCodeBinaryHubProtocol.cs (consumer logic), AyCode.Services/SignalRs/BinaryProtocolMode.cs (enum)