Adam
/
AyCode.Core
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Improve binary/SignalR docs, add protocol and writer deep-dives 

Major documentation overhaul for binary serialization and SignalR:
- Simplified and clarified AyCode.Core README, with direct links to new deep-dive docs (BINARY_IMPLEMENTATION.md, BINARY_WRITERS.md)
- Added BINARY_WRITERS.md: detailed design and rationale for ArrayBinaryOutput and BufferWriterBinaryOutput, chunk sizing, and buffer management
- Refined BINARY_IMPLEMENTATION.md: clearer buffer management, output strategies, and hot-path rules; references new writers doc
- Added SIGNALR_BINARY_PROTOCOL.md: full wire format, zero-copy pipeline, dual BWO pattern, and read path for custom SignalR protocol
- Updated SignalRs README and SIGNALR.md: clarified protocol, tag system, request/response flow, and technical debt
- Improved cross-linking and discoverability throughout

These changes make the technical documentation clearer, more maintainable, and easier to navigate for advanced contributors.
This commit is contained in:
Loretta 2026-04-04 09:27:36 +02:00
parent 0cb2b6c2d8
commit 9150df6982
7 changed files with 330 additions and 194 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.github/copilot-instructions.md vendored
View File

File diff suppressed because one or more lines are too long

101
AyCode.Core/Serializers/Binaries/README.md
View File

@ -1,95 +1,90 @@
# Binaries # Binaries
High-performance binary serialization/deserialization with two-phase processing, multiple wire modes, string interning, and source generation support. The primary goal is **speed**: every design decision prioritizes minimal latency and maximum throughput. High-performance binary serialization/deserialization. Two-phase processing, multiple wire modes, string interning, source generation. Primary goal: **speed**.
> For deep technical implementation details (Zero Virtual Dispatch, Direct Buffer Management), see `../../docs/BINARY_IMPLEMENTATION.md`. > Implementation details (zero virtual dispatch, buffer management): `../../docs/BINARY_IMPLEMENTATION.md`
> Output writers (ArrayBinaryOutput, BufferWriterBinaryOutput, chunk sizing): `../../docs/BINARY_WRITERS.md`
## Architecture ## Architecture
### Two-Phase Serialization ### Two-Phase Serialization
1. **Scan Pass** (`AcBinarySerializer.ScanPass.cs`) — Walks the object graph to detect multi-referenced objects and build the reference table. 1. **Scan** (`ScanPass.cs`) — walks graph, detects multi-ref objects, builds reference table
2. **Serialize Pass** (`AcBinarySerializer.BinarySerializationContext.cs`) — Writes the binary output using the reference table from the scan pass. 2. **Serialize** (`BinarySerializationContext.cs`) — writes binary using reference table
The serializer is generic over `TOutput` for strategy selection (`ArrayBinaryOutput` vs `BufferWriterBinaryOutput`). Generic over `TOutput` for strategy selection (`ArrayBinaryOutput` vs `BufferWriterBinaryOutput`).
### Wire Format ### Wire Format
`BinaryTypeCode.cs` defines 100+ type markers: `BinaryTypeCode.cs` — 100+ type markers:
| Range | Purpose | | Range | Purpose |
|---|---| |---|---|
| 0–63 | **FixObj** — Compiled type slot indices | | 0–63 | **FixObj** — compiled type slot indices |
| 64–71 | **Complex types** — Object, ObjectRef, Array, Dictionary, ByteArray (with/without metadata) | | 64–71 | **Complex** — Object, ObjectRef, Array, Dictionary, ByteArray |
| 72–75 | **Polymorphic** — ObjectWithTypeName, ObjectWithTypeIndex (with/without RefFirst) | | 72–75 | **Polymorphic** — ObjectWithTypeName/TypeIndex (±RefFirst) |
| 76–90 | **Primitives** — Null, Bool, Int8–64, Float32–64, Decimal, Char | | 76–90 | **Primitives** — Null, Bool, Int8–64, Float32–64, Decimal, Char |
| 91–94 | **Strings** — String, StringInterned, StringEmpty, StringInternFirst | | 91–94 | **Strings** — String, StringInterned, StringEmpty, StringInternFirst |
| 95–98 | **Date/Time** — DateTime, DateTimeOffset, TimeSpan, Guid | | 95–98 | **Date/Time** — DateTime, DateTimeOffset, TimeSpan, Guid |
| 99–102 | **Other** — Enum, Legacy headers, PropertySkip | | 99–102 | **Other** — Enum, Legacy, PropertySkip |
| 103–134 | **FixStr** — Short strings with length encoded in marker | | 103–134 | **FixStr** — short strings (length in marker) |
| 144+ | **Headers** — Metadata, RefHandling, CacheCount flags | | 144+ | **Headers** — Metadata, RefHandling, CacheCount |
| 192–255 | **Tiny ints** — Single-byte encoding for values -16 to 47 | | 192–255 | **Tiny ints** — single-byte -16..47 |
For the complete wire format specification (encoding rules, header format, interning protocol), see `docs/BINARY_FORMAT.md`. Full spec: `docs/BINARY_FORMAT.md`
## Key Files ## Key Files
### Serialization ### Serialization
- **`AcBinarySerializer.cs`** — Main serializer entry point, context pool management. - **`AcBinarySerializer.cs`** — entry point, context pool
- **`AcBinarySerializer.BinarySerializationContext.cs`** — Core serialization logic, object graph traversal, type writing. - **`...BinarySerializationContext.cs`** — core logic, graph traversal, type writing
- **`AcBinarySerializer.BinarySerializationContext.PropertyWriters.cs`** — Per-type property write methods. - **`...PropertyWriters.cs`** — per-type property write methods
- **`AcBinarySerializer.BinarySerializationResult.cs`** — Result wrapper (byte[] or IBufferWriter output). - **`...BinarySerializationResult.cs`** — result wrapper
- **`AcBinarySerializer.BinarySerializeTypeMetadata.cs`** — Cached type metadata for serialization. - **`...BinarySerializeTypeMetadata.cs`** — cached type metadata
- **`AcBinarySerializer.ScanPass.cs`** — Reference scanning phase. - **`...ScanPass.cs`** — reference scanning
### Deserialization ### Deserialization
- **`AcBinaryDeserializer.cs`** — Main deserializer entry point. - **`AcBinaryDeserializer.cs`** — entry point
- **`AcBinaryDeserializer.BinaryDeserializationContext.cs`** — Core deserialization logic. - **`...BinaryDeserializationContext.cs`** — core logic
- **`AcBinaryDeserializer.BinaryDeserializationContext.Read.cs`** — Primitive read operations. - **`...BinaryDeserializationContext.Read.cs`** — primitive reads
- **`AcBinaryDeserializer.BinaryDeserializeTypeMetadata.cs`** — Cached type metadata for deserialization. - **`...BinaryDeserializeTypeMetadata.cs`** — cached type metadata
- **`AcBinaryDeserializer.CrossType.cs`** — Cross-type deserialization (schema evolution). - **`...CrossType.cs`** — schema evolution
- **`AcBinaryDeserializer.Populate.cs`** — Object population and merge operations. - **`...Populate.cs`** — object population/merge
### I/O Strategies ### I/O
- **`BinaryOutputBase.cs`** — Output interface. - **`BinaryOutputBase.cs`** — output interface
- **`ArrayBinaryOutput.cs`** — `ArrayPool`-backed output, fastest for `byte[]` result. - **`ArrayBinaryOutput.cs`** — ArrayPool-backed, fastest for `byte[]`
- **`BufferWriterBinaryOutput.cs`** — `IBufferWriter<byte>`-backed output for streaming. Two modes: context mode (serialization pipeline) and standalone mode (direct write methods for framing, e.g. `AcBinaryHubProtocol`). - **`BufferWriterBinaryOutput.cs`** — IBufferWriter-backed, zero-copy streaming. Context mode (serializer) + standalone mode (direct writes, e.g. `AcBinaryHubProtocol`)
- **`ArrayPooledBufferWriter.cs`** — Concrete `IBufferWriter` implementation. - **`ArrayPooledBufferWriter.cs`** — concrete IBufferWriter
- **`IBinaryInputBase.cs`** — Input interface. - **`IBinaryInputBase.cs`** / **`ArrayBinaryInput.cs`** / **`SequenceBinaryInput.cs`** — input strategies
- **`ArrayBinaryInput.cs`** — Single contiguous `byte[]` input.
- **`SequenceBinaryInput.cs`** — Multi-segment `ReadOnlySequence<byte>` input.
### Configuration & Types ### Config & Types
- **`AcBinarySerializerOptions.cs`** — All configuration options and presets. - **`AcBinarySerializerOptions.cs`** — options, presets
- **`BinaryTypeCode.cs`** — Wire format type markers. - **`BinaryTypeCode.cs`** — wire format markers
- **`StringInterningMode.cs`** — Enum: `None`, `Attribute`, `All`. - **`StringInterningMode.cs`** — None/Attribute/All
- **`AcStringInternAttribute.cs`** — Property-level string interning control. - **`AcStringInternAttribute.cs`** — property-level interning
- **`TypeConversionInfo.cs`** — Type conversion tracking for cross-type scenarios. - **`TypeConversionInfo.cs`** — cross-type tracking
- **`BinaryPropertyFilterContext.cs`** — Instance-dependent property filtering. - **`BinaryPropertyFilterContext.cs`** — instance-dependent filtering
### Property Handling ### Property Handling
- **`BinaryPropertyAccessorBase.cs`** — Property accessor for serialization. - **`BinaryPropertyAccessorBase.cs`** / **`BinaryPropertySetterBase.cs`**
- **`BinaryPropertySetterBase.cs`** — Property setter for deserialization.
### Source Generation ### Source Generation
- **`IGeneratedBinaryWriter.cs`** — Interface for source-generated writers. - **`IGeneratedBinaryWriter.cs`** / **`IGeneratedBinaryReader.cs`**
- **`IGeneratedBinaryReader.cs`** — Interface for source-generated readers.
### Exceptions ### Exceptions
- **`AcBinaryDeserializationException.cs`** — Rich exception with binary offset context. - **`AcBinaryDeserializationException.cs`** — rich exception with binary offset
## Configuration Options ## Config Options
Key options that change wire format: `WireMode` (Compact/Fast), `ReferenceHandling` (None/OnlyId/All), `UseMetadata`, `UseStringInterning` (None/Attribute/All), `MaxDepth`, `UseCompression`, `PropertyFilter`. Key wire-format options: `WireMode` (Compact/Fast), `ReferenceHandling` (None/OnlyId/All), `UseMetadata`, `UseStringInterning` (None/Attribute/All), `MaxDepth`, `UseCompression`, `PropertyFilter`.
`ReferenceHandling=None` + `UseStringInterning=None` = no scan pass (fastest, single-phase). `ReferenceHandling=None` + `UseStringInterning=None` = no scan pass (single-phase, fastest).
Presets: `Default`, `FastMode`, `ShallowCopy`, `WasmOptimized`. Presets: `Default`, `FastMode`, `ShallowCopy`, `WasmOptimized`. Details: `docs/BINARY_OPTIONS.md`.
For detailed option documentation with wire format impact, code branches, and interactions, see `docs/BINARY_OPTIONS.md`.
## Dependencies ## Dependencies
- Base classes from parent `Serializers/` folder (`AcSerializerContextBase`, `TypeMetadataBase`, `IdentityMap`, etc.) - Base classes: parent `Serializers/` (`AcSerializerContextBase`, `TypeMetadataBase`, `IdentityMap`)
- `System.Buffers` (ArrayPool, IBufferWriter) - `System.Buffers` (ArrayPool, IBufferWriter)
- LZ4 (optional compression) - LZ4 (optional compression)

74
AyCode.Core/docs/BINARY_IMPLEMENTATION.md
View File

File diff suppressed because one or more lines are too long

88
AyCode.Core/docs/BINARY_WRITERS.md Normal file
View File

@ -0,0 +1,88 @@
# Binary Output Writers
Output strategies for `AcBinarySerializer`. Generic over `TOutput : struct, IBinaryOutputBase` → compile-time specialization, zero virtual dispatch.
> Buffer management, hot-path rules: `BINARY_IMPLEMENTATION.md`
## IBinaryOutputBase
Cold-path contract between serializer context and output strategy:
```csharp
void Initialize(out byte[] buffer, out int position, out int bufferEnd);
void Grow(ref byte[] buffer, ref int position, ref int bufferEnd, int needed);
int GetTotalPosition(int currentPosition);
void Reset();
```
**Critical:** All write methods (`WriteByte`, `WriteVarUInt`, `WriteStringUtf8`, etc.) live on `BinarySerializationContext<TOutput>` sealed class — NOT on the output. Output handles only buffer lifecycle. See [Why Writes Are on the Context](#why-writes-are-on-the-context).
## ArrayBinaryOutput
`struct ArrayBinaryOutput : IBinaryOutputBase, IDisposable` — fastest for `byte[]`/`ReadOnlySpan<byte>` result.
- **Initialize:** provides pooled buffer, position=0
- **Grow:** rent doubled buffer from `ArrayPool`, copy, return old
- **Pooling:** ≤32KB kept across serializations (faster than pool round-trip); >32KB returned, next rent halved
- **Results:** `ToArray` (allocate+memcpy), `DetachResult` (caller owns pooled buffer), `AsSpan` (zero-alloc view)
- **OutputInitialized** flag: single instance per pooled context, reused
## BufferWriterBinaryOutput
`struct BufferWriterBinaryOutput : IBinaryOutputBase` — writes directly to `IBufferWriter<byte>` (PipeWriter, ArrayBufferWriter). Zero-copy streaming.
### Cached Chunk Pattern
Instead of `GetSpan`/`Advance` per write (interface dispatch), acquires large chunk once:
1. `GetMemory(chunkSize)` → `TryGetArray` → backing `byte[]` + offset
2. All writes: `buffer[position++]` — direct array indexing, zero dispatch
3. `Grow`: `Advance(bytesInChunk)` → acquire next chunk
4. `Flush`: commit final bytes via `Advance`
5. **Fallback:** `TryGetArray` fails → rent temp buffer, copy on `Grow`/`Flush`
### Two Usage Modes
Separate buffer states, never concurrent:
**Context mode** — serialization pipeline:
- Buffer state on context: `_buffer`, `_position`, `_bufferEnd`
- BWO invoked only via `Initialize`/`Grow`/`Flush` with `out`/`ref` params
- Context write methods operate on own fields → max JIT optimization
**Standalone mode** — direct writes outside serializer (e.g. `AcBinaryHubProtocol` framing):
- Buffer state on struct: `_buffer`, `_position`, `_bufferEnd`
- Write methods: `WriteByte`, `WriteVarUInt`, `WriteStringUtf8`, `WriteBytes`, `WriteRaw<T>`
- `Position`: total bytes (committed + pending)
- `Flush()`: commit pending, finalize
- `FlushAndReset()`: commit pending, invalidate chunk → `IBufferWriter` available for another writer
Context/standalone share only `IBufferWriter` ref and `_committedBytes`.
### Known Limitations
1. **Struct copy semantics:** value type → assignment creates independent copy. State changes (e.g. `_committedBytes` via `Grow`/`Flush`) not reflected in original. Copy back after use if needed.
2. **Initialize resets tracking:** `_committedBytes = 0`. Standalone bytes lost if BWO then passed to context. Use `FlushAndReset()` before, track standalone bytes separately.
3. **Constructor acquires chunk:** `AcquireChunk` in ctor for standalone readiness. Redundant if only context mode used (context `Initialize` acquires its own). Not a leak — consecutive `GetMemory` without `Advance` returns overlapping memory.
4. **No mode mixing:** single instance must not use context+standalone simultaneously. Buffer states desynchronize. One mode per lifecycle phase, `FlushAndReset()` as boundary.
### Chunk Size
Default 65536 (64KB), configurable via `AcBinarySerializerOptions.BufferWriterChunkSize`.
- **Memory-backed** (ArrayBufferWriter): 64KB optimal — fewer `Grow` calls
- **Network-backed** (PipeWriter): smaller (4096) aligns with transport segments. But 64KB default is safe — `PipeWriter` may return less than requested
- Too-small default would cause excessive `Grow`; 64KB is never catastrophic
## Why Writes Are on the Context
Most important architectural decision in the output layer.
**Attempted:** write methods on output struct. Context calls `Output.WriteByte(value)`.
**Result:** measurably slower, even with struct + `AggressiveInlining` + generic constraint devirtualization.
**Root cause:** JIT generates better code for sealed class field access (`this._buffer`, `this._position` — fixed offsets) than generic struct field access (`this.Output._buffer` — extra address computation at lower optimization tiers).
**Current:** writes on `BinarySerializationContext<TOutput>` (sealed class, hot path). Output struct handles only `Initialize`/`Grow`/`Flush` (cold path).
**Rule:** Do NOT move write methods to output. Measure with full benchmark suite before proposing changes.

1
AyCode.Services/SignalRs/README.md
View File

@ -3,6 +3,7 @@
Custom binary SignalR protocol, client infrastructure, message tagging, and serialization helpers. Custom binary SignalR protocol, client infrastructure, message tagging, and serialization helpers.
> **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR.md`. > **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR.md`.
> **Binary protocol:** For wire format, zero-copy pipeline, and dual BWO pattern see `AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md`.
## Key Files ## Key Files

151
AyCode.Services/docs/SIGNALR.md
View File

@ -1,24 +1,23 @@
# SignalR Client # SignalR Client
Client-side SignalR transport with custom binary protocol and tag-based dispatch. Source: `SignalRs/` in this project. Client-side SignalR transport: custom binary protocol, tag-based dispatch. Source: `SignalRs/`
> For server-side hub, session, broadcast see `AyCode.Services.Server/docs/SIGNALR_SERVER.md`. > Server-side hub, session, broadcast: `AyCode.Services.Server/docs/SIGNALR_SERVER.md`
> For the DataSource collection see `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. > DataSource collection: `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`
## Design Overview ## Design
All communication flows through a **single hub method** with **tag-based dispatch**: Single hub method, tag-based dispatch:
``` ```
Client ──OnReceiveMessage(tag, bytes, requestId)──► Server Client ──OnReceiveMessage(tag, bytes, requestId)──► Server
Client ◄──OnReceiveMessage(tag, bytes, requestId)── Server Client ◄──OnReceiveMessage(tag, bytes, requestId)── Server
``` ```
`OnReceiveMessage` is the only transport channel — all calls go through it. The `tag` (int) determines **which server method to invoke**. Projects call any endpoint by specifying the tag; the server dispatches accordingly. Tag (int) determines server method. All calls go through `OnReceiveMessage`.
``` ```
Client side: Server side: Client: Server:
───────────────────── ─────────────────────
AcSignalRClientBase AcWebSignalRHubBase<TTags, TLogger> AcSignalRClientBase AcWebSignalRHubBase<TTags, TLogger>
├─ HubConnection (WebSocket) ├─ Hub<IAcSignalRHubItemServer> ├─ HubConnection (WebSocket) ├─ Hub<IAcSignalRHubItemServer>
├─ AcBinaryHubProtocol ├─ DynamicMethodRegistry ├─ AcBinaryHubProtocol ├─ DynamicMethodRegistry
@ -28,9 +27,7 @@ AcSignalRClientBase AcWebSignalRHubBase<TTags, TLogger>
## Tag System ## Tag System
### Tag Definition Base tags in AyCode.Core:
AyCode.Core defines only the base class and built-in infrastructure tags:
```csharp ```csharp
public class AcSignalRTags public class AcSignalRTags
@ -41,153 +38,105 @@ public class AcSignalRTags
} }
``` ```
**Consuming projects** define their own tags by inheriting `AcSignalRTags`: Consuming projects inherit and define own tags (plain int constants, must be unique per hub):
```csharp ```csharp
public abstract class MyProjectTags : AcSignalRTags public abstract class MyProjectTags : AcSignalRTags
{ {
public const int GetOrders = 100; public const int GetOrders = 100;
public const int GetOrderById = 101; public const int GetOrderById = 101;
public const int SaveOrder = 102;
} }
``` ```
Tags are plain `int` constants. The project decides the numbering. Tags must be unique across all registered services within a hub. **Attributes:**
- `[Tag(42)]` — base: maps int tag → method
- `[SignalR(messageTag, sendToOtherClientTag, sendToOtherClientType)]` — server routing + broadcast
- `[SignalRSendToClient(100)]` — client receive dispatch
### Tag Attributes **SendToClientType:** `None` | `Others` | `Caller` | `All`
Three attribute levels:
**Usage:**
```csharp ```csharp
// Base: maps an integer tag to a method var orders = await client.PostAsync<List<Order>>(MyTags.GetOrders, companyId);
[Tag(42)] await client.PostDataAsync(MyTags.SaveOrder, order);
await client.PostDataAsync(MyTags.SaveOrder, order, async response => { ... }); // async callback
// Server method: tag + optional broadcast behavior
[SignalR(messageTag: 100, sendToOtherClientTag: 100, sendToOtherClientType: SendToClientType.Others)]
// Client receive: marks method for server→client dispatch
[SignalRSendToClient(100)]
``` ```
**SendToClientType:** `None` (no broadcast), `Others` (all except caller), `Caller` (response only), `All` (everyone). CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are generic transport, not tied to DataSource.
### How Projects Use Tags
Projects call the SignalR transport directly — not only through DataSource:
```csharp
var orders = await signalRClient.PostAsync<List<Order>>(MyTags.GetOrders, companyId);
var order = await signalRClient.GetByIdAsync<Order>(MyTags.GetOrderById, orderId);
await signalRClient.PostDataAsync(MyTags.SaveOrder, order);
// Async callback (non-blocking)
await signalRClient.PostDataAsync(MyTags.SaveOrder, order, async response => { ... });
// Fire-and-forget
await signalRClient.PostDataAsync(MyTags.SaveOrder, order, response => { ... });
```
The CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are all generic transport methods that work with any tag — they are not tied to DataSource.
## Wire Protocol ## Wire Protocol
### AcBinaryHubProtocol ### AcBinaryHubProtocol
Custom `IHubProtocol` (name: `"acbinary"`), replaces default JSON. Frame format: Custom `IHubProtocol` (`"acbinary"`), replaces JSON. Zero-copy via `BufferWriterBinaryOutput` standalone mode. `byte[]` args bypass serializer.
``` > Wire format, argument framing, dual BWO pattern, length prefix patching: `SIGNALR_BINARY_PROTOCOL.md`
[4 bytes: payload length, little-endian] [1 byte: message type] [payload...]
```
Message types: Invocation(1), StreamItem(2), Completion(3), Ping(6), Close(7), Ack(8), Sequence(9).
Arguments serialized individually with INT32 length prefix (patched in-place after payload is written).
**Zero-copy write pipeline:**
All writes go through a single `BufferWriterBinaryOutput` in standalone mode (cached chunk pattern, zero virtual dispatch). For argument payloads, the BWO flushes to the pipe via `FlushAndReset()`, then `AcBinarySerializer.Serialize()` writes directly to the `IBufferWriter` (pipe) — zero-copy, no intermediate `byte[]` allocation.
**Raw `byte[]` Fast-Path:**
When an argument is a pure `byte[]`, the size is known upfront. The protocol writes `BinaryTypeCode.ByteArray` (68) marker, `VarUInt` length, and raw bytes entirely through the `BufferWriterBinaryOutput`, completely skipping the `AcBinarySerializer` context, the two-phase scan, and the internal array pools.
### Response Message ### Response Message
`SignalResponseDataMessage` carries: `SignalResponseDataMessage`:
| Field | Type | Purpose | | Field | Type | Purpose |
|-------|------|---------| |-------|------|---------|
| `MessageTag` | int | Tag identifying the operation | | `MessageTag` | int | Operation tag |
| `Status` | SignalResponseStatus | Success/Error | | `Status` | SignalResponseStatus | Success/Error |
| `ResponseData` | byte[] | Binary-serialized payload (or GZip+JSON fallback) | | `ResponseData` | byte[] | Serialized payload |
| `DataSerializerType` | AcSerializerType | Binary or Json | | `DataSerializerType` | AcSerializerType | Binary or Json |
**Binary mode** (default): `AcBinarySerializer.ToBinary(data)` → raw bytes. Binary (default): `AcBinarySerializer.ToBinary(data)`. JSON fallback: `ToJson` → `GzipHelper.Compress`.
**JSON fallback**: `ToJson(data)` → `GzipHelper.Compress(json)` → bytes.
## Request/Response Flow ## Request/Response Flow
### Client → Server ### Client → Server
``` ```
1. PostAsync<T>(tag, postData) or PostDataAsync(tag, data, callback) 1. PostAsync<T>(tag, postData) / PostDataAsync(tag, data, callback)
2. CreatePostMessage(postData): 2. CreatePostMessage(postData):
├─ Primitives/strings/enums/value types → IdMessage (JSON array of serialized values) ├─ Primitives/strings/enums/value types → IdMessage
└─ Complex objects → SignalPostJsonDataMessage<T> ⚠️ tech debt: JSON-in-Binary └─ Complex → SignalPostJsonDataMessage<T> ⚠️ JSON-in-Binary tech debt
3. SerializeToBinary(message) wraps in Binary envelope 3. SerializeToBinary(message)
4. HubConnection.SendAsync("OnReceiveMessage", tag, bytes, requestId) 4. HubConnection.SendAsync("OnReceiveMessage", tag, bytes, requestId)
5. AcBinaryHubProtocol frames it on the wire 5. AcBinaryHubProtocol frames on wire
``` ```
### Server → Client ### Server → Client
``` ```
13. Client.OnReceiveMessage(tag, bytes, requestId) OnReceiveMessage(tag, bytes, requestId)
14. Has matching requestId in pending ConcurrentDictionary? ├─ Matching requestId in pending dict:
├─ YES (response to own request): │ ├─ DeserializeFromBinary<SignalResponseDataMessage>(bytes)
│ 15. DeserializeFromBinary<SignalResponseDataMessage>(bytes) │ ├─ Route: null→sync wait, Action→invoke, Func<Task>→await
│ 16. Route based on pending request type: │ └─ GetResponseData<T>(): Binary→BinaryTo<T>(), JSON→Decompress→Deserialize
│ ├─ null (sync wait) → set ResponseByRequestId, WaitToAsync completes └─ No match (broadcast):
│ ├─ Action<SignalResponseDataMessage> → invoke directly └─ abstract MessageReceived(tag, bytes).Forget()
│ └─ Func<SignalResponseDataMessage, Task> → invoke and await
│ 17. GetResponseData<T>():
│ ├─ Binary: ResponseData.BinaryTo<T>()
│ └─ JSON: GzipDecompress → AcJsonDeserializer.Deserialize<T>()
│
└─ NO (broadcast from another client's action):
18. abstract MessageReceived(tag, bytes).Forget()
└─ Consuming project overrides to handle server push
``` ```
**Broadcast receive:** When the server broadcasts to `Others`/`All`, receiving clients have no matching `requestId`. The message falls through to `MessageReceived(int messageTag, byte[] messageBytes)` — an abstract method the consuming project overrides. Request pooling: `SignalRRequestModel` via `SignalRRequestModelPool` (ObjectPool + IResettable).
**Request model pooling:** `SignalRRequestModel` instances are managed via `SignalRRequestModelPool` (ObjectPool<T> + IResettable) to avoid allocations per request.
## Response Patterns ## Response Patterns
| Pattern | Method | Blocking | | Pattern | Method | Blocking |
|---------|--------|----------| |---------|--------|----------|
| Sync wait | `PostAsync<T>(tag, data)` | Yes, polls up to `TransportSendTimeout` (60s) | | Sync wait | `PostAsync<T>(tag, data)` | Yes (60s timeout) |
| Async callback | `PostDataAsync(tag, data, async msg => {...})` | No, `Func<SignalResponseDataMessage, Task>` | | Async callback | `PostDataAsync(tag, data, async msg => {...})` | No |
| Fire-and-forget | `PostDataAsync(tag, data, msg => {...})` | No, `Action<SignalResponseDataMessage>` | | Fire-and-forget | `PostDataAsync(tag, data, msg => {...})` | No |
## Connection Lifecycle ## Connection
**Client configuration:** - WebSockets only (`SkipNegotiation = true`)
- Transport: WebSockets only (`SkipNegotiation = true`) - 30MB transport + 30MB application buffer
- Buffer: 30MB transport + 30MB application - `WithAutomaticReconnect()` + `WithStatefulReconnect()`
- Reconnection: `WithAutomaticReconnect()` + `WithStatefulReconnect()` - Keepalive 60s, server timeout 180s
- Keepalive: 60s interval, 180s server timeout
## Diagnostics ## Diagnostics
Enable with `AcSignalRClientBase.EnableBinaryDiagnostics = true`. `AcSignalRClientBase.EnableBinaryDiagnostics = true` — hex dump, header parsing, property enumeration.
Logs: hex dump (500 byte sample), header parsing (version, marker), property count + names via VarUInt reading. ## Tech Debt
## Known Technical Debt **JSON-in-Binary:** client→server wraps params in JSON inside binary envelope (`SignalPostJsonDataMessage`). Do NOT fix as side effect — requires coordinated cross-project changes.
**JSON-in-Binary request parameters:** Client→server requests currently wrap parameters in JSON inside the binary envelope (`SignalPostJsonDataMessage`). This adds an unnecessary serialization round-trip. Responses are already pure binary. Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects. ## Source Files
## Key Source Files
| Component | Path | | Component | Path |
|-----------|------| |-----------|------|

106
AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md Normal file
View File

@ -0,0 +1,106 @@
# SignalR Binary Protocol
`AcBinaryHubProtocol` — custom `IHubProtocol` (name: `"acbinary"`) replacing SignalR JSON+Base64 with `AcBinarySerializer`.
> Architecture (tag system, dispatch, request/response): `SIGNALR.md`
> Output writers (cached chunk, buffer states, chunk sizing): `AyCode.Core/docs/BINARY_WRITERS.md`
## Wire Format
```
[INT32 LE payload length] [1 byte message type] [message-specific fields]
```
| Type | Byte | Fields |
|------|------|--------|
| Invocation | 1 | nullable invocationId, target, arguments, streamIds, headers |
| StreamItem | 2 | invocationId, argument, headers |
| Completion | 3 | invocationId, nullable error, hasResult, optional result, headers |
| StreamInvocation | 4 | invocationId, target, arguments, streamIds, headers |
| CancelInvocation | 5 | invocationId, headers |
| Ping | 6 | (empty) |
| Close | 7 | nullable error, allowReconnect |
| Ack | 8 | sequenceId (INT64) |
| Sequence | 9 | sequenceId (INT64) |
**Argument framing:** `[INT32 LE arg length] [serialized bytes]` — deferred deserialization (target parsed first → `IInvocationBinder` resolves types).
**Strings:** `[VarUInt byte length] [UTF-8]`. Nullable: `0`=null, `1`=value follows.
**Headers:** `[VarUInt count] [key-value pairs...]`. Count=0 → no headers.
## Zero-Copy Write Pipeline
Writes directly to SignalR pipe's `IBufferWriter<byte>`, no intermediate `byte[]`.
```
WriteMessage(HubMessage, IBufferWriter<byte> output)
├─ Reserve 4-byte outer length prefix
├─ BWO = BufferWriterBinaryOutput(output) [standalone mode]
│ ├─ WriteByte(messageType)
│ ├─ WriteStringUtf8(invocationId, target)
│ ├─ WriteVarUInt(argCount)
│ ├─ Per argument:
│ │ ├─ byte[] → write through BWO (size known, no patching)
│ │ └─ object → FlushAndReset() → reserve INT32 arg prefix
│ │ → AcBinarySerializer.Serialize(value, output) → patch prefix
│ ├─ WriteStringArray(streamIds)
│ └─ WriteHeaders(headers)
├─ Patch outer length = BWO.Position + externalBytes
└─ BWO.Flush()
```
### Dual BWO Pattern
Protocol and serializer each create own `BufferWriterBinaryOutput` on the same `IBufferWriter`. Sequential, never concurrent:
1. **Protocol BWO** (standalone): framing — message type, strings, headers. Cached chunk, zero dispatch.
2. `FlushAndReset()`: commits bytes to pipe, invalidates chunk.
3. **Serializer BWO** (context mode): `Serialize()` creates internal BWO, acquires fresh chunk, writes, flushes.
4. Protocol BWO re-acquires chunk on next write (via `Grow`).
**Cost:** one extra `GetMemory` per argument (nanoseconds).
**Benefit:** zero-copy end-to-end, no intermediate `byte[]`, no wrapper class.
Why two BWOs: serializer writes must live on `BinarySerializationContext` (sealed class) for JIT optimization — context owns its own BWO. See `AyCode.Core/docs/BINARY_WRITERS.md` § "Why Writes Are on the Context".
### Length Prefix Patching
```csharp
var lengthSpan = output.GetSpan(4);
output.Advance(4);
// ... write payload ...
Unsafe.WriteUnaligned(ref lengthSpan[0], payloadLength);
```
Safe for `PipeWriter` — segments writable until `FlushAsync`.
**`GetMessageBytes` caveat:** `ArrayBufferWriter` initial capacity must include `LengthPrefixSize` to prevent resize after prefix reservation (stale span).
## byte[] Fast-Path
When argument is `byte[]`, bypasses serializer:
1. Size upfront: `1 (BinaryTypeCode) + VarUIntSize(length) + length`
2. INT32 prefix written with actual value (no patching)
3. `BinaryTypeCode.ByteArray(68)` + VarUInt length + raw bytes via BWO
Read side mirrors: if `targetType == typeof(byte[])` and first byte is `ByteArray`, deserializer bypassed → direct `SpanReader`.
## Read Path
`SpanReader` — `ref struct` for sequential `ReadOnlySpan<byte>` reading:
1. Read INT32 length. If `input.Length < total` → false (incomplete).
2. Multi-segment `ReadOnlySequence` → rent contiguous buffer from `ArrayPool`.
3. Parse message type → type-specific parser.
4. Fields via `SpanReader` methods (`ReadByte`, `ReadString`, `ReadVarUInt`, `ReadInt32`, `ReadInt64`, `ReadSpan`).
5. Arguments: INT32 length → slice → `AcBinaryDeserializer.Deserialize(span, targetType)`.
## Config
| Property | Default | Purpose |
|----------|---------|---------|
| `Options` | `AcBinarySerializerOptions.Default` | Serializer options (volatile, runtime-replaceable) |
| `BufferWriterChunkSize` | 65536 | Chunk size for both BWOs |
**Source:** `AyCode.Services/SignalRs/AcBinaryHubProtocol.cs`