Simplify byte[] wire format in SignalR binary protocol
Refactored AcBinaryHubProtocol and AyCodeBinaryHubProtocol to remove the VarUInt length prefix for raw byte[] arguments. Now, the protocol writes a tag (0x44) followed directly by the raw bytes, with argLength implying the payload size. Updated read logic to match: on detecting the tag, the code skips it and returns the remaining bytes as the payload. Updated documentation to clarify the new fast-path, protocol roles, and AcBinary detection. Set BufferWriterChunkSize to 4096 for SignalR in the base protocol for better alignment with Kestrel. Marked the related issue as resolved.
This commit is contained in:
parent
f825552ae2
commit
27cac570be
|
|
@ -401,11 +401,9 @@ public class AcBinaryHubProtocol : IHubProtocol
|
|||
}
|
||||
else
|
||||
{
|
||||
// Raw byte[] (image, file, etc.): wrap with ByteArray tag + VarUInt length
|
||||
var argPayload = 1 + VarUIntSize((uint)byteArray.Length) + byteArray.Length;
|
||||
bw.WriteRaw(argPayload);
|
||||
// Raw byte[] (image, file, etc.): tag + raw bytes, no VarUInt (argLength implies size)
|
||||
bw.WriteRaw(1 + byteArray.Length);
|
||||
bw.WriteByte(BinaryTypeCode.ByteArray);
|
||||
bw.WriteVarUInt((uint)byteArray.Length);
|
||||
}
|
||||
|
||||
bw.WriteBytes(byteArray);
|
||||
|
|
@ -473,13 +471,11 @@ public class AcBinaryHubProtocol : IHubProtocol
|
|||
LogReadSingleArgument(argSlice, argLength, targetType);
|
||||
|
||||
// byte[] fast-path: first byte is BinaryTypeCode.ByteArray tag →
|
||||
// strip tag + VarUInt length prefix, return raw payload. No deserializer.
|
||||
// strip tag, rest is raw payload. No VarUInt length (argLength implies size).
|
||||
var argReader = new SequenceReader<byte>(argSlice);
|
||||
if (argReader.TryPeek(out byte tag) && tag == BinaryTypeCode.ByteArray)
|
||||
{
|
||||
argReader.Advance(1); // skip tag
|
||||
var payloadLength = (int)ReadVarUInt(ref argReader);
|
||||
return SequenceToByteArray(argReader.UnreadSequence.Slice(0, payloadLength));
|
||||
return SequenceToByteArray(argSlice.Slice(1));
|
||||
}
|
||||
|
||||
return DeserializeFromSequence(argSlice, targetType, _options);
|
||||
|
|
|
|||
|
|
@ -41,13 +41,11 @@ public class AyCodeBinaryHubProtocol : AcBinaryHubProtocol
|
|||
var argSlice = r.UnreadSequence.Slice(0, argLength);
|
||||
r.Advance(argLength);
|
||||
|
||||
// byte[] fast-path
|
||||
// byte[] fast-path: tag only, no VarUInt (argLength implies size)
|
||||
var argReader = new SequenceReader<byte>(argSlice);
|
||||
if (argReader.TryPeek(out byte tag) && tag == BinaryTypeCode.ByteArray)
|
||||
{
|
||||
argReader.Advance(1);
|
||||
var payloadLength = (int)ReadVarUInt(ref argReader);
|
||||
return SequenceToByteArray(argReader.UnreadSequence.Slice(0, payloadLength));
|
||||
return SequenceToByteArray(argSlice.Slice(1));
|
||||
}
|
||||
|
||||
// IsRawBytesData: return raw bytes, consumer deserializes
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@ CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are g
|
|||
|
||||
Custom `IHubProtocol` (`"acbinary"`), replaces JSON. Zero-copy write via `BufferWriterBinaryOutput` standalone mode + `AcBinarySerializer.Serialize(value, output)` directly to pipe. Zero-copy read via `SequenceReader<byte>` from pipe's `ReadOnlySequence`.
|
||||
|
||||
`AcBinaryHubProtocol` is the base (unsealed, generic). `AyCodeBinaryHubProtocol` derives from it (currently empty — exists for registration and future project-specific hooks). Register `AyCodeBinaryHubProtocol` in both client and server.
|
||||
`AcBinaryHubProtocol` is the base (unsealed) — general binary framing only. `AyCodeBinaryHubProtocol` derives from it with consumer-specific logic: `SignalParams` capture (via `OnArgumentRead` hook), `IsRawBytesData` path, `SignalDataType` type resolution. Register `AyCodeBinaryHubProtocol` in both client and server.
|
||||
|
||||
> Wire format, argument framing, dual BWO pattern, length prefix patching: `SIGNALR_BINARY_PROTOCOL.md`
|
||||
|
||||
|
|
@ -94,9 +94,9 @@ Typed access via methods (PostDataJson pattern):
|
|||
- Protocol never sees `byte[][]` — only `byte[]`.
|
||||
|
||||
`object data` (4th hub argument) — protocol handles three cases on read:
|
||||
1. **byte[] fast-path**: first byte is `BinaryTypeCode.ByteArray(0x44)` → strip tag + VarUInt length → return raw payload bytes. No deserializer.
|
||||
2. **IsRawBytesData**: `SignalParams.IsRawBytesData == true` → return entire argSlice as raw `byte[]`. No deserialization. Consumer handles deserialization.
|
||||
3. **Typed deserialization**: resolve type from `SignalParams.SignalDataType` → `AcBinaryDeserializer.Deserialize(sequence, type)` → return typed object.
|
||||
1. **byte[] fast-path**: first byte is `BinaryTypeCode.ByteArray(0x44)` → skip tag, rest is raw payload bytes. No VarUInt (argLength implies size). No deserializer.
|
||||
2. **IsRawBytesData** (AyCodeBinaryHubProtocol): `SignalParams.IsRawBytesData == true` → return entire argSlice as raw `byte[]`. No deserialization. Consumer handles deserialization.
|
||||
3. **Typed deserialization** (AyCodeBinaryHubProtocol): resolve type from `SignalParams.SignalDataType` → `AcBinaryDeserializer.Deserialize(sequence, type)` → return typed object.
|
||||
|
||||
`Parameters` and `data` are **independent** — both can be null or filled in any direction (SignalR is bidirectional).
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
# SignalR Binary Protocol
|
||||
|
||||
`AcBinaryHubProtocol` (unsealed base) — custom `IHubProtocol` (name: `"acbinary"`) replacing SignalR JSON+Base64 with `AcBinarySerializer`. `AyCodeBinaryHubProtocol` (derived, currently empty) exists for registration and future project-specific hooks.
|
||||
`AcBinaryHubProtocol` (unsealed base) — custom `IHubProtocol` (name: `"acbinary"`) replacing SignalR JSON+Base64 with `AcBinarySerializer`. General binary framing only — no consumer-specific logic.
|
||||
|
||||
`AyCodeBinaryHubProtocol` (derived) — project-specific consumer logic: `SignalParams` capture via `OnArgumentRead`, `IsRawBytesData` path, `SignalDataType` type resolution in `ReadSingleArgument` override.
|
||||
|
||||
> Architecture (tag system, dispatch, request/response): `SIGNALR.md`
|
||||
> Output writers (cached chunk, buffer states, chunk sizing): `AyCode.Core/docs/BINARY_WRITERS.md`
|
||||
|
|
@ -41,7 +43,8 @@ WriteMessage(HubMessage, IBufferWriter<byte> output)
|
|||
│ ├─ WriteStringUtf8(invocationId, target)
|
||||
│ ├─ WriteVarUInt(argCount)
|
||||
│ ├─ Per argument:
|
||||
│ │ ├─ byte[] → byte[] fast-path through BWO (size known, no patching)
|
||||
│ │ ├─ byte[] (AcBinary) → raw length + bytes (no tag, no VarUInt)
|
||||
│ │ ├─ byte[] (other) → tag (0x44) + raw bytes (no VarUInt — argLength implies size)
|
||||
│ │ └─ object → FlushAndReset() → reserve INT32 arg prefix
|
||||
│ │ → AcBinarySerializer.Serialize(value, output) → patch prefix
|
||||
│ ├─ WriteStringArray(streamIds)
|
||||
|
|
@ -50,6 +53,19 @@ WriteMessage(HubMessage, IBufferWriter<byte> output)
|
|||
└─ BWO.Flush()
|
||||
```
|
||||
|
||||
### byte[] Write: isAcBinary Detection
|
||||
|
||||
When argument is `byte[]`, the protocol checks if it's already AcBinary-serialized data:
|
||||
|
||||
```csharp
|
||||
var isAcBinary = byteArray.Length >= 2
|
||||
&& byteArray[0] == AcBinarySerializerOptions.FormatVersion // 1
|
||||
&& (byteArray[1] & 0xF0) == BinaryTypeCode.HeaderFlagsBase; // 0x90
|
||||
```
|
||||
|
||||
- **isAcBinary = true**: `[argLength=N][raw AcBinary bytes]` — no tag, reader deserializes directly
|
||||
- **isAcBinary = false**: `[argLength=1+N][0x44 tag][raw bytes]` — tag for type detection, no VarUInt (argLength implies size)
|
||||
|
||||
### Dual BWO Pattern
|
||||
|
||||
Protocol and serializer each create own `BufferWriterBinaryOutput` on the same `IBufferWriter`. Sequential, never concurrent:
|
||||
|
|
@ -77,36 +93,9 @@ Safe for `PipeWriter` — segments writable until `FlushAsync`.
|
|||
|
||||
**`GetMessageBytes` caveat:** `ArrayBufferWriter` initial capacity must include `LengthPrefixSize` to prevent resize after prefix reservation (stale span).
|
||||
|
||||
## Write: byte[] Fast-Path
|
||||
## Read: Argument Deserialization
|
||||
|
||||
When argument is `byte[]`, bypasses serializer entirely — writes through BWO with known size:
|
||||
|
||||
```
|
||||
WriteArgument(byte[] value):
|
||||
argPayload = 1 (BinaryTypeCode) + VarUIntSize(length) + length
|
||||
Write INT32 argPayload (no patching needed — size known upfront)
|
||||
Write BinaryTypeCode.ByteArray (0x44)
|
||||
Write VarUInt length
|
||||
Write raw bytes via BWO
|
||||
```
|
||||
|
||||
## Write: Object Zero-Copy Path
|
||||
|
||||
When argument is any other object, serializes directly to the pipe (zero-copy):
|
||||
|
||||
```
|
||||
WriteArgument(object value):
|
||||
FlushAndReset() BWO — hand pipe to serializer
|
||||
Reserve INT32 arg length prefix on pipe
|
||||
AcBinarySerializer.Serialize(value, output, options) — writes directly to pipe
|
||||
Patch arg length prefix with actual bytes written
|
||||
```
|
||||
|
||||
No intermediate `byte[]` — serializer writes to the pipe's `IBufferWriter` segments.
|
||||
|
||||
## Read: Three-Path Argument Deserialization
|
||||
|
||||
`ReadSingleArgument` reads `[INT32 argLength] [argBytes]` from the pipe's `ReadOnlySequence` via `SequenceReader<byte>`:
|
||||
Base `AcBinaryHubProtocol.ReadSingleArgument` reads `[INT32 argLength] [argBytes]` from the pipe's `ReadOnlySequence` via `SequenceReader<byte>`:
|
||||
|
||||
```
|
||||
ReadSingleArgument(SequenceReader, targetType):
|
||||
|
|
@ -119,26 +108,33 @@ ReadSingleArgument(SequenceReader, targetType):
|
|||
|
||||
1. byte[] fast-path:
|
||||
if first byte == BinaryTypeCode.ByteArray (0x44):
|
||||
skip tag + VarUInt length → return payload as byte[]
|
||||
Detection is wire-format only — 0x44 is unambiguous (no AcBinary object starts with it)
|
||||
return argSlice.Slice(1) as byte[] — skip tag, rest is raw payload
|
||||
No VarUInt — argLength implies size
|
||||
|
||||
2. Default: DeserializeFromSequence(argSlice, targetType, options)
|
||||
```
|
||||
|
||||
`AyCodeBinaryHubProtocol.ReadSingleArgument` overrides with consumer-specific paths:
|
||||
|
||||
```
|
||||
(same argLength/null/slice logic as base)
|
||||
|
||||
1. byte[] fast-path (same as base)
|
||||
|
||||
2. IsRawBytesData path:
|
||||
if _currentSignalParams.IsRawBytesData == true:
|
||||
return SequenceToByteArray(argSlice) — entire arg as raw byte[], no deserialization
|
||||
return SequenceToByteArray(argSlice) — entire arg as raw byte[]
|
||||
Consumer (DataSource.PopulateMerge) handles deserialization
|
||||
|
||||
3. Typed deserialization:
|
||||
if targetType == object && SignalDataType != null:
|
||||
resolve Type from SignalDataType (AssemblyQualifiedName)
|
||||
DeserializeFromSequence(argSlice, resolvedType, options)
|
||||
→ AcBinaryDeserializer.Deserialize(ReadOnlySequence, Type)
|
||||
→ single-segment: ArrayBinaryInput (zero-copy via TryGetArray)
|
||||
→ multi-segment: SequenceBinaryInput (lazy iteration, no pre-allocation)
|
||||
```
|
||||
|
||||
### SignalParams Capture
|
||||
|
||||
`_currentSignalParams` field captures the parsed `SignalParams` (arg[2]) during `ReadArguments`. The 4th arg (data) uses it for type-aware deserialization. Thread-safe: SignalR processes messages sequentially per connection.
|
||||
Base `AcBinaryHubProtocol.ReadArguments` calls `OnArgumentRead(value, index)` after each argument. `AyCodeBinaryHubProtocol` overrides this hook to capture `SignalParams` (arg[2]) for type-aware deserialization of subsequent args (arg[3] = data). Thread-safe: SignalR processes messages sequentially per connection.
|
||||
|
||||
### SequenceToByteArray
|
||||
|
||||
|
|
@ -156,9 +152,10 @@ Typical overhead for 225KB payload with 4096-byte segments: ~224.5KB zero-copy,
|
|||
|
||||
## Config
|
||||
|
||||
| Property | Default | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `Options` | `AcBinarySerializerOptions.Default` | Serializer options (volatile, runtime-replaceable) |
|
||||
| `BufferWriterChunkSize` | 65536 | Chunk size for both BWOs |
|
||||
| Property | Default | SignalR | Purpose |
|
||||
|----------|---------|---------|---------|
|
||||
| `Options` | `AcBinarySerializerOptions.Default` | — | Serializer options (volatile, runtime-replaceable) |
|
||||
| `BufferWriterChunkSize` | 65536 | 4096 | Chunk size for BWO. SignalR sets 4096 in `AcBinaryHubProtocol` constructor. |
|
||||
| `InitialBufferCapacity` | 16384 | — | Starting buffer for `ArrayBinaryOutput` (byte[] serialize path) |
|
||||
|
||||
**Source:** `AyCode.Services/SignalRs/AcBinaryHubProtocol.cs` (base), `AyCode.Services/SignalRs/AyCodeBinaryHubProtocol.cs` (derived)
|
||||
**Source:** `AyCode.Services/SignalRs/AcBinaryHubProtocol.cs` (base), `AyCode.Services/SignalRs/AyCodeBinaryHubProtocol.cs` (consumer logic)
|
||||
|
|
|
|||
|
|
@ -31,12 +31,10 @@ Uses `ToBinary()` / `BinaryTo()` exclusively. JSON parameter support would requi
|
|||
|
||||
### TRANS-1: BufferWriterChunkSize defaults to 64KB for SignalR
|
||||
|
||||
**Status:** Optimization opportunity
|
||||
**Affects:** `AyCodeBinaryHubProtocol` default constructor, write path
|
||||
**Status:** DONE
|
||||
**Affects:** `AcBinaryHubProtocol` constructor, write path
|
||||
|
||||
The default `BufferWriterChunkSize` is 65536 (from `AcBinarySerializerOptions.Default`). For SignalR/Kestrel, 4096 aligns better with the transport's internal segment size, reducing latency-to-first-byte.
|
||||
|
||||
**Plan:** Set `BufferWriterChunkSize = 4096` in `AyCodeBinaryHubProtocol` default constructor. The options property already exists (`AcBinarySerializerOptions.BufferWriterChunkSize`). Non-SignalR paths keep 64KB default.
|
||||
`BufferWriterChunkSize = 4096` set in `AcBinaryHubProtocol` constructor. Aligns with Kestrel slab size, reduces latency-to-first-byte. Non-SignalR paths keep 64KB default.
|
||||
|
||||
### TRANS-2: WebSocket buffer sizes are hardcoded
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue