Adam
/
AyCode.Core
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
AyCode.Core/AyCode.Services/docs/adr/0001-acbinary-decorator-fea...

20 KiB
Raw Blame History

ADR 0001: AcBinaryHubProtocol optional feature stack — decorator-based composition design

Status

Proposed (2026-04-25)

Context

AcBinaryHubProtocol (AyCode.Services/SignalRs/AcBinaryHubProtocol.cs) is the AyCode binary IHubProtocol implementation for SignalR — the canonical wire format for binary hub messaging. The base implementation handles AcBinary serialization, three protocol modes (Bytes / Segment / AsyncSegment), chunked AsyncSegment streaming, and the base layer of features every consumer needs.

A class of optional features is planned for NuGet competitiveness (currently P3 idea status, see ../SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_TODO.md SBP-T-5..8): encryption, compression with MinSize, OpenTelemetry tracing, HMAC signing-only. These features:

  • Are NOT current-priority — speculative additions for public NuGet positioning.
  • Must be opt-in (default-off, zero overhead when not used).
  • Must be stackable in some combinations (encrypt + sign + compress simultaneously).
  • Must work with all three BinaryProtocolMode send strategies (Bytes / Segment / AsyncSegment).
  • Each carries its own design + threat model decisions (own ADR per leaf feature).

This ADR establishes the composition mechanism — how these features attach to and stack with AcBinaryHubProtocol. Each leaf feature gets a follow-up ADR (0002 encryption, 0003 compression, 0004 tracing, 0005 signing) inheriting this composition decision. Without an explicit umbrella, every leaf ADR would re-derive the same composition pattern → 4× redundancy and risk of drift.

Decision

Adopt the decorator-chain pattern for composing optional features onto AcBinaryHubProtocol. Each feature is a self-contained IHubProtocol implementation that wraps an inner protocol; stacking order is encoded at DI registration time via composition.

AcHubProtocolDecoratorBase abstract base

A new abstract class in AyCode.Services/SignalRs/ (concrete API spec is impl-level, tracked as SBP-T-9 in SIGNALR_BINARY_PROTOCOL_TODO.md):

  • Delegates passthrough IHubProtocol members (Name, Version, TransferFormat, IsVersionSupported, GetMessageBytes) to the wrapped inner protocol.
  • Exposes protected IHubProtocol Inner { get; } to derived classes.
  • Declares virtual WriteMessage and TryParseMessage with default passthrough impl.
  • Each concrete decorator overrides ONLY the methods relevant to its feature (~10-15 lines of feature-specific code per decorator; no boilerplate per-impl).

Stacking order (canonical)

compress → encrypt → sign (innermost decorator wraps outwards):

AcBinaryHubProtocol  ←  CompressingHubProtocol  ←  EncryptingHubProtocol  ←  SigningHubProtocol
                                                                                       ↑
                                                                             (outermost wrapper)

DI registration:

services.AddSingleton<IHubProtocol>(sp =>
    new SignDecorator(
        new EncryptDecorator(
            new CompressDecorator(
                new AyCodeBinaryHubProtocol(opts)))));

Send order: AyCodeBinaryHubProtocol serializes → CompressingHubProtocol (innermost wrap) compresses → EncryptingHubProtocol encrypts → SigningHubProtocol (outermost) appends HMAC tag. Receive order is reversed.

Rationale: compressed ciphertext is uncompressible (compression must precede encryption to avoid wasted CPU AND CRIME/BREACH-class side-channel attacks — TLS 1.3 deprecated TLS-level compression for this reason); encrypt-then-MAC is the industry-standard signing+encryption combination order. The DI registration order encodes this canonically; non-canonical orderings emit a startup warning (validation in DI extension methods — follow-up #2).

Handshake-based feature negotiation

At connection setup (1 application-level handshake roundtrip after SignalR's protocol handshake), both sides exchange a feature manifest:

{
  "compression": { "active": bool, "algorithm": "lz4" | "brotli" | "zstd" | ... },
  "encryption":  { "active": bool, "algorithm": "aes-gcm" | "chacha20-poly1305", "keyId": "..." },
  "signing":     { "active": bool, "algorithm": "hmac-sha256" | "hmac-sha512", "keyId": "..." },
  "tracing":     { "active": bool, "samplingRate": float? }
}

MinSize for compression is sender-side config, NOT a handshake parameter — the two endpoints may use different MinSize thresholds. The receiver detects per-message via the IsCompressed wire flag (see below).

Mismatch handling: asymmetric registration (one side has decorator, other doesn't) → handshake fails fast with a logged reason at Error level. Per-message asymmetry (handshake-skip artefacts) is handled by the decorator-failure unified protocol below.

The app-level handshake mechanism's concrete shape (Option A: a special __ac_handshake HubMessage as first message; Option B: wire-level WriteHandshake virtual on AcHubProtocolDecoratorBase) is deferred to a follow-up — the umbrella commits to "handshake-based" but does not pick the mechanism. Tracked as part of SBP-T-9.

Per-message IsCompressed flag — semantic (A): standalone byte

The IsCompressed flag is a standalone byte in the wire format, NOT a value-range extension of the existing message-type byte. Wire format with the compression decorator active:

[INT32 LE length: 4 byte][IsCompressed: 1 byte][message-type byte: 1 byte][message body: N byte]
  • The INT32 length is the existing AcBinaryHubProtocol length prefix (unchanged).
  • The length value INCLUDES the IsCompressed byte: length = 1 (IsCompressed) + 1 (message-type) + N (body) = N + 2.
  • The IsCompressed byte takes values 0x00 (uncompressed) or 0x01 (compressed). Future extension values reserved (0x02..0xFF unused for now).
  • Per-message overhead: +1 byte / message constant, regardless of compressed-or-not state. Even uncompressed messages carry the 0x00 byte.

In AsyncSegment mode the IsCompressed byte sits at the SAME position (offset 4 in the wire — directly after the INT32 length prefix), BEFORE the 200 CHUNK_START marker. Per-chunk overhead: 0 bytes — the chunk-level framing ([201][UINT16 size][data]) is unchanged.

Why semantic (A) and not semantic (B) — value-range extension: an alternative considered was encoding the compressed flag into the unused high bit of the message-type byte (compressed values disjoint from {1..9, 200..202}), giving 0 byte overhead in the uncompressed case. Rejected because:

  • Adds branching complexity to the receiver (parse high bit, strip it, then dispatch on remaining bits).
  • Future feature flags would compete for the same byte's bit space.
  • 1 byte / message overhead is negligible at typical message sizes.
  • Simpler is better for a NuGet-public protocol.

Simple compression rule

IF handshake.compression.active AND message_size >= MinSize:
    → IsCompressed = 0x01, compress whole message bytes (no per-arg inspection)
ELSE:
    → IsCompressed = 0x00, passthrough

No per-arg inspection (no isAcBinary magic-byte gating, no mixed-message policy handling). If a developer sends raw byte[] (e.g., already-compressed JPEG) with handshake compression on, the bytes are compressed once more — at most ~1-5% size growth (negligible vs CPU cost; HTTP Content-Encoding: gzip follows the same "compress everything" pattern industry-wide).

The isAcBinary detection in the existing wire format remains a per-arg discriminator (first-byte 0x44 vs 0x01) for the byte[] fast-path's serialization branching — independent of compression decision.

Layer responsibility: the protocol decorator orchestrates (handshake state, MinSize check, IsCompressed flag emit/read); the AcBinarySerializer / AcBinaryDeserializer provide the algorithm code (Compress(bytes, algo) / Decompress(bytes, algo) static helpers). The protocol delegates to these helpers without owning the algorithm implementations.

BufferSize semantics — wire-chunk target with internal source-chunk derivation

AcBinaryHubProtocolOptions.BufferSize semantically denotes the wire-chunk target size (post-decoration), aligned to the SignalR / Kestrel slab convention (default 4096). The protocol transparently derives the actual source-chunk size by subtracting the cumulative per-chunk decoration overhead:

ActualSourceChunkSize = BufferSize - Σ(decorator.PerChunkOverhead)

Each decorator MUST expose a PerChunkOverhead constant (or MaxPerChunkOverhead worst-case bound for compression where output size is data-dependent). The decorator chain sums these at composition time. The chunking logic in AcBinaryHubProtocol reads ActualSourceChunkSize for source-chunk granularity.

Feature Algorithm Per-chunk overhead Type
Encryption AES-GCM +28 byte Fixed (12 nonce + 16 auth tag)
Encryption ChaCha20-Poly1305 +28 byte Fixed (12 nonce + 16 auth tag)
Signing HMAC-SHA256 +32 byte Fixed (256-bit tag)
Signing HMAC-SHA512 +64 byte Fixed (512-bit tag)
Compression LZ4 frame @ 4 KB input up to +32 byte (worst case) Bounded
Compression Zstd @ 4 KB input up to +28 byte (worst case) Bounded
Compression Brotli @ 4 KB input up to +160 byte (worst case) Bounded
Tracing (any) 0 byte Fixed

Result: user-facing BufferSize stays Kestrel-slab-aligned regardless of decoration stack. Wire-chunk size ≤ BufferSize is guaranteed; UINT16 chunk size prefix never overflows because BufferSize is well below 65535.

Constraint: BufferSize MUST be ≥ 256 byte (sanity floor); decoration stacks summing to ≥ BufferSize overhead are rejected at registration time.

AsyncSegment per-chunk decoration constraint

In AsyncSegment mode, decorators MUST operate per-chunk (each chunk independently encrypted / compressed / signed) to preserve the protocol's pipeline-parallelism guarantee. Per-message decoration in AsyncSegment defeats the purpose of chunked streaming (the receiver would have to accumulate the entire message before decoding could begin).

Bytes and Segment modes have no such constraint and may use per-message decoration freely.

UINT16 size prefix design intent (chunk-level)

The [201][UINT16 size][data] chunk framing intentionally caps wire-chunks at 65535 bytes. Larger chunks would defeat the AsyncSegment design: pipeline parallelism benefits diminish past Kestrel slab size (~4 KB), MTU-aligned (~1.5 KB) sizes are typical. A 4-byte INT32 size prefix per chunk would add 2× the overhead at multi-megabyte payloads (20+ chunks × 2 bytes saved per chunk vs the 4-byte alternative). UINT16 is the correct choice for the AsyncSegment use-case. This umbrella ADR codifies the design intent so leaf ADRs do not propose changing it as a side-effect.

Decorator-failure unified protocol

When a decorator's TryParseMessage cannot complete (decryption auth-tag fails, decompression error, signature mismatch, opaque feature data on a peer with no matching decorator):

  1. The decorator's TryParseMessage returns false (per IHubProtocol contract — message not parseable).
  2. The decorator logs the failure reason at Error level via the ILogger<> infrastructure (logger threaded through AcBinaryHubProtocolOptions.Logger).
  3. The SignalR HubConnection closes the connection on persistent parse-failure (per SignalR's standard behavior — never silent).

Never silent: a decorator MUST NOT swallow a parse error and return a partial / default message. If the decorator cannot extract the original message, the connection closes with a logged reason. This contract holds for ALL decorators (uniform failure semantics across the entire feature stack).

Backward compatibility (handshake-mismatch handling)

A v1 client (no decorators registered) connecting to a v2 server (decorators in chain), or vice versa, must NOT silently break. The handshake is the SOLE point of feature negotiation:

  • Handshake mismatch (one side advertises feature, other does not): handshake fails fast. The connection closes with a logged Error-level reason indicating the missing capability. Clients can retry with a reduced feature set (or accept the failure and signal a configuration error to ops).
  • Symmetric DI registration encouraged: for clean rolling upgrades, both ends of a connection should register matching decorators before either side enables a feature.
  • No on-wire fallback heuristic — the design relies on handshake state, not first-byte-detection or magic-byte sniffing on a per-message basis. This avoids the fragile heuristic disambiguation that earlier protocol designs (e.g., HTTP/1 vs HTTP/2 on the same port) had to engineer around.

Consequences

Positive

  • Single Responsibility: each feature lives in 1 class, 1 file, 1 concern. AcBinaryHubProtocol.cs (~1400 lines) stays unchanged.
  • Truly zero overhead when opted out: a decorator absent from the DI chain means the feature's code path does not exist in the runtime — not JIT-eliminated, not branch-not-taken, but literally no method to invoke. Contrast in-protocol embedding (Alt-2) where opt-out depends on JIT successfully eliminating null-checked branches.
  • Boilerplate mitigated by AcHubProtocolDecoratorBase: passthrough members live in the base; each concrete decorator only writes ~10-15 lines of actual feature code.
  • Composable: stacking order = DI registration order, runtime-explicit and reviewable.
  • Extensible: new feature = new MyFeatureDecorator : AcHubProtocolDecoratorBase class + DI extension method. Neither the base class nor existing decorators change.
  • Industry-standard handshake negotiation: matches TLS, HTTP/2 ALPN, WebSocket per-message-deflate, gRPC patterns. Familiar to .NET developers; checks the "polished NuGet" box for procurement / security audits.
  • Backward-compatibility paths preserved: v1 ↔ v2 mixed deployments fail fast at handshake (early, debuggable) instead of mysteriously corrupting messages.
  • Wire protocol stays minimal: the only NuGet-feature-related new wire byte is the per-message IsCompressed (1 byte). Encryption and signing add only their cryptographic data overhead (nonce + tag); tracing has zero wire impact.

Negative

  • 1 vtable indirection per protocol-method call per layer: 4-deep chain = up to 4 virtual call frames before reaching the inner AcBinaryHubProtocol. Negligible at message granularity (sub-microsecond), but real.
  • Debug stack up to 4 frames deeper when all features active: navigating call stacks in production debugging is mildly noisier.
  • Stacking order fixed at registration: no runtime-adaptive composition (e.g., "compress only on payloads > 1MB"); intra-decorator logic (like the MinSize policy in ADR-0003) handles per-message branching internally.
  • +1 byte / message constant overhead when compression decorator is in the chain (semantic A: the IsCompressed standalone byte) — accepted as the cost of architectural simplicity.
  • App-level handshake mechanism is new infrastructure: requires designing an upfront message-exchange before normal traffic. Adds 1 roundtrip to connection setup. Concrete shape (Option A vs B) deferred to a follow-up.

Follow-ups required

  1. AcHubProtocolDecoratorBase implementation — abstract class added to AyCode.Services/SignalRs/; tracked as new TODO entry SBP-T-9 in SIGNALR_BINARY_PROTOCOL_TODO.md. Implementation deferred until at least one leaf ADR (0002-0005) reaches Status: Accepted.
  2. Stacking-order validation in DI — AddAcEncryption / AddAcSigning / AddAcCompression extension methods should warn or throw if registered in a non-canonical order (concrete impl part of SBP-T-9).
  3. App-level handshake mechanism design — Option A (__ac_handshake HubMessage as first message) vs Option B (WriteHandshake virtual on AcHubProtocolDecoratorBase). Tracked as part of SBP-T-9.
  4. Benchmark scaffolding — message-granularity vtable cost measurement for the "negligible" claim. Each leaf ADR's acceptance criteria includes verifying this in their feature's context.
  5. Cross-references — Step 8 of the adr-author skill: this ADR links to the 4 forthcoming leaf ADRs (0002-0005); ../SIGNALR_BINARY_PROTOCOL/README.md gets a new ## Related ADRs section pointing back to 0001-acbinary-decorator-feature-stack-design.md.

Alternatives considered

  • In-protocol embedding (rejected): all features inline inside AcBinaryHubProtocol via null-checked _options branches. Loses SRP (AcBinaryHubProtocol.cs would grow from ~1400 to 2300+ lines), zero-overhead depends on JIT successfully eliminating null branches (not a static guarantee), and cross-feature ordering becomes hardcoded in method bodies.

    • Reversibility: low — switching to decorator later means a near-total rewrite.
    • Future flexibility: low — every new feature pressures the monolithic class.

  • Middleware pipeline (rejected): ASP.NET Core-style IAcHubProtocolMiddleware with next delegate chaining. Heavier infrastructure (per-message delegate allocation + closure capture), doesn't align with IHubProtocol.WriteMessage's sync-void contract, and the middleware abstraction shines for HTTP request branching — not linear feature stacking. Effectively reinvents the decorator pattern with extra ceremony.

  • Source-gen specialization (rejected): Roslyn generator emits compile-time-specialized AcBinaryHubProtocol_AesGcm_Lz4_Tracing types per feature combination. True zero overhead but combinatorial type explosion (4 features × on/off + algorithm choice → 16+ generated types), build-time complexity, and runtime configuration via appsettings becomes architecturally hard. Reconsider only if benchmarks reveal vtable indirection becoming non-negligible at extreme message rates.

    • Cost: very high.
    • Future flexibility: low — every feature change is generator surgery.

  • Per-message wire-marker byte ranges with magic-byte detection (rejected — earlier draft iteration): each feature would emit a wire-marker byte in 0x10..0xFF ranges, with the receiver dispatching via first-byte detection. Ruled out because (a) handshake-based negotiation is the industry-standard pattern and avoids fragile per-message heuristics; (b) per-feature wire-marker byte ranges add up to 4 bytes / message overhead in the 4-feature stack, vs the chosen 1 byte for IsCompressed plus handshake-decided everything else; (c) backward-compat fail-closed via handshake mismatch is cleaner than per-message disambiguation.

  • IsCompressed as message-type-byte value-range extension (semantic B) (rejected): encode the compressed flag into the unused high bit of the message-type byte, so uncompressed messages carry 0 added byte overhead. Adds branching complexity to receivers; future feature flags would compete for the same byte's bit space; the per-message overhead saving (1 byte) is negligible at typical message sizes. Semantic (A) — standalone byte — chosen for simplicity and forward extensibility.

Related

  • Related TODOs: ../SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_TODO.md SBP-T-5 (encryption), SBP-T-6 (compression+MinSize), SBP-T-7 (tracing), SBP-T-8 (signing); SBP-T-9 (AcHubProtocolDecoratorBase impl + handshake mechanism — to be added in Step 8 cross-reference round).
  • Forthcoming ADRs: 0002 (payload encryption), 0003 (message compression with MinSize), 0004 (OpenTelemetry tracing), 0005 (HMAC signing-only) — all will declare Depends on: ADR-0001.
  • Topic-folder cross-ref: ../SIGNALR_BINARY_PROTOCOL/README.md to add ## Related ADRs section in Step 8.