# AcBinary Features

Advanced serialization features on top of the wire format. Wire format: `BINARY_FORMAT.md` | Options/presets: `BINARY_OPTIONS.md` | Internal architecture: `BINARY_IMPLEMENTATION.md` | Source generation: `BINARY_SGEN.md`.

> **Architectural framing — why use AcBinary at all?** See [`BINARY_WHYUSE.md`](BINARY_WHYUSE.md) for the category positioning vs wire-only serializers (Protobuf / MessagePack / MemoryPack), the three-pillar value proposition (graph integrity + bandwidth + streaming), real-world WASM SignalR reference numbers, fit/not-fit decision lists, and the framing for how to read AcBinary's benchmark numbers.

## Optimization Policy (LLM)

AcBinary is a **general-purpose serializer**, not a benchmark-only implementation.

When proposing or implementing performance work, optimize for broad real-world workloads and maintain balanced trade-offs across:

- mixed payload shapes (small/medium/large/deep)
- language distributions (ASCII-heavy, mixed Latin, multi-byte UTF-8 such as CJK)
- throughput, latency, allocation, and wire size

Do not accept a change solely because one benchmark cell improves. Any optimization should be validated across multiple representative scenarios and must avoid benchmark-specific overfitting.

## Compact Encoding Selection

The serializer applies compact encodings automatically:

| Data | Condition | Encoding | Savings |
|------|-----------|----------|---------|
| Integer | −16 ≤ v ≤ 47 | TinyInt (1 byte) | 2–5 bytes |
| String | ≤31 bytes UTF-8, any content | FixStr (1+N bytes) | 1 byte (no length prefix) |
| String | ≤31 bytes, pure ASCII | FixStrAscii (1+N bytes) | 1 byte + reader skips UTF-8 decode |
| String | >31 bytes, pure ASCII | StringAscii (1+VarUInt+N bytes) | reader skips UTF-8 decode |
| Object | type index < 64 | FixObj (1 byte) | 1–5 bytes (no VarUInt index) |
| String | empty | StringEmpty (1 byte) | 1+ bytes |
| Bool | — | True/False (1 byte) | no payload |

### ASCII marker-dispatch

The writer's `WriteStringWithDispatch` runs a single-pass UTF-8 encode and detects pure-ASCII content for free via `bytesWritten == charLength` (every UTF-16 char < 0x80 produces exactly 1 UTF-8 byte; non-ASCII chars always produce 2-4 bytes). Based on the result it emits one of four markers:

- `FixStrAscii` (135-166) — short ASCII (≤31 bytes)
- `FixStr` (103-134) — short UTF-8 (≤31 bytes, mixed/multi-byte content)
- `StringAscii` (167) — long ASCII (>31 bytes)
- `String` (91) — long UTF-8 (>31 bytes, mixed/multi-byte content)

The reader uses the marker as the ASCII-validity contract — `FixStrAscii` / `StringAscii` payloads byte→char widen directly via `Encoding.Latin1.GetString` (BCL SIMD-accelerated, ~memcpy class throughput), no UTF-8 decode, no run-time `Ascii.IsValid` scan. `FixStr` / `String` payloads use the custom 3-phase UTF-8 decoder (Vector256 ASCII prefix widen + DWORD ASCII batch + scalar multi-byte branch). Wire format unchanged across format versions — the new markers occupy previously-unused codepoints, so wire produced without ASCII detection (older writers) is forward-compatible.

## String Interning Protocol

Controls deduplication of repeated string values.

**Modes** (`StringInterningMode`):
- `None` — all strings inline, no overhead
- `Attribute` — only `[AcStringIntern]` properties interned (default)
- `All` — all strings within length limits interned

**Length limits:** `MinStringInternLength=4`, `MaxStringInternLength=64` (configurable).

**Wire protocol:**
1. Serializer pre-scans all eligible strings to build a plan (which strings repeat)
2. First occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]`
3. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]`
4. Single-occurrence strings: written as normal `String`/`FixStr` (no interning overhead)

## Reference Tracking

Prevents infinite loops and preserves object identity for repeated references.

**Modes** (`ReferenceHandlingMode`):
- `None` — no tracking (fastest, use when graph is a tree)
- `OnlyId` — track only `IId` objects (matched by ID value)
- `All` — track all reference types (two-phase scan required)

**Two-phase process:**
1. **Scan pass** (`ScanPass.cs`) — walks the object graph, detects multi-referenced objects and repeated strings. Builds a `WriteDuplicateEntry[]` array (the "write plan") containing `VisitIndex`, `CacheMapIndex`, `IsFirst`, and `Value` for each duplicate.
2. **Sort** — write plan entries are sorted by `VisitIndex` to match the write pass traversal order.
3. **Serialize pass** — consumes the sorted write plan via `TryConsumeWritePlanEntry()`. A cursor (`_nextWritePlanVisitIndex`) advances through the plan in O(1) — no dictionary lookups during serialization.

**Wire protocol:**
- First occurrence: `[ObjectRefFirst(70)] [VarUInt refCacheIndex] [object body...]`
- Subsequent: `[ObjectRef(65)] [VarUInt refCacheIndex]`

**Example — same object referenced twice:**

```
Input:  { Users: [userA, userA] }   (same instance)

Scan pass → WritePlan:
  [{VisitIndex:2, CacheMapIndex:0, IsFirst:true},
   {VisitIndex:3, CacheMapIndex:0, IsFirst:false}]

Wire output (Compact mode, ReferenceHandling=All):
  [version=1] [flags=0x96]  [VarUInt cacheCount=1]     ← header
  [FixObj(0)]                                           ← root object
    [Array(66)] [VarUInt(2)]                            ← Users array, 2 elements
      [ObjectRefFirst(70)] [VarUInt(0)] [props...]      ← userA, 1st occurrence
      [ObjectRef(65)] [VarUInt(0)]                      ← userA, 2nd (2 bytes only)
```

## Hybrid Execution Model (Runtime vs Source Generated)

Two execution modes, seamlessly interoperable in a single serialization run:

| Mode | Trigger | Property access | When to use |
|------|---------|----------------|-------------|
| **SGen** | `[AcBinarySerializable]` + `UseGeneratedCode=true` | `Unsafe.As<T>` direct | Hot-path types |
| **Runtime** | No attribute or `UseGeneratedCode=false` | Compiled delegates | 3rd-party types, fallback |

SGen root types use a **fast path** that skips the full dispatch chain (~12 calls → 3 checks). SGen children call directly into other SGen writers; non-SGen children fall back to runtime via bridge methods. Wire format is identical regardless of mode.

> Full SGen architecture, bridge methods, generated code patterns, wrapper slots: `BINARY_SGEN.md`

## NativeAOT Compatibility

Both execution modes work under NativeAOT publish. SGen is the recommended path; Runtime works but is significantly slower.

| Path | AOT compatible? | Performance vs JIT | Recommendation |
|------|-----------------|--------------------|----|
| **SGen** (`[AcBinarySerializable]` + `UseGeneratedCode=true`) | ✅ Full | Same as JIT | Production AOT default |
| **Runtime** (no attribute or `UseGeneratedCode=false`) | ✅ Functional | ~5-7x slower than SGen | Fallback for unattributed types |

### Two-axis AOT strategy

NativeAOT has two distinct constraints — both must be addressed for the Runtime path to work:

1. **Dynamic code generation prohibited.** `Expression.Compile()` and `Reflection.Emit` fail under AOT (no JIT engine in the binary). The 7 property-accessor + constructor factories in `AcSerializerCommon` use `if (!RuntimeFeature.IsDynamicCodeSupported)` runtime guards to fall back to plain reflection delegates (`PropertyInfo.GetValue/SetValue`, `ConstructorInfo.Invoke`).

2. **Reflection metadata trim.** The trimmer drops `Type.GetConstructor()` / `Type.GetProperties()` metadata for types not statically referenced. The library propagates `[DynamicallyAccessedMembers(PublicParameterlessConstructor | PublicProperties)]` from the public `Deserialize<T>` / `Serialize<T>` entry points down through the metadata classes. The `RequiredMembers` constant on `TypeMetadataBase` is the single source of truth for the DAMs requirement.

### Documented trimmer blind spots

Two well-known limitations require consumer cooperation:

- **Polymorphism via `obj.GetType()`** — `WritePropertyOrSkip` writes the runtime concrete type for polymorphic properties; the trimmer cannot follow `GetType()` flow. Consumer must root polymorphic concrete types: either via `[AcBinarySerializable]` on each (SGen path covers them) or `<TrimmerRootAssembly>` for the data-model assembly. The IL2072 warning is suppressed at this site with documented justification.
- **Nested types via property-type chain** — `List<TestOrderItem>` element type extracted via `Type.GetGenericArguments()` loses DAMs context. Same mitigation: `[AcBinarySerializable]` on each nested type, or `<TrimmerRootAssembly>`. The IL2072/IL2075 warnings are suppressed at the relevant sites with documented justification.

### Consumer guidance for AOT publish

- Annotate every type in the serialization graph with `[AcBinarySerializable]` — SGen path is AOT-clean and ~5-7x faster than the Runtime fallback.
- Or, for unattributed types: add `<TrimmerRootAssembly Include="MyDataModel" />` in the consumer's csproj to keep reflection metadata for the data-model assembly intact (cost: larger AOT binary).
- The `MessagePackSerializer.Standard` resolver has the same `Reflection.Emit` problem — use MessagePack source generators or skip MessagePack under AOT (the AcBinary benchmark project does the latter via `#if !AYCODE_NATIVEAOT`).
- `System.Text.Json` reflection mode similarly requires `JsonSerializerIsReflectionEnabledByDefault=true` AND consumer awareness; for AOT the source-gen path is preferred.

### Verified

The full benchmark suite (Small/Medium/Large/Repeated Strings/Deep Nested) runs cleanly under NativeAOT publish on Windows x64 + .NET 9. AcBinary SGen retains its **+12-35% speed and -22-33% wire-size advantage over MemoryPack** under AOT, identical to JIT. Runtime path is functional but ~5-7x slower (reflection-delegate overhead).

## Property Ordering

Properties are serialized in a deterministic order defined by `TypeMetadataBase.GetUnfilteredProperties()`:

1. Walk the inheritance chain from **derived → base** (`currentType.BaseType` loop)
2. At each level, collect declared public instance properties
3. Sort **alphabetically** (`StringComparer.Ordinal`) within each level
4. Result: **base properties first, then derived, alphabetical within each level**

This order is stable across serializer/deserializer as long as the type hierarchy doesn't change.

### Cross-Type Deserialization (UseMetadata)

When `UseMetadata=true`, property name hashes (FNV-1a via `FnvHash.ComputeString`) are written per type, enabling schema evolution:

- **Serializer** writes property hashes in the metadata section (`ObjectWithMetadata(69)`)
- **Deserializer** builds an index mapping array (`GetIndexMapping()`) — maps source property indices to destination via FNV-1a hash matching
- Allows deserialization across different property sets or ordering

When `UseMetadata=false`, properties are matched by **positional index only** — source and destination must have identical property layouts.

**Edge cases:**
- **Hash collision** (`CheckDuplicatePropName=true`, default): throws `InvalidOperationException`. ⚠️ When `false`: collision silently ignored — **risk of data corruption**, see [`BINARY_ISSUES.md#accore-bin-i-c5r7`](BINARY_ISSUES.md#accore-bin-i-c5r7-checkduplicatepropnamefalse-silently-corrupts-on-fnv-1a-hash-collision).
- **Source has unknown property** (not in destination): silently skipped via `SkipValue()`, no error.
- **Destination has extra property** (not in source): left at default value (new instance) or unchanged (populate mode).