AyCode.Core/AyCode.Core/Serializers
Loretta 4a6e101410 Unify AcBinary string marker; prefix-tier VarUInt encoding
Refactored AcBinary to use a single String marker (167) for long-form strings, replacing StringLen8/16/32. Implemented prefix-tier VarUInt encoding for string lengths, introduced FixStrCount constant, and removed legacy LEB128 code paths. Updated all serialization/deserialization logic and documentation to match the new format. Includes related micro-optimizations and code cleanup.
2026-05-26 16:24:33 +02:00
..
Attributes Fix SGen ref-handling asymmetry; add regression tests 2026-05-19 08:32:39 +02:00
Binaries Unify AcBinary string marker; prefix-tier VarUInt encoding 2026-05-26 16:24:33 +02:00
Expressions Document AcBinary wire format, sync docs, update conventions 2026-03-29 09:11:57 +02:00
Jsons [LOADED_DOCS: 3 files, no new loads] 2026-05-15 08:40:52 +02:00
Toons Remove depth param from serializers; use context field 2026-05-13 23:02:15 +02:00
AcSerializerCommon.cs [LOADED_DOCS: 3 files, no new loads] 2026-05-03 22:35:40 +02:00
AcSerializerContextBase.cs Refactor string serialization, CLI args, and test data 2026-05-21 21:03:03 +02:00
AcSerializerOptions.cs Refactor AcBinary string markers; charset/test renames 2026-05-22 20:04:11 +02:00
DeserializationContextBase.cs
DeserializeChainBase.cs
DeserializeCrossTypeBase.cs
DeserializeTypeMetadataBase.cs [LOADED_DOCS: 3 files, no new loads] 2026-05-03 22:35:40 +02:00
FnvHash.cs Enable cross-type deserialization via property hashes 2026-02-04 09:38:49 +01:00
IIdCollectionMergeHelper.cs Track buffer growth stats in DEBUG; disable IId merge helper 2026-01-29 09:41:53 +01:00
IdentityMap.cs Optimize serializer with write plan for interning & refs 2026-02-15 17:28:06 +01:00
PropertyAccessorBase.cs [LOADED_DOCS: 3 files, no new loads] 2026-05-03 22:35:40 +02:00
PropertyMetadataBase.cs [LOADED_DOCS: 3 files, no new loads] 2026-05-07 23:54:57 +02:00
PropertySetterBase.cs [LOADED_DOCS: 3 files, no new loads] 2026-05-03 22:35:40 +02:00
README.md [LOADED_DOCS: 3 files, no new loads] 2026-05-01 14:01:23 +02:00
ReferenceTracker.cs
SerializationContextBase.cs Optimize cache index assignment during scan pass 2026-02-06 15:48:48 +01:00
SerializeTypeMetadataBase.cs [LOADED_DOCS: 3 files, no new loads] 2026-05-03 22:35:40 +02:00
TypeMetadataBase.cs [LOADED_DOCS: 2 files, no new loads] 2026-05-10 19:01:30 +02:00
TypeMetadataWrapper.cs Polymorphic serialization: slot-based prefix system overhaul 2026-03-09 15:04:46 +01:00

README.md

Serializers

High-performance serialization framework — Binary, JSON, Toon — on shared infrastructure.

Folder Structure

Folder Purpose
Binaries/ High-performance binary serialization with two-phase scan+serialize
Jsons/ Custom JSON serialization using Utf8JsonWriter/Utf8JsonReader
Toons/ LLM-optimized Token-Oriented Object Notation with @meta/@data sections
Expressions/ Expression tree serialization (LINQ Expression ↔ DTO)
Attributes/ Source generator marker attributes

Shared Infrastructure (Root Files)

Core Base Classes

  • AcSerializerContextBase.cs — Generic base context AcSerializerContextBase<TMetadata, TOptions> with metadata caching, wrapper slot management, and context pooling. All three serializers derive from this.
  • AcSerializerOptions.cs — Base options: reference handling mode, max depth, custom property mapping, ThrowOnCircularReference.
  • TypeMetadataBase.cs — Cached type analysis: readable/writable properties, IId detection, complexity flags (HasComplexProperties, NeedsReferenceTracking, IsCollection).
  • TypeMetadataWrapper.cs — Wraps metadata with per-context tracking state: typed IdentityMap instances (Int32/Int64/Guid), SmallIdBitmap, polymorphic cache, source-gen reader/writer registry lookup.

Serialization/Deserialization Bases

  • SerializationContextBase.cs — Base for serialization contexts.
  • DeserializationContextBase.cs — Base for deserialization contexts.
  • SerializeTypeMetadataBase.cs — Base for format-specific serialize metadata.
  • DeserializeTypeMetadataBase.cs — Base for format-specific deserialize metadata.
  • DeserializeChainBase.cs — Deserialization chain pattern.
  • DeserializeCrossTypeBase.cs — Cross-type deserialization support.

Property Handling

  • PropertyAccessorBase.cs — Base for reading property values during serialization.
  • PropertySetterBase.cs — Base for writing property values during deserialization.
  • PropertyMetadataBase.cs — Base property metadata shared across formats.

Utilities

  • AcSerializerCommon.cs — Expression type checking, queryable type refs, ThreadLocal cache helpers, compiled constructor creation.
  • IdentityMap.cs — High-perf hash table optimized for small int keys (bitmap) + chaining for large keys. Used for reference tracking.
  • FnvHash.cs — Deterministic FNV-1a property name hashing (used in Binary UseMetadata mode).
  • ReferenceTracker.csReferenceEqualityComparer for object identity-based tracking dictionaries.
  • IIdCollectionMergeHelper.cs — Collection merge operations during PopulateMerge (handles orphaned item removal).

Architecture

Generic Specialization Pattern

AcSerializerContextBase<TMetadata, TOptions>
├─ BinarySerializationContext  <BinarySerializeTypeMetadata,  AcBinarySerializerOptions>
├─ JsonSerializationContext    <JsonSerializeTypeMetadata,     AcJsonSerializerOptions>
└─ ToonSerializationContext    <ToonSerializeTypeMetadata,     AcToonSerializerOptions>

Key Design Decisions

  • Sealed derived classes — All context implementations are sealed for JIT direct calling (no virtual dispatch on hot paths).
  • Context pooling — Contexts are reused across serializations via thread-safe pools. ResetTracking() clears reference maps on pool return.
  • Property ordering — Hierarchy-aware (base→derived) then alphabetical. Ensures stable property indices across type versions.
  • Typed reference access — No boxing for common ID types. Pre-cast delegates: RefIdGetterInt32, RefIdGetterInt64, RefIdGetterGuid.
  • Zero-allocation hot paths — Buffer owned by context, Span-based operations, ArrayPool for large allocations.
  • Global metadata cache — Thread-safe per TMetadata type, shared across all contexts of that serializer type.

Reference Handling Modes

Mode Description
None No tracking, fastest path
OnlyId Only IId<T> types tracked (Binary only)
All All reference types tracked (required for JSON $ref)