Major refactor of binary serialization codegen and runtime:
- Added property writer bridge methods for markerless/metadata paths
- Centralized object marker logic via new bridge methods
- Simplified SGen output: single bridge call replaces branching
- FixObj slot markers now supported in serialization/deserialization
- Refactored collection/dictionary element serialization
- Removed redundant WritePropertyMarkerless method
- Improved tests: use BinaryTypeCode constants, FixObj parsing
- Added InternalsVisibleTo for test project access
- Annotated TestSimpleClass for SGen support
Reduces generated code size, improves maintainability, and ensures correct handling of new binary format features.
Split WriteObjectProperties into markerless and metadata variants for clarity and performance. Adjust method inlining attributes to favor hot path optimization. Comment out WritePropertyValue and some AcBinaryBenchmark variants to streamline code and benchmarks. Improves maintainability and serialization efficiency.
Refactored BinaryTypeCode to reserve 0..63 for FixObj slot indices, enabling direct array access for object wrappers. Introduced a new polymorphic type prefix system for properties whose runtime type differs from their declared type, with first/repeated occurrence markers and combined ref-tracking support. Unified wrapper slot caching for SGen and runtime types, improving performance and eliminating dictionary lookups in hot paths. Updated code generation, tests, and constants to use the new slot system. Added new settings and utility scripts. Overall, serialization is now faster, more robust, and extensible.
Activate FastWire encoding for both serialization and deserialization, using UTF-16 and fixed-width lengths for strings. WireMode is now public and defaults to FastWire. Removes unused ObjectEnd marker, clarifying object end handling. This improves string (de)serialization speed at the cost of larger output.
Refactor serialization context to use precomputed boolean flags
(HasRefHandling, HasAllRefHandling, HasStringInterning) for faster
reference and string interning checks, replacing repeated enum
comparisons. Update source generator to emit code using these flags.
Add AcBinarySerializer.ScanOnly for isolated scan benchmarking.
Set MaxDepth in test options. Improves performance and maintainability.
Switch from caching next entry object to caching VisitIndex for write plan tracking in binary serialization. Updates TryConsumeWritePlanEntry and related logic for improved simplicity and performance. Comments revised to match new approach.
Pre-compute InternBit and reference handling flags in context to avoid repeated field access and shifting. Refactor all intern mode checks to use InternBit, improving performance and code clarity in generated code, property accessors, and scan/write passes. Optimize write plan entry access for faster serialization.
Introduce new test models for circular refs, update tests to stress reference handling, and enhance deserializer to support ObjectRefFirst/WithMetadataRefFirst type codes. Fix intern cache index assignment, track generated readers in TypeMetadataWrapper, and disable UseGeneratedCode by default. Update benchmarks for reliability and diagnostics. These changes strengthen reference resolution, circular ref support, and performance.
Added capacity-check-free "unsafe" write methods to BinarySerializationContext for all primitive and specialized types, plus ReserveCapacity for bulk writes. Introduced MinWriteSize property in BinarySerializeTypeMetadata to precompute worst-case buffer requirements per type. Improved documentation and added Copilot instructions to discourage code removal as a solution.
Refactored binary serializer to use a single wrapper slot per SGen type for both metadata registration and reference tracking. Removed slot-based IdentityMap arrays and scan pass tracking methods, replacing them with wrapper-based TryTrack logic. Updated generated code to use wrapper slots for all slot-based operations. Changed UseMetadata registration to use a MetadataSeen flag on the wrapper. Added fast slot-indexed wrapper access in context base. Default UseMetadata option is now false. Simplifies and optimizes SGen tracking, reducing dictionary lookups and unifying tracking logic.
- Set `[AcBinarySerializable(true)]` on all SharedTestModels types
- Default `UseMetadata` to true for property hash footer
- Reset slotted ID maps and cache indices in serializer context
- Reduce test iterations for DEBUG builds to speed up runs
- Add debug comments in context `Clear()` method
Implements a fully source-generated scan pass for duplicate/reference tracking in SGen binary serialization. The generator now emits ScanObject and ScanForDuplicates methods for each writer, handling null/depth checks, slot-based ref tracking (by Id or object hash), and recursive property scanning (strings, complex types, collections). String interning and reference tracking are feature-flagged via attributes. The runtime scan path now delegates to generated code when available, eliminating reflection and delegate overhead. Adds slot-based IdentityMap arrays to the serialization context for efficient duplicate detection. Also updates metadata, attributes, and test stubs to support these features.
Implements inline type metadata emission in the source generator, matching runtime TypeMetadataBase. Computes FNV-1a hashes for type and property names, stores them in generated code, and emits metadata when UseMetadata is enabled. Adds per-type slot allocation and tracking for first/repeated metadata writes. Removes runtime fallback for UseMetadata, ensuring all logic is handled inline. Updates property filtering/order to match runtime, and optimizes Int32/Int64 skip logic. Thread-safe slot allocation is used for metadata tracking.
- Generate writers for nested types using flat class names (Outer_Inner_Leaf) to ensure uniqueness and validity.
- Apply property filters in generated code for all non-markerless properties, matching runtime behavior.
- Emit skip labels for each property in generated code for correct control flow.
- Remove PropertyFilter check from IsDirectObjectWrite; generated code now handles filtering.
- Change default ReferenceHandlingMode to All.
- Make BinaryPropertyFilterContext constructor public.
- Increase release warmup iterations in Program.cs from 3000 to 5000.
All FastWire-related code paths, fields, and options have been commented out in both AcBinarySerializer and AcBinaryDeserializer. This removes support for fixed-width integer and UTF-16 encoding, forcing the use of compact VarInt and UTF-8 encoding exclusively. The WireMode option is also commented out, so FastWire can no longer be selected. This change reduces output size and simplifies the codebase, but may impact serialization/deserialization speed for some scenarios.
Introduces a WireMode enum to select between Compact (VarInt + UTF-8) and Fast (fixed-width + UTF-16) wire formats for binary serialization. Updates AcBinarySerializerOptions to include a WireMode property (default: Fast). Serialization and deserialization logic now conditionally uses fixed-width or variable-length encoding for integers and strings based on the selected mode, enabling a tradeoff between output size and performance.
- Always write length prefix before ASCII and UTF-8 strings in AcBinarySerializer for consistency and correctness.
- Increase release warmup iterations from 2000 to 3000 in Program.cs.
Introduces direct object write mode in source-generated binary serialization, allowing generated writers to bypass the runtime WriteObject pipeline for eligible complex properties. Adds detection of generated writer presence and IId<T> implementation, inlines reference tracking logic, and replaces .GetType() with typeof() for type safety. String interning attribute detection and precomputed interning flags ensure safe cursor alignment. Expands PropInfo structure and enables fast path for all generated writers. Improves serialization performance, correctness, and reduces runtime overhead. Adds new web fetch domains to settings.
Refactor AcBinarySerializer to use ReadOnlySpan<T> for bulk writing of primitive arrays and List<T>, replacing multiple specialized methods with a single TryWritePrimitiveCollection. This improves efficiency and reduces code duplication. Change default string interning mode to Attribute (opt-in). Update generated code path to allow reference tracking but not string interning. Adjust benchmarks to test correct serializer options. Reorder options for clarity.
Introduce [AcStringIntern] for selective string interning on properties. Update serializer to use a per-property eligibility flag, improving efficiency and control over which string fields participate in interning during binary serialization. Update test models and internal context accordingly.
Implement write plan mechanism for string interning and IId object reference tracking. Scan pass now builds pre-computed WriteDuplicateEntry instructions, eliminating hot path IdentityMap lookups and redundant getter calls in the write pass. Update BinarySerializationContext, tracking visit indices and managing write plan array. Refactor ScanInternString and TryTrack methods to record visit indices and build write instructions for all duplicate occurrences. Update write pass logic to consume write plan entries. Add debug validation for scan/write pass order. Update benchmarks and test harness. Set UseGeneratedCode default to false. Improves performance for scenarios with interning and reference tracking.
Refactored AcBinaryDeserializer to read the type code marker byte only once per property, eliminating redundant PeekByte/ReadByte calls and improving efficiency. Updated all property population branches to use the already-consumed type code. Adjusted handling of nested complex objects to rewind the marker byte when needed. Modified TryReadAndSetTypedValue to assume the marker is already consumed, removing unnecessary reads. Exception messages now report the actual type code read.
Added UseGeneratedCode option (default true) to AcBinarySerializerOptions and exposed it in the serialization context. The generated code fast path is now gated by this option, allowing users to enable or disable source-generated serialization. These changes improve deserialization performance, code clarity, and configurability.
Introduce JitDisassemblyBenchmark for analyzing JIT-generated x64 assembly of AcBinarySerializer hot paths, accessible via --jitasm. Refactor string interning logic to support per-property and string collection interning, adding IsStringCollectionProperty and ScanStringCollection. Update ScanPass and WriteString for finer-grained control. Remove DEBUG-only CurrentPropertyPath in favor of a more robust property tracking approach. Update usage instructions and clean up related code.
Removed Newtonsoft.Json from benchmarks and codebase. Added AcBinaryBufferWriterBenchmark using ArrayBufferWriter and AcBinarySerializer's buffer writer API. Optimized WriteStringUtf8 for ASCII fast path. Improved ArrayBinaryOutput buffer reuse and memory management. Introduced Reset method to IBinaryOutputBase and implemented it in outputs. Streamlined serializer benchmarks to focus on AcBinary and System.Text.Json.
Major serialization pipeline refactor: all hot-path buffer and position management is now owned by BinarySerializationContext<TOutput>, with all write methods inlined for zero virtual dispatch. TOutput (now struct, IBinaryOutputBase) handles only cold-path buffer management (Initialize, Grow, GetTotalPosition). ArrayBinaryOutput and BufferWriterBinaryOutput are simplified to buffer managers. IBinaryOutput and BinaryOutputBase are removed. All serialization logic now uses context write methods (~130 call sites updated). This yields significant performance gains by eliminating virtual/interface calls on the serialization hot path.
Major internal refactor: AcBinarySerializer and BinarySerializationContext are now generic on TOutput : BinaryOutputBase, enabling JIT devirtualization and eliminating virtual dispatch in hot serialization loops. All serialization logic (WriteValue, WriteObject, WriteArray, etc.) is now generic on TOutput and delegates buffer operations to the output instance (ArrayBinaryOutput or BufferWriterBinaryOutput). Context pooling is now per output type. All buffer management is moved to output classes. The public API is unchanged, but the internal architecture is now fully generic and ready for further JIT optimizations. Also disables the source generator and sets UseMetadata=false by default.
Refactored serialization for performance:
- Precompute type metadata (primitive, collection, element info) in TypeMetadataBase
- Remove runtime type caches from JsonUtilities
- Rewrite primitive/collection checks to use direct logic or metadata
- Update scan pass and serialization hot path to use wrappers/metadata
- Improve buffer management (halve oversized buffers)
- Increase profiler warmup iterations, comment out deserialization in hot path
- Clean up code and clarify documentation/comments
Reduces runtime overhead and memory usage, streamlines hot path execution.
Refactored AcBinarySerializer to write the binary header directly after scanning for duplicates, removing the placeholder and patching logic for improved performance and simplicity. Introduced a high-performance, ArrayPool-backed PooledBufferWriter for efficient buffer management and pooling. Added a new Serialize<T> overload supporting direct serialization to IBufferWriter<byte>, enabling zero-copy and high-throughput scenarios. These changes streamline serialization, reduce memory copying, and enhance extensibility.
Refactored AcBinarySerializer to assign cache indices immediately upon detecting duplicates during the scan pass, eliminating the need for a separate post-processing step. Updated TryTrack methods to take a ref nextCacheIndex for inline assignment. Removed AssignCacheIndicesInOrder and related code, simplified string interning, and made RegisterMetadataType static. This reduces allocations and improves performance by making cache index assignment a single-pass operation.
Refactor binary serializer to use a true two-pass process for string interning and object reference tracking. Adds a scan pass to identify duplicates and assigns cache indices deterministically in first-occurrence order. Updates wire format to write explicit cache indices after *First markers. Refactors InternEntry, removes marker rewriting, and updates deserializer to match new format. Improves performance, correctness, and robustness for complex object graphs with shared references and repeated strings.
Refactored AcBinarySerializer to write type property metadata inline
after the ObjectWithMetadata marker, eliminating the need for a
separate metadata footer section. Updated serialization, deserialization,
and diagnostic test logic to support the new inline metadata format.
Also updated settings.local.json to allow "Bash(git stash:*)" commands.
Implemented LZ4 compression/decompression in pure managed code, compatible with all platforms including WASM. Added new helpers (`Lz4`, `Lz4Compressor`, `Lz4Decompressor`) and a `Lz4CompressionMode` enum. Updated `AcBinarySerializerOptions` to support compression, and modified all relevant serializer methods to apply LZ4 when enabled. Benchmarks and buffer handling updated to support zero-allocation compression. No native dependencies required.
Introduce UseMetadata mode with FNV-1a property name hashing.
Write per-type property hashes to metadata footer for robust
property matching during deserialization. Remove legacy property
name table logic. Add ObjectWithMetadata marker and cachemap
logic for nested objects. Enable duplicate hash detection and
make UseMetadata default. Improves schema evolution support.
Major refactor: merges string interning and object reference tracking (IId/Non-IId) into a unified position-based cache and footer in binary serialization. Wire format now uses cache indices for all references; hashcode/Id prefixes removed. Serialization and deserialization logic simplified, improving performance and maintainability. Legacy code paths and redundant dictionary lookups eliminated.
Replaces Dictionary-based string interning in AcBinarySerializer.BinarySerializationContext with a new IdentityMap<string, InternEntry> approach. Introduces the InternEntry struct for efficient tracking of stream position and cache index. Updates all related logic and iteration to use the new IdentityMap API, improving performance, memory usage, and code clarity for interned string and identity tracking during serialization.
Reduced default pool sizes from 16 to 8 for serializers and object pools, now configurable via AcSerializerOptions.MaxContextPoolSize. Improved IdentityMap<TId> memory usage and cache locality by shrinking small int bitmap/array. Refactored hash table logic to use cached bucket length. Optimized Reset to clear only used entries and adjusted array pooling. String key equality now always uses ordinal comparison. Updated context pool logic to respect per-serializer pool size. Includes minor code cleanups and comments.
Refactor IId reference tracking with a new allocation-free, high-performance IdentityMap<TId> using bitmaps and pooled hash tables. Add async context cleanup for serializers, with pre-rented arrays for improved hot-path performance. Update AcSerializerOptions and context classes for better pooling, immutability, and platform support. Centralize and optimize array pooling and clearing to reduce memory pressure and GC impact.
Move IdentityMap and IdAccessorType to dedicated file and update all usages. Introduce pooling and cache-friendly optimizations for small int keys and hash tables, minimizing allocations and speeding up resets. Update buffer sizes and profiling loops. Add extensive comments and preserve old implementation for reference. This prepares the codebase for more efficient serialization reference tracking.
Add DEBUG-only tracking of buffer growth in AcBinarySerializer for benchmarking, with stats output in console app. Expose stats via static properties and reset at serialization start. Add InternalsVisibleTo for console access. Comment out IIdCollectionMergeHelper.cs. Minor code cleanups included.
Update serializer and deserializer to use VarUInt for (position, cacheIndex) pairs in interned string footers, replacing fixed int32 format. This reduces serialized size and improves efficiency for small values. Adjust deserializer to read VarUInt pairs into a flat int[] array. Update comments and docs accordingly.
Replaces DupEntry[] with flat int[] for position-based string interning in AcBinaryDeserializer and AcBinarySerializer. Serializer now writes (position, cacheIndex) pairs as fixed int32s in bulk, and deserializer reads them with MemoryMarshal.Cast for ultra-fast, cache-friendly access. This eliminates per-pair parsing overhead and streamlines the hot path for string interning.
Implement a new position-based string interning mechanism in AcBinarySerializer/AcBinaryDeserializer. This approach tracks stream positions for interned strings, ensuring 100% reliable cache matching during deserialization, even when strings are skipped or reordered. The serializer now writes (position, cacheIndex) pairs in the footer for all repeated strings, and the deserializer uses this mapping for robust cache population. Removes the old buffer-based interned string logic, updates all relevant code paths, and simplifies interned string handling for greater correctness and maintainability. Also updates benchmarks and test data construction to use the new interning mode.
Simplifies property name registration by removing per-accessor
index caching and related reset logic. Deletes
RegisterPropertyNameAndCache and CachedPropertyNameIndex,
switching to direct property name registration. Also comments
out the global metadata cache in TypeMetadataBase, signaling
a move away from global caching. This reduces complexity and
potential for stale cache issues during context reuse.
Replaces boolean UseStringInterning with StringInterningMode enum for more granular control (None, Attribute, All). Introduces AcStringInternAttribute for per-property interning configuration. Updates all usages and documentation to reflect the new approach, ensuring explicit and flexible string interning behavior in serialization. Default mode is now All to preserve legacy behavior.
- Refactor all serializer options to use properties returning new instances (no shared mutable state); update all usages accordingly
- Extract AcSerializerOptions, BinaryTypeCode, and BinaryPropertyFilterContext to dedicated files for clarity and reuse
- Add DEBUG-only string interning analysis/reporting tools to AcBinarySerializer
- Improve AcBinarySerializer string property serialization with direct typed getter and SIMD-optimized ASCII path
- Increase benchmark/test warmup iterations and add JIT warmup delays for more reliable performance measurements
- Remove redundant usings and update documentation/comments throughout
- No breaking API changes, but static readonly options fields are now properties
- Add "profiler" mode for memory profiling AcBinary serialization
- Reduce warmup iterations from 10 to 5 for faster benchmarks
- Save large test binary output to separate .output file (hex dump)
- Improve robustness of AcBinary vs MessagePack result comparison
- Use DisplayName for test data in result output for clarity
- Optimize AcBinary string interning: use single contiguous buffer
- Update WriteFooterStrings to avoid per-string allocations
- Clarify WithoutReferenceHandling() disables string interning for speed
- Make ReferenceHandlingMode type-aware; OnlyId fully supported for binary, All is default for JSON
- ReferenceHandling is now settable; add ThrowOnCircularReference option
- Always sort Id property first for IId types to optimize tracking
- Serialize/deserialize IId.Id without type marker when reference handling is enabled
- Contexts now delegate options-derived properties to Options
- Improve skip logic and property filter performance in binary serializer
- Update tests to explicitly set ReferenceHandlingMode.All
- Refactor internal APIs for clarity, safety, and efficiency
Refactored AcSerializerContextBase and all derived context classes to accept a generic TOptions parameter, ensuring type-safe access to serializer/deserializer options. Updated Reset methods and option-dependent properties to use the strongly-typed Options property. Added helper methods for reference handling checks and performed minor code cleanups for consistency. This improves type safety, reduces runtime errors, and clarifies context usage across serialization formats.
- Replace UseReferenceHandling bool with ReferenceHandlingMode enum across all serializers and options
- Move AcBinary string interning to footer (no header shifting); remove bloom filter
- Overhaul benchmark console: multi-serializer, grouped/colorized results, CSV/log output, more test data
- Set UseMetadata = false by default (header property names unused)
- Update all context Reset/Pool logic for new options signature
- Update AcJson/Toon serializers for new reference handling
- Update tests and usages for new enum-based options
- Add Newtonsoft.Json to benchmark dependencies
- Misc: code cleanups, improved comments, clarify logic
Loretta