Refactored binary deserialization to use a pooled BinaryDeserializationContextClass, eliminating per-call heap allocations and enabling cache reuse for string and intern caches. Introduced DeserializationContextClassPool for efficient context management. Updated all deserialization entry points to use the pool with proper disposal. Added efficient ReadOnlySequence<byte> support. Changed AcBinarySerializerOptions.UseMetadata default to false. These changes reduce GC pressure and improve performance, especially for high-throughput and WASM scenarios.
Refactored AcBinarySerializer to write the binary header directly after scanning for duplicates, removing the placeholder and patching logic for improved performance and simplicity. Introduced a high-performance, ArrayPool-backed PooledBufferWriter for efficient buffer management and pooling. Added a new Serialize<T> overload supporting direct serialization to IBufferWriter<byte>, enabling zero-copy and high-throughput scenarios. These changes streamline serialization, reduce memory copying, and enhance extensibility.
Refactored AcBinarySerializer to assign cache indices immediately upon detecting duplicates during the scan pass, eliminating the need for a separate post-processing step. Updated TryTrack methods to take a ref nextCacheIndex for inline assignment. Removed AssignCacheIndicesInOrder and related code, simplified string interning, and made RegisterMetadataType static. This reduces allocations and improves performance by making cache index assignment a single-pass operation.
Refactor binary serializer to use a true two-pass process for string interning and object reference tracking. Adds a scan pass to identify duplicates and assigns cache indices deterministically in first-occurrence order. Updates wire format to write explicit cache indices after *First markers. Refactors InternEntry, removes marker rewriting, and updates deserializer to match new format. Improves performance, correctness, and robustness for complex object graphs with shared references and repeated strings.
Introduce a lazily-computed ReferenceProperties array to AcBinarySerializer, containing only complex and string properties. This enables efficient iteration and reference tracking during serialization by filtering relevant properties on first access.
Move CacheMap building to dedicated method for efficiency.
Remove incremental cache logic and SourceHashes field.
Simplify property population and update documentation.
Improves performance and code clarity.
Refactored deserialization logic to use inline metadata for UseMetadata mode, replacing footer-based property hash lookup. Context now tracks inline metadata entries and builds cache maps incrementally per source type. Unified property population for both metadata modes. Updated TypeMetadataWrapper to manage cache map and source hashes per context. Improved robustness for runtime source type changes and streamlined object skipping logic. Updated method signatures to use BinaryPropertySetterBase abstraction.
Refactored AcBinarySerializer to write type property metadata inline
after the ObjectWithMetadata marker, eliminating the need for a
separate metadata footer section. Updated serialization, deserialization,
and diagnostic test logic to support the new inline metadata format.
Also updated settings.local.json to allow "Bash(git stash:*)" commands.
Implemented LZ4 compression/decompression in pure managed code, compatible with all platforms including WASM. Added new helpers (`Lz4`, `Lz4Compressor`, `Lz4Decompressor`) and a `Lz4CompressionMode` enum. Updated `AcBinarySerializerOptions` to support compression, and modified all relevant serializer methods to apply LZ4 when enabled. Benchmarks and buffer handling updated to support zero-allocation compression. No native dependencies required.
Introduce UseMetadata mode with FNV-1a property name hashing.
Write per-type property hashes to metadata footer for robust
property matching during deserialization. Remove legacy property
name table logic. Add ObjectWithMetadata marker and cachemap
logic for nested objects. Enable duplicate hash detection and
make UseMetadata default. Improves schema evolution support.
Update list factory cache and GetOrCreateListFactory to accept a capacity parameter, enabling preallocation of lists during deserialization. Adjust deserializer code to pass collection size where available, improving performance and memory usage. Also, pre-size dictionaries on creation and add Bash(find:*) to allowed commands in settings.local.json.
Reduce allocations by pooling int[] and object?[] arrays used for duplicate data and interned references during binary deserialization. Arrays are now rented and reused via ArrayPool<T>, with logic to clear or return them as appropriate. This improves performance and reduces GC pressure in steady-state scenarios.
Major refactor: merges string interning and object reference tracking (IId/Non-IId) into a unified position-based cache and footer in binary serialization. Wire format now uses cache indices for all references; hashcode/Id prefixes removed. Serialization and deserialization logic simplified, improving performance and maintainability. Legacy code paths and redundant dictionary lookups eliminated.
Replaces Dictionary-based string interning in AcBinarySerializer.BinarySerializationContext with a new IdentityMap<string, InternEntry> approach. Introduces the InternEntry struct for efficient tracking of stream position and cache index. Updates all related logic and iteration to use the new IdentityMap API, improving performance, memory usage, and code clarity for interned string and identity tracking during serialization.
IdentityMap is now fully generic as IdentityMap<TKey, TValue>, enabling type-safe value storage and improved flexibility. All internal logic and method signatures are updated to use TKey and TValue. The small int optimization for value storage is removed, and the _useSmallInt flag is disabled by default. Legacy IIdentityMap code is deleted. TypeMetadataWrapper is updated to use the new generic IdentityMap signatures. This refactor improves type safety, eliminates boxing, and prepares the code for value-type scenarios.
Clarified and restricted the "small int" optimization path to tracking (serialization) only, not value storage. Made _useSmallInt a constant, removed unnecessary _smallValues initialization, and improved comments for clarity. Updated code to safely handle uninitialized _smallValues and documented that hash table is preferred for value storage.
Reduced default pool sizes from 16 to 8 for serializers and object pools, now configurable via AcSerializerOptions.MaxContextPoolSize. Improved IdentityMap<TId> memory usage and cache locality by shrinking small int bitmap/array. Refactored hash table logic to use cached bucket length. Optimized Reset to clear only used entries and adjusted array pooling. String key equality now always uses ordinal comparison. Updated context pool logic to respect per-serializer pool size. Includes minor code cleanups and comments.
Refactor IId reference tracking with a new allocation-free, high-performance IdentityMap<TId> using bitmaps and pooled hash tables. Add async context cleanup for serializers, with pre-rented arrays for improved hot-path performance. Update AcSerializerOptions and context classes for better pooling, immutability, and platform support. Centralize and optimize array pooling and clearing to reduce memory pressure and GC impact.
Move IdentityMap and IdAccessorType to dedicated file and update all usages. Introduce pooling and cache-friendly optimizations for small int keys and hash tables, minimizing allocations and speeding up resets. Update buffer sizes and profiling loops. Add extensive comments and preserve old implementation for reference. This prepares the codebase for more efficient serialization reference tracking.
Add DEBUG-only tracking of buffer growth in AcBinarySerializer for benchmarking, with stats output in console app. Expose stats via static properties and reset at serialization start. Add InternalsVisibleTo for console access. Comment out IIdCollectionMergeHelper.cs. Minor code cleanups included.
Added output of GrowBufferCount and GrowBufferTotalBytes from BinarySerializationContext after the results table. This provides visibility into how often and how much the buffer grows during binary serialization.
The entire IdentityMap<TId> class and related types have been commented out, leaving the code inactive but preserved for reference. No functional code remains in the file; all logic is now present only as comments.
Update serializer and deserializer to use VarUInt for (position, cacheIndex) pairs in interned string footers, replacing fixed int32 format. This reduces serialized size and improves efficiency for small values. Adjust deserializer to read VarUInt pairs into a flat int[] array. Update comments and docs accordingly.
Replaces DupEntry[] with flat int[] for position-based string interning in AcBinaryDeserializer and AcBinarySerializer. Serializer now writes (position, cacheIndex) pairs as fixed int32s in bulk, and deserializer reads them with MemoryMarshal.Cast for ultra-fast, cache-friendly access. This eliminates per-pair parsing overhead and streamlines the hot path for string interning.
Refactored AcBinaryDeserializer to use a cached _nextDupPosition
for ultra-fast string interning (single int comparison). Updated
initialization/reset logic and streamlined RegisterInternedString
to avoid unnecessary array access and branching. Commented out
MinStringInternLength threshold checks to always register interned
strings. Simplified GetInternedString and made a minor update to
the serializer's analysis report wording. These changes improve
performance and reliability of string interning during
deserialization.
Implement a new position-based string interning mechanism in AcBinarySerializer/AcBinaryDeserializer. This approach tracks stream positions for interned strings, ensuring 100% reliable cache matching during deserialization, even when strings are skipped or reordered. The serializer now writes (position, cacheIndex) pairs in the footer for all repeated strings, and the deserializer uses this mapping for robust cache population. Removes the old buffer-based interned string logic, updates all relevant code paths, and simplifies interned string handling for greater correctness and maintainability. Also updates benchmarks and test data construction to use the new interning mode.
Introduce IsStringInternProperty to cache the [AcStringIntern] attribute value for each property. Update the constructor to initialize this property, and revise class documentation to reflect the new addition. This enables efficient access to string interning settings per property.
Simplifies property name registration by removing per-accessor
index caching and related reset logic. Deletes
RegisterPropertyNameAndCache and CachedPropertyNameIndex,
switching to direct property name registration. Also comments
out the global metadata cache in TypeMetadataBase, signaling
a move away from global caching. This reduces complexity and
potential for stale cache issues during context reuse.
Replaces boolean UseStringInterning with StringInterningMode enum for more granular control (None, Attribute, All). Introduces AcStringInternAttribute for per-property interning configuration. Updates all usages and documentation to reflect the new approach, ensuring explicit and flexible string interning behavior in serialization. Default mode is now All to preserve legacy behavior.
- Refactor all serializer options to use properties returning new instances (no shared mutable state); update all usages accordingly
- Extract AcSerializerOptions, BinaryTypeCode, and BinaryPropertyFilterContext to dedicated files for clarity and reuse
- Add DEBUG-only string interning analysis/reporting tools to AcBinarySerializer
- Improve AcBinarySerializer string property serialization with direct typed getter and SIMD-optimized ASCII path
- Increase benchmark/test warmup iterations and add JIT warmup delays for more reliable performance measurements
- Remove redundant usings and update documentation/comments throughout
- No breaking API changes, but static readonly options fields are now properties
- Add "profiler" mode for memory profiling AcBinary serialization
- Reduce warmup iterations from 10 to 5 for faster benchmarks
- Save large test binary output to separate .output file (hex dump)
- Improve robustness of AcBinary vs MessagePack result comparison
- Use DisplayName for test data in result output for clarity
- Optimize AcBinary string interning: use single contiguous buffer
- Update WriteFooterStrings to avoid per-string allocations
- Clarify WithoutReferenceHandling() disables string interning for speed
- Test data now controls IId shared ref % for realistic deduplication benchmarks; display names include IId ref ratio.
- Added deep-level clearing of IId refs for realistic object graphs.
- Pallet, Measurement, and Point models now support shared IId refs.
- TestDataFactory passes shared refs to all hierarchy levels.
- Refactored TypeMetadataWrapper for type-specific Id getters, identity maps, and registration—removes hot path type checks/switches.
- AcBinary deserializer now uses new typed methods for reference tracking and registration.
- SerializationContextBase uses pre-cast Id getters for zero-overhead tracking.
- Reduced quick benchmark warmup iterations for faster startup.
- Improves performance, clarity, and maintainability of reference handling and benchmarks.
- Unify and clarify object reference tracking for IId and non-IId types
- Always write/read Id as a normal property with type marker for IId types
- For non-IId types (All mode), use hashcode prefix for reference tracking
- Remove special Id prefix logic; all properties use type markers
- Centralize hashcode registration logic in deserializer
- Improve error handling for missing references
- Refactor tests to cover all ReferenceHandlingMode values and verify both data integrity and reference identity
- Add hex dump utility for debugging serialized bytes
- Make TypeMetadataBase.SourceType public for better diagnostics
- Make ReferenceHandlingMode type-aware; OnlyId fully supported for binary, All is default for JSON
- ReferenceHandling is now settable; add ThrowOnCircularReference option
- Always sort Id property first for IId types to optimize tracking
- Serialize/deserialize IId.Id without type marker when reference handling is enabled
- Contexts now delegate options-derived properties to Options
- Improve skip logic and property filter performance in binary serializer
- Update tests to explicitly set ReferenceHandlingMode.All
- Refactor internal APIs for clarity, safety, and efficiency
Refactored AcSerializerContextBase and all derived context classes to accept a generic TOptions parameter, ensuring type-safe access to serializer/deserializer options. Updated Reset methods and option-dependent properties to use the strongly-typed Options property. Added helper methods for reference handling checks and performed minor code cleanups for consistency. This improves type safety, reduces runtime errors, and clarifies context usage across serialization formats.
- Replace UseReferenceHandling bool with ReferenceHandlingMode enum across all serializers and options
- Move AcBinary string interning to footer (no header shifting); remove bloom filter
- Overhaul benchmark console: multi-serializer, grouped/colorized results, CSV/log output, more test data
- Set UseMetadata = false by default (header property names unused)
- Update all context Reset/Pool logic for new options signature
- Update AcJson/Toon serializers for new reference handling
- Update tests and usages for new enum-based options
- Add Newtonsoft.Json to benchmark dependencies
- Misc: code cleanups, improved comments, clarify logic
- Introduced PropertyMetadataBase to unify property metadata and dynamic getter logic, now shared by PropertyAccessorBase and PropertySetterBase.
- Moved PropertyAccessorType enum to PropertyMetadataBase.
- Replaced all GetDynamicValue usages with GetValue for consistent property access in serializers/deserializers.
- Refactored PropertyAccessorBase and PropertySetterBase inheritance and responsibilities.
- Added MaxStringInternLength option to AcBinarySerializerOptions for configurable string interning.
- Improved collection deserialization: clear destination if source is empty.
- Added AyCode.Core.Serializers.Console project for performance profiling (with MessagePack comparison).
- Updated solution file to include new project and build configs.
- Minor code cleanups and documentation improvements.
Move strongly-typed getter/setter logic and PropertyAccessorType enum into PropertyAccessorBase and PropertySetterBase, eliminating duplication in binary accessor classes. Expose direct typed getter/setter methods and new SetValueTyped/SetToDefault helpers. Rename ObjectGetter to DynamicGetter and update all serializers/deserializers to use GetDynamicValue. Centralize default value logic and improve performance by reducing boxing/unboxing. This unifies and streamlines property accessor infrastructure across all serializers.
Introduce SerializationContextBase and DeserializationContextBase to separate serialization and deserialization logic. Move tracking API from AcSerializerContextBase to SerializationContextBase, and add generic identity map methods to TypeMetadataWrapper for type-safe object tracking. Update all context classes to inherit from the new base classes. Comment out shared reference tracker and property mapping cache classes. Clean up unused usings and obsolete code. This improves code organization, performance, and maintainability.
Enhances both binary deserialization and JSON serialization to support IId-based reference tracking and handling. Refactors object reference reading in AcBinaryDeserializer to use a unified TryGetValue approach. Adds IId-aware tracking, ID writing, and reference lookup methods to AcJsonSerializer and updates ScanReferences and WriteObject to use them. Improves support for custom identity types and robust reference management.
Replaces flat object reference dictionary with per-type identity maps in deserializer, improving type safety and efficiency for IId types. TypeMetadataWrapper now uses cached typed delegates for reference ID access. Centralizes complex type detection and exposes IsComplexType and TypedIdGetter in metadata. Updates all registration and lookup logic to use wrappers, removes obsolete metadata cache, and ensures thread-safe, type-aware reference handling throughout. Comments out legacy code for easier review and rollback.
Unifies IId-based reference handling for binary serialization and deserialization. Introduces BinaryDeserializationContextClass for heap-based IId tracking, refactors IdentityMap<TId> for unified object storage, and removes legacy IIdReferenceTracker logic. Updates deserializer to use the new infrastructure for all IId types (int, long, Guid) and correct wire formats. Enhances tests for reference identity and object graph integrity. Improves code clarity and maintainability.
Refactored serialization reference tracking to a single-pass, inline approach, removing the previous two-pass scan. Only int, long, and Guid are now supported as IId<T> types; exotic ID types are no longer allowed. Cleaned up related enum values and code paths, defaulting non-IId types to int-based reference IDs. This simplifies the codebase and improves performance and type safety.
Major refactor of serialization infrastructure:
- Removed AcSerializeBase; replaced with AcSerializerContextBase<TMetadata> for unified context management.
- Added TypeMetadataWrapper<TMetadata> to combine metadata and per-context tracking state.
- All serializer contexts now inherit from AcSerializerContextBase and use context.GetWrapper(type) for metadata and tracking.
- Reference tracking for IId types is now type-safe and efficient (bitmaps for small int IDs, generic identity maps for others).
- Removed generic ThreadLocal caching from TypeMetadataBase; caching now uses global ConcurrentDictionary.
- Updated all type metadata classes to inherit from non-generic base.
- Added IdPropertyInfo and MetadataType to TypeMetadataBase.
- Added stub context base classes for JSON and Toon.
This centralizes and optimizes metadata/tracking, improves performance, and prepares for future extensibility.
- Rework SerializationReferenceTracker to use a unified Bloom filter + HashSet for both IId and reference-based tracking, improving efficiency and reducing allocations.
- Introduce AcSerializeBase as a common base class for serialization contexts; update Binary, JSON, and Toon contexts to inherit from it.
- Move AcBinaryDeserializationException, AcJsonDeserializationException, and TypeConversionInfo to separate files for better organization.
- Remove obsolete code and update documentation to reflect new reference tracking logic.
Introduce SerializeTypeMetadataBase<TMetadata> as a new abstract base class for serializer type metadata, extending TypeMetadataBase. Update Binary, JSON, and Toon serializer metadata classes to inherit from this new base, enabling shared serializer-specific logic and improving code organization. No functional changes to serialization behavior.
Replaces BinaryTypeMetadata, JsonTypeMetadata, and ToonTypeMetadata with BinarySerializeTypeMetadata, JsonSerializeTypeMetadata, and ToonSerializeTypeMetadata, moving each to its own file. Updates all references and documentation to use new names. Property accessor classes are retained and relocated. Also sets SignalR logging minimum level to Error. No changes to serialization logic; this is a structural/naming refactor for clarity and separation of concerns.
Adds robust IId<T>-based reference deduplication to both serialization and deserialization. Objects with the same type and Id are now treated as the same reference, reducing output size and ensuring correct reference identity in complex graphs.
Key changes:
- TypeMetadataBase: Adds IdAccessorType, typed Id getters, and precomputed property arrays for zero-boxing and fast access.
- AcSerializerCommon: Introduces IIdReferenceTracker for efficient (Type, Id) → object mapping.
- SerializationReferenceTracker: Now supports both ReferenceEquals and IId-based tracking for scanning and writing.
- AcBinarySerializer: Reference scan uses IId-aware deduplication, skipping types that don't need tracking.
- AcBinaryDeserializer: Adds per-context IId caches to ensure reference identity on deserialization.
- PropertyAccessorBase: Adds IsComplexType for fast scan decisions.
- Populate/Chain: Integrated with new IId cache for correct deduplication.
- Tests: Adds/updates tests for IId identity, cross-type safety, and diagnostics.
- Benchmarks: Adds WithRef/NoRef and AcJson vs System.Text.Json.
- Documentation: Includes detailed implementation plan and rationale.
No breaking changes for types that do not use IId. Zero-boxing for int/long/Guid Ids. Reference identity is now preserved for all IId objects.
- Support optional domain context in Toon serializer metadata (@meta section now includes "context" property if provided).
- Update internal and public APIs to accept and propagate domain context.
- Add overloads for Serialize, SerializeTypeMetadata, and SerializeMetadata to allow specifying domain context.
- Add AcBinarySerializerIIdReferenceTests and AcJsonSerializerIIdReferenceTests to verify IId-based reference deduplication and reference identity for both binary and JSON serializers.
- Tests cover both int and Guid IId scenarios, data integrity, and output size efficiency.
Loretta