Reduce allocations by pooling int[] and object?[] arrays used for duplicate data and interned references during binary deserialization. Arrays are now rented and reused via ArrayPool<T>, with logic to clear or return them as appropriate. This improves performance and reduces GC pressure in steady-state scenarios.
Major refactor: merges string interning and object reference tracking (IId/Non-IId) into a unified position-based cache and footer in binary serialization. Wire format now uses cache indices for all references; hashcode/Id prefixes removed. Serialization and deserialization logic simplified, improving performance and maintainability. Legacy code paths and redundant dictionary lookups eliminated.
Replaces Dictionary-based string interning in AcBinarySerializer.BinarySerializationContext with a new IdentityMap<string, InternEntry> approach. Introduces the InternEntry struct for efficient tracking of stream position and cache index. Updates all related logic and iteration to use the new IdentityMap API, improving performance, memory usage, and code clarity for interned string and identity tracking during serialization.
IdentityMap is now fully generic as IdentityMap<TKey, TValue>, enabling type-safe value storage and improved flexibility. All internal logic and method signatures are updated to use TKey and TValue. The small int optimization for value storage is removed, and the _useSmallInt flag is disabled by default. Legacy IIdentityMap code is deleted. TypeMetadataWrapper is updated to use the new generic IdentityMap signatures. This refactor improves type safety, eliminates boxing, and prepares the code for value-type scenarios.
Clarified and restricted the "small int" optimization path to tracking (serialization) only, not value storage. Made _useSmallInt a constant, removed unnecessary _smallValues initialization, and improved comments for clarity. Updated code to safely handle uninitialized _smallValues and documented that hash table is preferred for value storage.
Reduced default pool sizes from 16 to 8 for serializers and object pools, now configurable via AcSerializerOptions.MaxContextPoolSize. Improved IdentityMap<TId> memory usage and cache locality by shrinking small int bitmap/array. Refactored hash table logic to use cached bucket length. Optimized Reset to clear only used entries and adjusted array pooling. String key equality now always uses ordinal comparison. Updated context pool logic to respect per-serializer pool size. Includes minor code cleanups and comments.
Refactor IId reference tracking with a new allocation-free, high-performance IdentityMap<TId> using bitmaps and pooled hash tables. Add async context cleanup for serializers, with pre-rented arrays for improved hot-path performance. Update AcSerializerOptions and context classes for better pooling, immutability, and platform support. Centralize and optimize array pooling and clearing to reduce memory pressure and GC impact.
Move IdentityMap and IdAccessorType to dedicated file and update all usages. Introduce pooling and cache-friendly optimizations for small int keys and hash tables, minimizing allocations and speeding up resets. Update buffer sizes and profiling loops. Add extensive comments and preserve old implementation for reference. This prepares the codebase for more efficient serialization reference tracking.
Add DEBUG-only tracking of buffer growth in AcBinarySerializer for benchmarking, with stats output in console app. Expose stats via static properties and reset at serialization start. Add InternalsVisibleTo for console access. Comment out IIdCollectionMergeHelper.cs. Minor code cleanups included.
Added output of GrowBufferCount and GrowBufferTotalBytes from BinarySerializationContext after the results table. This provides visibility into how often and how much the buffer grows during binary serialization.
The entire IdentityMap<TId> class and related types have been commented out, leaving the code inactive but preserved for reference. No functional code remains in the file; all logic is now present only as comments.
Update serializer and deserializer to use VarUInt for (position, cacheIndex) pairs in interned string footers, replacing fixed int32 format. This reduces serialized size and improves efficiency for small values. Adjust deserializer to read VarUInt pairs into a flat int[] array. Update comments and docs accordingly.
Replaces DupEntry[] with flat int[] for position-based string interning in AcBinaryDeserializer and AcBinarySerializer. Serializer now writes (position, cacheIndex) pairs as fixed int32s in bulk, and deserializer reads them with MemoryMarshal.Cast for ultra-fast, cache-friendly access. This eliminates per-pair parsing overhead and streamlines the hot path for string interning.
Refactored AcBinaryDeserializer to use a cached _nextDupPosition
for ultra-fast string interning (single int comparison). Updated
initialization/reset logic and streamlined RegisterInternedString
to avoid unnecessary array access and branching. Commented out
MinStringInternLength threshold checks to always register interned
strings. Simplified GetInternedString and made a minor update to
the serializer's analysis report wording. These changes improve
performance and reliability of string interning during
deserialization.
Implement a new position-based string interning mechanism in AcBinarySerializer/AcBinaryDeserializer. This approach tracks stream positions for interned strings, ensuring 100% reliable cache matching during deserialization, even when strings are skipped or reordered. The serializer now writes (position, cacheIndex) pairs in the footer for all repeated strings, and the deserializer uses this mapping for robust cache population. Removes the old buffer-based interned string logic, updates all relevant code paths, and simplifies interned string handling for greater correctness and maintainability. Also updates benchmarks and test data construction to use the new interning mode.
Introduce IsStringInternProperty to cache the [AcStringIntern] attribute value for each property. Update the constructor to initialize this property, and revise class documentation to reflect the new addition. This enables efficient access to string interning settings per property.
Simplifies property name registration by removing per-accessor
index caching and related reset logic. Deletes
RegisterPropertyNameAndCache and CachedPropertyNameIndex,
switching to direct property name registration. Also comments
out the global metadata cache in TypeMetadataBase, signaling
a move away from global caching. This reduces complexity and
potential for stale cache issues during context reuse.
Replaces boolean UseStringInterning with StringInterningMode enum for more granular control (None, Attribute, All). Introduces AcStringInternAttribute for per-property interning configuration. Updates all usages and documentation to reflect the new approach, ensuring explicit and flexible string interning behavior in serialization. Default mode is now All to preserve legacy behavior.
- Refactor all serializer options to use properties returning new instances (no shared mutable state); update all usages accordingly
- Extract AcSerializerOptions, BinaryTypeCode, and BinaryPropertyFilterContext to dedicated files for clarity and reuse
- Add DEBUG-only string interning analysis/reporting tools to AcBinarySerializer
- Improve AcBinarySerializer string property serialization with direct typed getter and SIMD-optimized ASCII path
- Increase benchmark/test warmup iterations and add JIT warmup delays for more reliable performance measurements
- Remove redundant usings and update documentation/comments throughout
- No breaking API changes, but static readonly options fields are now properties
- Add "profiler" mode for memory profiling AcBinary serialization
- Reduce warmup iterations from 10 to 5 for faster benchmarks
- Save large test binary output to separate .output file (hex dump)
- Improve robustness of AcBinary vs MessagePack result comparison
- Use DisplayName for test data in result output for clarity
- Optimize AcBinary string interning: use single contiguous buffer
- Update WriteFooterStrings to avoid per-string allocations
- Clarify WithoutReferenceHandling() disables string interning for speed
- Test data now controls IId shared ref % for realistic deduplication benchmarks; display names include IId ref ratio.
- Added deep-level clearing of IId refs for realistic object graphs.
- Pallet, Measurement, and Point models now support shared IId refs.
- TestDataFactory passes shared refs to all hierarchy levels.
- Refactored TypeMetadataWrapper for type-specific Id getters, identity maps, and registration—removes hot path type checks/switches.
- AcBinary deserializer now uses new typed methods for reference tracking and registration.
- SerializationContextBase uses pre-cast Id getters for zero-overhead tracking.
- Reduced quick benchmark warmup iterations for faster startup.
- Improves performance, clarity, and maintainability of reference handling and benchmarks.
- Unify and clarify object reference tracking for IId and non-IId types
- Always write/read Id as a normal property with type marker for IId types
- For non-IId types (All mode), use hashcode prefix for reference tracking
- Remove special Id prefix logic; all properties use type markers
- Centralize hashcode registration logic in deserializer
- Improve error handling for missing references
- Refactor tests to cover all ReferenceHandlingMode values and verify both data integrity and reference identity
- Add hex dump utility for debugging serialized bytes
- Make TypeMetadataBase.SourceType public for better diagnostics
- Make ReferenceHandlingMode type-aware; OnlyId fully supported for binary, All is default for JSON
- ReferenceHandling is now settable; add ThrowOnCircularReference option
- Always sort Id property first for IId types to optimize tracking
- Serialize/deserialize IId.Id without type marker when reference handling is enabled
- Contexts now delegate options-derived properties to Options
- Improve skip logic and property filter performance in binary serializer
- Update tests to explicitly set ReferenceHandlingMode.All
- Refactor internal APIs for clarity, safety, and efficiency
Refactored AcSerializerContextBase and all derived context classes to accept a generic TOptions parameter, ensuring type-safe access to serializer/deserializer options. Updated Reset methods and option-dependent properties to use the strongly-typed Options property. Added helper methods for reference handling checks and performed minor code cleanups for consistency. This improves type safety, reduces runtime errors, and clarifies context usage across serialization formats.
- Replace UseReferenceHandling bool with ReferenceHandlingMode enum across all serializers and options
- Move AcBinary string interning to footer (no header shifting); remove bloom filter
- Overhaul benchmark console: multi-serializer, grouped/colorized results, CSV/log output, more test data
- Set UseMetadata = false by default (header property names unused)
- Update all context Reset/Pool logic for new options signature
- Update AcJson/Toon serializers for new reference handling
- Update tests and usages for new enum-based options
- Add Newtonsoft.Json to benchmark dependencies
- Misc: code cleanups, improved comments, clarify logic
- Introduced PropertyMetadataBase to unify property metadata and dynamic getter logic, now shared by PropertyAccessorBase and PropertySetterBase.
- Moved PropertyAccessorType enum to PropertyMetadataBase.
- Replaced all GetDynamicValue usages with GetValue for consistent property access in serializers/deserializers.
- Refactored PropertyAccessorBase and PropertySetterBase inheritance and responsibilities.
- Added MaxStringInternLength option to AcBinarySerializerOptions for configurable string interning.
- Improved collection deserialization: clear destination if source is empty.
- Added AyCode.Core.Serializers.Console project for performance profiling (with MessagePack comparison).
- Updated solution file to include new project and build configs.
- Minor code cleanups and documentation improvements.
Move strongly-typed getter/setter logic and PropertyAccessorType enum into PropertyAccessorBase and PropertySetterBase, eliminating duplication in binary accessor classes. Expose direct typed getter/setter methods and new SetValueTyped/SetToDefault helpers. Rename ObjectGetter to DynamicGetter and update all serializers/deserializers to use GetDynamicValue. Centralize default value logic and improve performance by reducing boxing/unboxing. This unifies and streamlines property accessor infrastructure across all serializers.
Introduce SerializationContextBase and DeserializationContextBase to separate serialization and deserialization logic. Move tracking API from AcSerializerContextBase to SerializationContextBase, and add generic identity map methods to TypeMetadataWrapper for type-safe object tracking. Update all context classes to inherit from the new base classes. Comment out shared reference tracker and property mapping cache classes. Clean up unused usings and obsolete code. This improves code organization, performance, and maintainability.
Enhances both binary deserialization and JSON serialization to support IId-based reference tracking and handling. Refactors object reference reading in AcBinaryDeserializer to use a unified TryGetValue approach. Adds IId-aware tracking, ID writing, and reference lookup methods to AcJsonSerializer and updates ScanReferences and WriteObject to use them. Improves support for custom identity types and robust reference management.
Replaces flat object reference dictionary with per-type identity maps in deserializer, improving type safety and efficiency for IId types. TypeMetadataWrapper now uses cached typed delegates for reference ID access. Centralizes complex type detection and exposes IsComplexType and TypedIdGetter in metadata. Updates all registration and lookup logic to use wrappers, removes obsolete metadata cache, and ensures thread-safe, type-aware reference handling throughout. Comments out legacy code for easier review and rollback.
Unifies IId-based reference handling for binary serialization and deserialization. Introduces BinaryDeserializationContextClass for heap-based IId tracking, refactors IdentityMap<TId> for unified object storage, and removes legacy IIdReferenceTracker logic. Updates deserializer to use the new infrastructure for all IId types (int, long, Guid) and correct wire formats. Enhances tests for reference identity and object graph integrity. Improves code clarity and maintainability.
Refactored serialization reference tracking to a single-pass, inline approach, removing the previous two-pass scan. Only int, long, and Guid are now supported as IId<T> types; exotic ID types are no longer allowed. Cleaned up related enum values and code paths, defaulting non-IId types to int-based reference IDs. This simplifies the codebase and improves performance and type safety.
Major refactor of serialization infrastructure:
- Removed AcSerializeBase; replaced with AcSerializerContextBase<TMetadata> for unified context management.
- Added TypeMetadataWrapper<TMetadata> to combine metadata and per-context tracking state.
- All serializer contexts now inherit from AcSerializerContextBase and use context.GetWrapper(type) for metadata and tracking.
- Reference tracking for IId types is now type-safe and efficient (bitmaps for small int IDs, generic identity maps for others).
- Removed generic ThreadLocal caching from TypeMetadataBase; caching now uses global ConcurrentDictionary.
- Updated all type metadata classes to inherit from non-generic base.
- Added IdPropertyInfo and MetadataType to TypeMetadataBase.
- Added stub context base classes for JSON and Toon.
This centralizes and optimizes metadata/tracking, improves performance, and prepares for future extensibility.
- Rework SerializationReferenceTracker to use a unified Bloom filter + HashSet for both IId and reference-based tracking, improving efficiency and reducing allocations.
- Introduce AcSerializeBase as a common base class for serialization contexts; update Binary, JSON, and Toon contexts to inherit from it.
- Move AcBinaryDeserializationException, AcJsonDeserializationException, and TypeConversionInfo to separate files for better organization.
- Remove obsolete code and update documentation to reflect new reference tracking logic.
Introduce SerializeTypeMetadataBase<TMetadata> as a new abstract base class for serializer type metadata, extending TypeMetadataBase. Update Binary, JSON, and Toon serializer metadata classes to inherit from this new base, enabling shared serializer-specific logic and improving code organization. No functional changes to serialization behavior.
Replaces BinaryTypeMetadata, JsonTypeMetadata, and ToonTypeMetadata with BinarySerializeTypeMetadata, JsonSerializeTypeMetadata, and ToonSerializeTypeMetadata, moving each to its own file. Updates all references and documentation to use new names. Property accessor classes are retained and relocated. Also sets SignalR logging minimum level to Error. No changes to serialization logic; this is a structural/naming refactor for clarity and separation of concerns.
Adds robust IId<T>-based reference deduplication to both serialization and deserialization. Objects with the same type and Id are now treated as the same reference, reducing output size and ensuring correct reference identity in complex graphs.
Key changes:
- TypeMetadataBase: Adds IdAccessorType, typed Id getters, and precomputed property arrays for zero-boxing and fast access.
- AcSerializerCommon: Introduces IIdReferenceTracker for efficient (Type, Id) → object mapping.
- SerializationReferenceTracker: Now supports both ReferenceEquals and IId-based tracking for scanning and writing.
- AcBinarySerializer: Reference scan uses IId-aware deduplication, skipping types that don't need tracking.
- AcBinaryDeserializer: Adds per-context IId caches to ensure reference identity on deserialization.
- PropertyAccessorBase: Adds IsComplexType for fast scan decisions.
- Populate/Chain: Integrated with new IId cache for correct deduplication.
- Tests: Adds/updates tests for IId identity, cross-type safety, and diagnostics.
- Benchmarks: Adds WithRef/NoRef and AcJson vs System.Text.Json.
- Documentation: Includes detailed implementation plan and rationale.
No breaking changes for types that do not use IId. Zero-boxing for int/long/Guid Ids. Reference identity is now preserved for all IId objects.
- Support optional domain context in Toon serializer metadata (@meta section now includes "context" property if provided).
- Update internal and public APIs to accept and propagate domain context.
- Add overloads for Serialize, SerializeTypeMetadata, and SerializeMetadata to allow specifying domain context.
- Add AcBinarySerializerIIdReferenceTests and AcJsonSerializerIIdReferenceTests to verify IId-based reference deduplication and reference identity for both binary and JSON serializers.
- Tests cover both int and Guid IId scenarios, data integrity, and output size efficiency.
Expanded use of the ToonDescription attribute across core entities and DTOs to provide rich metadata, including business rules, purposes, and status flags. Added class-level and property-level annotations to improve self-documentation and support automated tooling. Introduced new computed properties with ToonDescription in ProductDto, StockQuantityHistoryDto, and StockTakingItem. Updated ToonTypeRelation with an Entity constant. Enhanced test coverage for Toon metadata. Cleaned up imports and removed obsolete test_debug.ps1 script. Updated settings.local.json to support additional Bash commands for tooling. These changes improve introspectability and support for serialization, UI, and API documentation.
Introduce "business-logic" field in AcToonSerializer type metadata output, sourced from ToonDescription attributes on DTO properties. Annotate relevant OrderDto and OrderItemDto properties with business rules and constraints. Expand allowed Bash commands in settings.local.json. Add test script to verify business-logic metadata presence. Temporarily disable HasToonIgnoreAttribute logic in JsonUtilities.
Replaced Array.Empty<ToonPropertyAccessor>() with the new C# 12 collection expression [] for initializing the Properties field as an empty array, resulting in more concise and modern code.
Major refactor: split AcToonSerializer.MetaSection.cs into focused modules for meta writing, type/enum definitions, navigation, foreign key, validation, descriptions, placeholders, topological sort, and attribute detection. Extend ToonDescriptionAttribute with BusinessRule, TypeRelation, and RelatedTypes for richer metadata. Add ToonTypeRelation constants. Annotate all DTOs with ToonDescription for type relationships. Refactor TypeMetadataBase for customizable ignore filters. Update tests and settings. Improves maintainability, extensibility, and metadata accuracy.
- Introduce AcNavigationPropertyInfo for unified, cached relationship metadata per property (primary key, navigation type, FK, other key, inverse, target type)
- Refactor relationship detection to use AcNavigationPropertyInfo, replacing ad-hoc logic and old RelationshipMetadata
- Add AcSerializerCommon.GetCSharpTypeName for consistent, C#-style type name formatting (handles primitives, generics, nullables, enums, collections)
- Use topological sort for @types output to ensure dependency order
- Improve enum handling: avoid redundant constraints, use new type name formatter for underlying types
- Output navigation, foreign-key, other-key, and inverse-property metadata consistently in meta section
- Enhance convention-based detection for inverse properties and "other key", including unidirectional/polymorphic support
- Add comprehensive test for navigation metadata completeness and demo test entities
- Add "source-code-language: C#" to meta section
- Misc: code cleanup, remove unused cache, improve property filtering
- Remove redundant constraints (nullable, numeric, boolean) from metadata; only explicit [Required] is documented.
- Exclude enum values from constraints; add "readonly" for readonly/init-only properties.
- Filter out primitives from documented types; only complex types and enums are included.
- Detect and document enum backing fields with "enum-type" constraint.
- Only output descriptions if explicitly provided; no fallback/inferred text.
- Add "not-mapped" constraint for [NotMapped]/[NotColumn] properties.
- Switch FruitBankHybrid.Shared.Tests.csproj to direct AyCode.Core project reference.
- Add both LinqToDB and DataAnnotations [Table] attributes to DTOs for ORM compatibility.
- Refined type metadata serialization: collections and dictionaries are now detected and described more accurately, avoiding generic type names (e.g., List`1) and redundant "object" element types.
- Added circular reference protection to type name generation to prevent stack overflows and duplicate type names.
- Updated AcToonSerializerOptions.Compact to use indentation for better readability.
- Introduced ToonTests with unit tests to ensure type metadata correctness, uniqueness, and clarity.
- Added AyCode.Core project to the solution and adjusted namespaces/usings for consistency.
Introduced AcToonSerializer, a new high-performance, LLM-friendly serialization format (Toon) with explicit meta/types/data sections, rich type metadata, and smart inference for property documentation. Added ToonDescriptionAttribute for custom schema annotations and relationship metadata. Implemented fallback/placeholder system merging custom, Microsoft, and inferred metadata. Enhanced logging: AcLoggerBase now implements ILogger, with provider/extensions for DI integration. Updated SignalR client to use AcLogger. Added ToonExtendedInfo.txt with full documentation. No breaking changes to existing serialization.
- Add automatic detection of primary/foreign keys, navigation types, and table names via ToonDescription, EF Core/Linq2Db attributes, or convention
- Extend ToonDescriptionAttribute with IsPrimaryKey, ForeignKey, Navigation, InverseProperty, and TableName
- Output relationship and table metadata in @types section (enhanced and simple modes)
- Document enums in @types with numeric values and descriptions
- Optimize token usage: compact output when indentation is off
- Show dictionary key/value types in output
- Add SerializeMetadata(IEnumerable<Type>) API for multi-type docs
- Refactor and improve documentation throughout
Introduces Toon (Token-Oriented Object Notation), a new serialization format designed for LLM readability and token efficiency. Adds core Toon serializer, options, attribute system (ToonDescriptionAttribute), and comprehensive documentation. Supports explicit type metadata, smart fallback/placeholder logic, multi-turn workflows, reference handling, and multi-line strings. No breaking changes; Toon is opt-in and complements existing JSON/binary serializers.
AyCode.Core loggers now implement the ILogger interface, enabling direct integration with Microsoft.Extensions.Logging. Added AcLoggerProvider and extension methods for easy DI registration. Internal LogLevel usages are now AcLogLevel to avoid confusion. This allows seamless use of AyCode loggers in ASP.NET Core and other .NET apps using standard logging abstractions.
Now the /reports directory will be ignored by Git, preventing it from being tracked in the repository. This helps keep generated or report files out of version control.
Loretta