- Add overflow/corruption guards to string (de)serialization (writer/reader now throw on invalid lengths)
- Remove dead string serialization methods per BINARY_TODO.md audit
- Update BINARY_TODO.md with closure/resolution for H2Q6, O7G2, V4N5, and related entries
- Add MaxStringCharLength constant and update marker reservations in BinaryTypeCode
- Simplify string cache ASCII verification in deserializer
- Add SGen/Runtime round-trip compatibility tests for large/deep data
- Minor code modernization and style improvements
Remove PipeReader APIs from AcBinaryDeserializer
Refactored to remove all PipeReader-based async deserialization methods from AcBinaryDeserializer. Updated BINARY_TODO.md to clarify that draining PipeReader to AsyncPipeReaderInput is now a consumer responsibility. Refactored AcBinaryInputFormatter to inline the drain-loop and background deserialization, following new layering guidance. Updated comments and docs to reflect these changes.
AcBinary: ASCII string opt, Type-based API, MVC support
- Add ASCII-optimized string serialization/deserialization with new FixStrAscii/StringAscii markers for fast byte→char widening.
- Introduce non-generic Type-based Serialize/Deserialize overloads for runtime-typed scenarios (plugin, MVC, model binding).
- Add AcBinaryInputFormatter/OutputFormatter and AddAcBinaryFormatters extensions for ASP.NET Core MVC integration.
- Update project references and close ACCORE-BIN-T-N9G6 in docs.
NativeAOT: full DAMs propagation, trimmer-safe serializers
- Propagate [DynamicallyAccessedMembers] from all public Serialize<T>/Deserialize<T> APIs through all type/property metadata and factories, centralizing requirements in TypeMetadataBase.RequiredMembers.
- Add [UnconditionalSuppressMessage] for known trimmer blind spots (polymorphism, inheritance, nested types) with detailed justifications.
- Update all internal delegate/factory signatures to preserve DAMs context.
- Annotate public APIs for AOT safety; document consumer requirements for SGen or rooted model assemblies.
- Update BINARY_FEATURES.md with NativeAOT/trimmer compatibility, guidance, and limitations.
- Adjust benchmark project for AOT/JIT parity and add i18n test data.
- No breaking API changes; SGen and Runtime paths remain, now fully AOT-compatible.
Remove SegmentBufferReader; unify on AsyncPipeReaderInput
Migrates all SignalR chunked streaming receive logic to AsyncPipeReaderInput, fully removing SegmentBufferReader and SegmentBufferReaderInput from the codebase. Updates all references, deserialization paths, and documentation to reflect the new unified primitive. Marks ADR-0003 as accepted (partially executed), closes related TODOs, and clarifies protocol docs. Sets DoubleBuffered as the default FlushPolicy. No wire format or behavioral changes; all tests pass.
Move DrainFromAsync to tests, add in-memory benchmarks
- Moved AsyncPipeReaderInputExtensions.DrainFromAsync from the main framework to test-only assembly; no longer public API.
- Removed AcBinaryDeserializer.DeserializeFromPipeReaderAsync<T> from public API; tests now inline drain+deserialize logic.
- Added AcBinaryInMemoryPipeBenchmark and AcBinaryInMemoryRawByteArrayBenchmark to complete 2x2 transport × wire-format benchmark matrix.
- Refactored benchmark runner for interactive menu, settings, and CLI parsing.
- Expanded XML docs for AsyncPipeReaderInput and AsyncPipeWriterOutput to clarify push-pattern and real-world usage.
- Updated BINARY_ASYNCPIPE_TODO.md and related docs to reflect these changes.
Refactor AcBinary NamedPipe to long-lived multi-message mode
Refactored the AcBinary NamedPipe benchmark to use a single long-lived AsyncPipeReaderInput in multi-message mode, with one background drain task and synchronous deserialization per message. Buffer recycling is now signaled by the consumer via a new MessageDone() method, called in the deserializer's finally block, preventing producer-consumer races. Added IsCompleted property to AsyncPipeReaderInput. Increased release-mode benchmark iteration counts. Updated documentation and comments to reflect the new architecture and rationale.
Refactor AcBinary streaming: multi-message protocol
- Renamed framing flags to multiMessage for clarity in AsyncPipeReaderInput/AsyncPipeWriterOutput.
- Multi-message mode ([202]=end-of-message) now auto-resets input for reuse; session end is explicit.
- Updated framing state machine, buffer cycling, and sentinel logic.
- Revised all serializer/deserializer entry points and tests for new protocol.
- Expanded docs and XML comments to detail wire format and protocol constraints.
- Updated benchmarks and tests for new streaming API and multi-message behavior.
- Documented protocol limits and added security issue/TODO for type-name deserialization in SignalR binary protocol.
Separate raw and framed streaming in AcBinarySerializer
Refactored AcBinarySerializer and AsyncPipeWriterOutput to support both raw (headerless) and multiplexed/framed ([201][UINT16][data]) streaming wire formats, controlled by a new flag and explicit APIs. Updated AsyncPipeReaderInput and AcBinaryDeserializer to match, with new constructor options and documentation. Expanded tests for both modes and added runtime type detection for flush strategy safety. Minor refactoring and doc improvements throughout.
Remove NamedPipe helpers; make binary serializer transport-agnostic
Removed NamedPipe-specific serializer/deserializer helpers from the framework. Refactored tests to manage NamedPipe lifecycle directly using generic PipeWriter/PipeReader APIs. Introduced a transport-agnostic async deserialization method for any PipeReader. Updated XML docs and method signatures to clarify usage and flush strategies. All NamedPipe logic is now test-only; the core binary serializer is fully transport-agnostic. Minor test cleanups included.
Refactor: replace PipeReaderBinaryInput with SegmentBufferReader
- Remove PipeReaderBinaryInput and all related code.
- Add SegmentBufferReader and SegmentBufferReaderInput for efficient, thread-safe chunked streaming deserialization.
- Update AcBinaryDeserializer and AcBinaryHubProtocol to use the new buffer for async segment protocol, improving state management and background deserialization.
- Enhance chunked protocol handling: skip re-presented chunk start frames, track consumed chunk frame bytes, and improve logging.
- Update test infrastructure to support async segment protocol and add AsyncSegmentPipeTransportWriter for realistic testing.
- Update settings.local.json to reflect new build/test commands and remove obsolete files.
- Improves performance, reliability, and testability of chunked SignalR streaming.
Implements segment-level streaming for SignalR binary protocol via new AsyncPipeWriterOutput and PipeReaderBinaryInput types, enabling chunked serialization/deserialization directly over PipeWriter/PipeReader. Adds BinaryProtocolMode enum to select between standard and streaming modes. Updates protocol classes and documentation. Lays groundwork for future async streaming support.
Enhances debugging of custom binary serialization/deserialization in SignalR by introducing DiagnosticLogger hooks in both AcBinaryDeserializer and AcBinaryHubProtocol. Adds DEBUG-only verification methods to compare array-based and multi-segment deserialization, as well as IBufferWriter and byte[] serialization outputs, logging mismatches for easier bug isolation. Diagnostic loggers are automatically integrated with the hub and client loggers. Also includes extra debug output and a commented workaround for a known serialization issue. Diagnostics are opt-in and only active in DEBUG builds.
- Rewrote SequenceBinaryInput for lazy TryGet iteration (no segment array allocation), zero-copy access to segment backing arrays, and efficient cross-boundary reads using a reusable ArrayPool scratch buffer.
- Added Release() to IBinaryInputBase; now always called after deserialization to return scratch buffer.
- BufferWriterChunkSize is now mutable; set to 4096 for SignalR protocol for better pipe alignment.
- Added and updated documentation: detailed input buffer lifecycle, cross-boundary handling, and new BINARY_ISSUES.md and SIGNALR_ISSUES.md for known limitations and planned optimizations.
- No breaking API changes; improves performance, memory usage, and diagnostics for multi-segment binary deserialization.
Major overhaul for SignalR response pipeline:
- All deserialization now uses byte[] (offset/length) for zero-copy, allocation-free operation; all span/memory overloads removed.
- SignalR protocol sends (signalParams, object) directly; SignalData envelope and related logic removed.
- Server sets SignalParams.SignalDataType so protocol deserializes to the correct runtime type on the client.
- SignalResponseDataMessage now only used for client request/response tracking and stream path; RawResponseData holds the actual object.
- All extension methods, helpers, and infrastructure updated to use new byte[]-based APIs.
- AcSignalRDataSource and all test/benchmark code updated for new object flow.
- Removes all diagnostics, logging, and error handling related to binary envelopes.
- Enables true zero-copy, type-safe, allocation-free SignalR response handling.
Major refactor of binary serialization codegen and runtime:
- Added property writer bridge methods for markerless/metadata paths
- Centralized object marker logic via new bridge methods
- Simplified SGen output: single bridge call replaces branching
- FixObj slot markers now supported in serialization/deserialization
- Refactored collection/dictionary element serialization
- Removed redundant WritePropertyMarkerless method
- Improved tests: use BinaryTypeCode constants, FixObj parsing
- Added InternalsVisibleTo for test project access
- Annotated TestSimpleClass for SGen support
Reduces generated code size, improves maintainability, and ensures correct handling of new binary format features.
Refactored BinaryTypeCode to reserve 0..63 for FixObj slot indices, enabling direct array access for object wrappers. Introduced a new polymorphic type prefix system for properties whose runtime type differs from their declared type, with first/repeated occurrence markers and combined ref-tracking support. Unified wrapper slot caching for SGen and runtime types, improving performance and eliminating dictionary lookups in hot paths. Updated code generation, tests, and constants to use the new slot system. Added new settings and utility scripts. Overall, serialization is now faster, more robust, and extensible.
Enable serialization of runtime type info for System.Object properties using new ObjectWithTypeName markers. Serializer now writes the runtime type name inline; deserializer resolves and instantiates the correct type. Added IsObjectDeclaredType property for detection, refactored WriteObject methods, and registered new deserialization logic. This ensures robust polymorphic (de)serialization even without metadata. Also includes minor cleanup of unused usings.
Introduce new test models for circular refs, update tests to stress reference handling, and enhance deserializer to support ObjectRefFirst/WithMetadataRefFirst type codes. Fix intern cache index assignment, track generated readers in TypeMetadataWrapper, and disable UseGeneratedCode by default. Update benchmarks for reliability and diagnostics. These changes strengthen reference resolution, circular ref support, and performance.
Introduced source-generated binary readers for all serializable types, alongside writers. Added IGeneratedBinaryReader interface and registry for fast lookup in deserializer. Generated readers eliminate runtime overhead by directly reading properties, bypassing wrapper and delegate calls. Updated test models to disable metadata for markerless mode. Integrated fast path in deserializer for improved performance.
Extend serializer/deserializer to efficiently handle arrays and lists of short, ushort, uint, ulong, sbyte, char, DateTimeOffset, and TimeSpan. Improve enum collection handling and support ICollection-based collections. Update source generator to distinguish between indexable and counted collections, generating optimal serialization code for each. These changes enhance performance and coverage for a broader range of collection types.
Refactored AcBinaryDeserializer to read the type code marker byte only once per property, eliminating redundant PeekByte/ReadByte calls and improving efficiency. Updated all property population branches to use the already-consumed type code. Adjusted handling of nested complex objects to rewind the marker byte when needed. Modified TryReadAndSetTypedValue to assume the marker is already consumed, removing unnecessary reads. Exception messages now report the actual type code read.
Added UseGeneratedCode option (default true) to AcBinarySerializerOptions and exposed it in the serialization context. The generated code fast path is now gated by this option, allowing users to enable or disable source-generated serialization. These changes improve deserialization performance, code clarity, and configurability.
Pre-caches TypeMetadataWrapper instances for complex properties,
eliminating repeated GetWrapper dictionary lookups in serialization
and deserialization. Adds ComplexPropertyIndex and ComplexPropertyCount
fields, and PropertyTypeWrappers array to TypeMetadataWrapper. Refactors
scan, write, and populate passes to use cached wrappers, improving
performance for deep and polymorphic object graphs. Updates benchmarks
to focus on FastMode variants. No breaking changes; internal efficiency
improved.
- Refactor collection scan to pre-cache element wrappers and optimize ScanItem for polymorphic types
- Add DisableStringCaching to deserializer; call it on first interned string marker
- Update benchmarks to restore default and no-ref variants, clarify string interning options
- Ensure property scanning respects property filters, skipping filtered properties
Refactored Populate<T>(byte[], T) to add null checks, early returns, and proper context management. Added PopulateMerge<T>(byte[], T) for merge semantics with IId collections, including merge mode and orphaned item removal.
Refactored deserialization to use IBinaryInputBase abstraction, supporting both ArrayBinaryInput (byte[]) and SequenceBinaryInput (ReadOnlySequence<byte>). All context and methods are now generic over TInput, enabling zero-copy for arrays and true streaming for multi-segment sources. Internal logic (ReadValue, Populate, Skip, etc.) is specialized per input type, improving performance and flexibility. This enables future extensibility for other input sources and optimizes handling of large or segmented payloads.
Eliminate BinaryDeserializationContextClass and merge all buffer, pooling, and cache logic into a single sealed BinaryDeserializationContext. All pooling and state management is now handled directly by the context, reducing allocations and indirection. All deserialization entry points and internal logic are updated to use the new context class and pool. String interning, metadata, and array pooling are now managed within the unified context, improving performance and maintainability.
- Moved test data creation to BenchmarkTestDataProvider.cs and removed from Program.cs for better organization.
- Added [MessagePackObject]/[Key] attributes to all test models for explicit MessagePack serialization.
- Updated MessagePack benchmark to use MessagePackSerializerOptions.Standard.
- Improved AcBinaryDeserializer string cache with ASCII byte match to prevent hash collision bugs.
- Optimized AcBinarySerializer/Deserializer for string property handling and non-primitive writes.
- Set AcBinarySerializerOptions.UseMetadata default to true for safer deserialization.
Refactored binary deserialization to use a pooled BinaryDeserializationContextClass, eliminating per-call heap allocations and enabling cache reuse for string and intern caches. Introduced DeserializationContextClassPool for efficient context management. Updated all deserialization entry points to use the pool with proper disposal. Added efficient ReadOnlySequence<byte> support. Changed AcBinarySerializerOptions.UseMetadata default to false. These changes reduce GC pressure and improve performance, especially for high-throughput and WASM scenarios.
Refactor binary serializer to use a true two-pass process for string interning and object reference tracking. Adds a scan pass to identify duplicates and assigns cache indices deterministically in first-occurrence order. Updates wire format to write explicit cache indices after *First markers. Refactors InternEntry, removes marker rewriting, and updates deserializer to match new format. Improves performance, correctness, and robustness for complex object graphs with shared references and repeated strings.
Move CacheMap building to dedicated method for efficiency.
Remove incremental cache logic and SourceHashes field.
Simplify property population and update documentation.
Improves performance and code clarity.
Refactored deserialization logic to use inline metadata for UseMetadata mode, replacing footer-based property hash lookup. Context now tracks inline metadata entries and builds cache maps incrementally per source type. Unified property population for both metadata modes. Updated TypeMetadataWrapper to manage cache map and source hashes per context. Improved robustness for runtime source type changes and streamlined object skipping logic. Updated method signatures to use BinaryPropertySetterBase abstraction.
Introduce UseMetadata mode with FNV-1a property name hashing.
Write per-type property hashes to metadata footer for robust
property matching during deserialization. Remove legacy property
name table logic. Add ObjectWithMetadata marker and cachemap
logic for nested objects. Enable duplicate hash detection and
make UseMetadata default. Improves schema evolution support.
Update list factory cache and GetOrCreateListFactory to accept a capacity parameter, enabling preallocation of lists during deserialization. Adjust deserializer code to pass collection size where available, improving performance and memory usage. Also, pre-size dictionaries on creation and add Bash(find:*) to allowed commands in settings.local.json.
Major refactor: merges string interning and object reference tracking (IId/Non-IId) into a unified position-based cache and footer in binary serialization. Wire format now uses cache indices for all references; hashcode/Id prefixes removed. Serialization and deserialization logic simplified, improving performance and maintainability. Legacy code paths and redundant dictionary lookups eliminated.
Move IdentityMap and IdAccessorType to dedicated file and update all usages. Introduce pooling and cache-friendly optimizations for small int keys and hash tables, minimizing allocations and speeding up resets. Update buffer sizes and profiling loops. Add extensive comments and preserve old implementation for reference. This prepares the codebase for more efficient serialization reference tracking.
Refactored AcBinaryDeserializer to use a cached _nextDupPosition
for ultra-fast string interning (single int comparison). Updated
initialization/reset logic and streamlined RegisterInternedString
to avoid unnecessary array access and branching. Commented out
MinStringInternLength threshold checks to always register interned
strings. Simplified GetInternedString and made a minor update to
the serializer's analysis report wording. These changes improve
performance and reliability of string interning during
deserialization.
Implement a new position-based string interning mechanism in AcBinarySerializer/AcBinaryDeserializer. This approach tracks stream positions for interned strings, ensuring 100% reliable cache matching during deserialization, even when strings are skipped or reordered. The serializer now writes (position, cacheIndex) pairs in the footer for all repeated strings, and the deserializer uses this mapping for robust cache population. Removes the old buffer-based interned string logic, updates all relevant code paths, and simplifies interned string handling for greater correctness and maintainability. Also updates benchmarks and test data construction to use the new interning mode.
- Test data now controls IId shared ref % for realistic deduplication benchmarks; display names include IId ref ratio.
- Added deep-level clearing of IId refs for realistic object graphs.
- Pallet, Measurement, and Point models now support shared IId refs.
- TestDataFactory passes shared refs to all hierarchy levels.
- Refactored TypeMetadataWrapper for type-specific Id getters, identity maps, and registration—removes hot path type checks/switches.
- AcBinary deserializer now uses new typed methods for reference tracking and registration.
- SerializationContextBase uses pre-cast Id getters for zero-overhead tracking.
- Reduced quick benchmark warmup iterations for faster startup.
- Improves performance, clarity, and maintainability of reference handling and benchmarks.
- Unify and clarify object reference tracking for IId and non-IId types
- Always write/read Id as a normal property with type marker for IId types
- For non-IId types (All mode), use hashcode prefix for reference tracking
- Remove special Id prefix logic; all properties use type markers
- Centralize hashcode registration logic in deserializer
- Improve error handling for missing references
- Refactor tests to cover all ReferenceHandlingMode values and verify both data integrity and reference identity
- Add hex dump utility for debugging serialized bytes
- Make TypeMetadataBase.SourceType public for better diagnostics
- Make ReferenceHandlingMode type-aware; OnlyId fully supported for binary, All is default for JSON
- ReferenceHandling is now settable; add ThrowOnCircularReference option
- Always sort Id property first for IId types to optimize tracking
- Serialize/deserialize IId.Id without type marker when reference handling is enabled
- Contexts now delegate options-derived properties to Options
- Improve skip logic and property filter performance in binary serializer
- Update tests to explicitly set ReferenceHandlingMode.All
- Refactor internal APIs for clarity, safety, and efficiency
Move strongly-typed getter/setter logic and PropertyAccessorType enum into PropertyAccessorBase and PropertySetterBase, eliminating duplication in binary accessor classes. Expose direct typed getter/setter methods and new SetValueTyped/SetToDefault helpers. Rename ObjectGetter to DynamicGetter and update all serializers/deserializers to use GetDynamicValue. Centralize default value logic and improve performance by reducing boxing/unboxing. This unifies and streamlines property accessor infrastructure across all serializers.
Enhances both binary deserialization and JSON serialization to support IId-based reference tracking and handling. Refactors object reference reading in AcBinaryDeserializer to use a unified TryGetValue approach. Adds IId-aware tracking, ID writing, and reference lookup methods to AcJsonSerializer and updates ScanReferences and WriteObject to use them. Improves support for custom identity types and robust reference management.
Replaces flat object reference dictionary with per-type identity maps in deserializer, improving type safety and efficiency for IId types. TypeMetadataWrapper now uses cached typed delegates for reference ID access. Centralizes complex type detection and exposes IsComplexType and TypedIdGetter in metadata. Updates all registration and lookup logic to use wrappers, removes obsolete metadata cache, and ensures thread-safe, type-aware reference handling throughout. Comments out legacy code for easier review and rollback.
Unifies IId-based reference handling for binary serialization and deserialization. Introduces BinaryDeserializationContextClass for heap-based IId tracking, refactors IdentityMap<TId> for unified object storage, and removes legacy IIdReferenceTracker logic. Updates deserializer to use the new infrastructure for all IId types (int, long, Guid) and correct wire formats. Enhances tests for reference identity and object graph integrity. Improves code clarity and maintainability.
Refactored serialization reference tracking to a single-pass, inline approach, removing the previous two-pass scan. Only int, long, and Guid are now supported as IId<T> types; exotic ID types are no longer allowed. Cleaned up related enum values and code paths, defaulting non-IId types to int-based reference IDs. This simplifies the codebase and improves performance and type safety.
Major refactor of serialization infrastructure:
- Removed AcSerializeBase; replaced with AcSerializerContextBase<TMetadata> for unified context management.
- Added TypeMetadataWrapper<TMetadata> to combine metadata and per-context tracking state.
- All serializer contexts now inherit from AcSerializerContextBase and use context.GetWrapper(type) for metadata and tracking.
- Reference tracking for IId types is now type-safe and efficient (bitmaps for small int IDs, generic identity maps for others).
- Removed generic ThreadLocal caching from TypeMetadataBase; caching now uses global ConcurrentDictionary.
- Updated all type metadata classes to inherit from non-generic base.
- Added IdPropertyInfo and MetadataType to TypeMetadataBase.
- Added stub context base classes for JSON and Toon.
This centralizes and optimizes metadata/tracking, improves performance, and prepares for future extensibility.
Loretta