Rename ShallowCopy to FlatCopy, add polymorph support
- Renamed all "ShallowCopy" serializer presets and references to "FlatCopy" for clarity and consistency.
- Expanded documentation to clarify flat serialization use cases, especially for delta-update and partial-write scenarios.
- Added EnablePolymorphDetectFeature to AcBinarySerializableAttribute and updated all constructor overloads.
- Set UsePolymorphType = true in AcBinarySourceGenerator to enable polymorphic type support by default.
- Updated all [AcBinarySerializable(...)] usages to include new feature flags, explicitly disabling property filter and polymorph detection for affected types.
- Improved comments and documentation for maintainability.
Add compile-time error for object-typed props w/o polymorph
Introduces ACBIN002: a compile-time diagnostic that errors if a [AcBinarySerializable] type declares a System.Object property while UsePolymorphType is false. This prevents silent wire corruption by requiring the developer to either enable polymorphic serialization, use a concrete type, or ignore the property. The check is integrated into the source generator initialization.
- Introduce `MaxDepthBehavior` option (`Throw`, `Truncate`, `Disable`) for explicit depth-limit handling in AcBinarySerializer and SGen.
- Default is now `Throw` (fail-fast); `ShallowCopy` preset uses `Truncate` for shallow-copy semantics.
- Refactor runtime and SGen paths to use unified `TryEnterRecursion`/`ExitRecursion` for correct wire output and inc/dec symmetry.
- Add focused tests to diagnose SGen+Truncate wire-misalignment bug (see `BINARY_ISSUES.md#accore-bin-i-t7k3`).
- Update docs and comments to clarify new behavior and document Toon serializer's current lack of `MaxDepthBehavior` support.
- Adjust tests and usages for new semantics and improved safety.
Introduced MaxDepthBehavior enum and option to control recursion depth handling (Truncate, Throw, Disable) in AcSerializerOptions. Refactored depth-check logic to use a precomputed NeedsDepthCheck flag. Enhanced exception messages for depth-limit violations. Updated tests to assert correct exception behavior for cycles. Improved documentation and added new test/log commands in settings.local.json.
Refactored AcBinary, AcJson, and AcToon serializers to eliminate the explicit depth parameter from all serialization/deserialization methods, generated code, and interfaces. Introduced a global RecursionDepth field on the serialization context, incremented/decremented at recursion entry/exit, and enforced against MaxDepth as a safety net (except when ReferenceHandling=All). Updated all usages, including property, array, and dictionary handling, to use the new context-based depth tracking. Ensured consistency across runtime and generated code.
Refactored WriteByte, WriteVarUInt, and WriteVarULong to use a new GrowOne helper for single-byte buffer growth. This moves the Output.Grow call out of the hot path, improving inlining and serialization performance for frequent single-byte writes. Added detailed comments explaining the rationale and AOT benefits.
The "Test Type" (TypeName) field was removed from all benchmark output locations, including console summaries, formatted output, and markdown result files. No functional changes were made to the test model definitions; only a BOM was added to SharedTestOrderModels.cs. All other benchmark summary details remain unchanged.
Refactored the benchmark and test data infrastructure to use generic, type-safe, and multi-variant models. Introduced generic base classes for the test data hierarchy and factories, with closing-generic aliases for _All_True and _All_False families. Benchmarks now select the correct test data variant per serializer options, and all serializers are generic over the order type. Output and result reporting now include the CLR type name for clarity. Centralized string property handling and improved documentation throughout.
Refactor tests to use _All_True model types throughout
Replaced all usages of legacy test model types (e.g., TestOrder, TestOrderItem, SharedTag, etc.) with new, feature-complete _All_True variants across SignalR test infrastructure, data sources, and service handlers. Updated all generic constraints, method signatures, and test data to use the new types. Added SharedTestBaseModels.cs and SharedTestOrderModels.cs to define abstract bases and concrete _All_True models with full serialization attributes. This enables more thorough and realistic serialization/deserialization testing and future extensibility.
Refactor benchmark suite to use enums for config
Replaced string parameters for layer, opMode, and serializerMode with strongly-typed enums (BenchmarkLayer, BenchmarkOpMode, SerializerSelectionMode) across BenchmarkLoop, Menu, and Program. Updated CLI parsing and menu logic to use Enum.TryParse and return enums. Added XML docs for new enums. Improves type safety, code clarity, and maintainability.
Refactor benchmarks to use typed enums for engine/mode
Replaced string-based identifiers for serializer engine, I/O mode, and dispatch mode with strongly-typed enums (BenchmarkEngine, BenchmarkIoMode, BenchmarkDispatchMode). Added BenchmarkEnums.cs with ToDisplay() helpers for consistent output. Updated all benchmark implementations, DTOs, and output logic to use enums. Removed obsolete string constants from Configuration.cs. Merged allocation measurement methods in BenchmarkLoop.cs for clarity. Improves type safety, maintainability, and output consistency.
Refactor: move benchmark logic to BenchmarkLoop.cs
Moved all benchmark execution logic (RunBenchmark, RunBenchmarksForTestData, CreateSerializers) from Program.cs into a new static class BenchmarkLoop in BenchmarkLoop.cs. Program.cs now delegates benchmark runs to BenchmarkLoop, improving separation of concerns. No changes to benchmark functionality.
Moved all ISerializerBenchmark implementations for AcBinary and MemoryPack from Program.cs into dedicated files under Benchmarks/. Improves code organization and maintainability; no logic changes, only file structure refactor.
Refactor: extract serializer benchmarks to separate files
Moved AcBinary, MemoryPack, MessagePack, and SystemTextJson benchmark classes into dedicated files for clarity. Centralized options formatting and MemoryPack selection logic in a new BenchmarkOptions helper. Updated Program.cs to use these helpers and removed redundant inline implementations, improving code organization without changing benchmark logic.
Extract ISerializerBenchmark to its own file
Moved ISerializerBenchmark from Program.cs to a new ISerializerBenchmark.cs file under the AyCode.Core.Serializers.Console.Benchmarks namespace. Updated all benchmark classes in Program.cs to implement the interface from the new namespace and made them internal. Added the necessary using directive to Program.cs. Adjusted a PowerShell script in settings.local.json to ensure the new using is present. Removed the old interface definition from Program.cs.
Refactor: move benchmark loop logic to BenchmarkLoop.cs
Refactored all benchmark execution infrastructure from Program.cs into a new internal static class BenchmarkLoop. This includes timing, allocation measurement, progress reporting, GC helpers, MemoryPack setup validation, and test data filtering. Updated Program.cs and all serializer benchmarks to use the new class. Added serAllocPct reporting in Output.cs and a PowerShell script for automated refactoring. No functional changes to benchmark logic.
Refactor: split Program.cs into Menu, Output, DTO
Refactored the benchmark console app for modularity:
- Moved all menu logic to Menu.cs (main/settings menus)
- Moved all output/result formatting to Output.cs
- Extracted BenchmarkResult DTO to BenchmarkResult.cs
- Program.cs now only handles orchestration and the benchmark loop
- Moved GetCurrentCharsetName to Configuration.cs
- Removed obsolete Warmup methods from serializers
No functional changes; improves clarity and maintainability.
Refactor: centralize config/state in Configuration.cs
Moved all benchmark configuration, mutable state, and attribute-flag aggregation from Program.cs to a new Configuration.cs static class. Updated all references in Program.cs and related benchmark classes to use Configuration.<value>. Removed the "profiler" CLI mode and its code. Updated README.md to reflect these changes. This improves maintainability and keeps Program.cs focused on orchestration and UX, with no changes to benchmark logic.
Refactor output, allocation, and summary logic in Program
- Switched if/else and range checks to C# switch expressions for clarity.
- Improved console progress display with cleaner line updates.
- Added Thread.Sleep after JIT pre-warmup for stable benchmarking.
- Enhanced allocation measurement for serializer/deserializer setup.
- Made options and summary output conditional and more consistent.
- Standardized string outputs and comparison headers.
- Improved comments, XML docs, and code style for maintainability.
- No changes to core algorithms; all changes are quality-of-life and output improvements.
Phase-isolated Ser/Des warmup & GC in benchmarks
Refactored benchmark loop to perform separate warmup and measurement for serialization and deserialization phases, with forced GC.Collect at each phase boundary for heap and cache isolation. Added ForceGcCollect() and new WarmupSerialize/WarmupDeserialize interface methods (with defaults). Updated output, documentation, and per-phase iteration handling for improved accuracy and clarity. Added detailed comments explaining rationale and effects.
AcBinary: add framing doc, buffer growth fixes, doc updates
- Added `BINARY_WHYUSE.md` for architectural framing and value proposition
- Updated `BINARY_FEATURES.md` and `README.md` to reference and prioritize the new doc
- Documented AsyncPipeWriterOutput chunk-size limitation and workarounds in `BINARY_ASYNCPIPE_ISSUES.md`
- Refactored buffer growth logic in `AcBinarySerializer.BinarySerializationContext.cs` to validate capacity after grow and throw clear exceptions on under-provisioning; removed dead method
- Fixed chunk size alignment bug in `AsyncPipeWriterOutput.cs` to prevent buffer under-provisioning
- Added `AYCODE_NATIVEAOT` build config support in `Program.cs`
- Improved documentation clarity and error diagnostics for streaming/buffered serialization edge cases
Enable per-type property filter opt-out in AcBinary
Adds EnablePropertyFilterFeature to AcBinarySerializableAttribute, allowing types to opt out of property filter codegen and runtime checks. Updates source generator, metadata, and runtime logic to honor this flag. Removes UsePropertyFilter constant; emission is now attribute-driven. Also optimizes string serialization for non-ASCII cases and refactors deserializer byte reads for trusted single-segment fast paths. Backward compatible: property filter remains enabled by default.
Optimize AcBinary: add IsTrustedSingleSegment fast-path
Introduce static abstract IsTrustedSingleSegment to IBinaryInputBase and implement in all input types. Update EnsureAvailable to leverage JIT specialization for array-backed inputs, eliminating per-read overhead. Add detailed XML docs on performance trade-offs. No breaking changes; improves deserialization efficiency and clarity.
FastWire: Add markerless string encoding/decoding
Introduced a markerless FastWire path for string properties and collection elements in AcBinary serialization. Strings are now encoded with a 4-byte int32 sentinel header (-1=null, 0=empty, N>0=content) and UTF-16 bytes, eliminating the type code marker in FastWire mode. Updated code generation, runtime, and documentation to support this, while preserving Compact mode behavior and cross-mode compatibility.
Use ReadOnlySequence<byte> in benchmarks for deserialization
Updated all AcBinary and MemoryPack benchmark deserialization and round-trip verification methods to use ReadOnlySequence<byte> overloads instead of byte[] or ToArray(). This ensures benchmarks exercise the production-realistic deserialization path (e.g., for SignalR/Pipe consumers) and aligns buffer writer semantics across serializers. Added comments to clarify intent. No business logic was changed.
Switch to BCL UTF-8 APIs for string (de)serialization
Replaced custom Utf8Transcoder logic with System.Text.Encoding.UTF8 and System.Text.Unicode.Utf8 for string encoding/decoding in AcBinarySerializer and AcBinaryDeserializer. PropertyMetadataBase now uses Encoding.UTF8.GetBytes for property name encoding. Retained Utf8Transcoder for any remaining SIMD/custom logic. No public API changes; internal refactoring for performance and maintainability.
Benchmark stabilization & charset-param workload support
Major overhaul of the custom benchmark harness:
- Per-serializer warmup, GC isolation, pilot discard, and CPU pinning for stable, reproducible results
- Adaptive per-cell iteration targeting (~250ms/sample) and statistical reporting (min/max/stddev/CV)
- CLI/menu support for single-cell A/B runs
- Test data refactored to ASCII baselines with configurable charset suffix (6 presets), selectable via menu; charset recorded in all outputs
- Markdown/console output now includes per-op µs, inter-sample range, CV warnings, and iteration counts
- Documentation updated with rationale, methodology, and notes on reverted/experimental optimizations
Enables reliable, cross-charset, release-grade performance measurement for AcBinary.
- Add overflow/corruption guards to string (de)serialization (writer/reader now throw on invalid lengths)
- Remove dead string serialization methods per BINARY_TODO.md audit
- Update BINARY_TODO.md with closure/resolution for H2Q6, O7G2, V4N5, and related entries
- Add MaxStringCharLength constant and update marker reservations in BinaryTypeCode
- Simplify string cache ASCII verification in deserializer
- Add SGen/Runtime round-trip compatibility tests for large/deep data
- Minor code modernization and style improvements
SIMD Utf8Transcoder.GetUtf8ByteCount + test suite
Introduced SIMD-accelerated Utf8Transcoder.GetUtf8ByteCount for efficient UTF-8 byte counting, replacing all writer-side Encoding.UTF8.GetByteCount usages. Added 29 unit tests for correctness across ASCII, Hungarian, CJK, emoji, and boundary cases. Updated benchmarks to ensure FixStr is bypassed and wire mode is selectable. Documented implementation and dead-code review in BINARY_TODO.md. No public API changes.
Fix Utf8Transcoder AVX2 bug, add SIMD boundary tests
- Added Hungarian language preference rule to copilot-instructions.md.
- Fixed AVX2 SIMD bug in Utf8Transcoder: corrected upper-half store offset from Vector128<ushort>.Count to Vector256<ushort>.Count, preventing memory overlap on 32+ byte ASCII runs.
- Added Utf8TranscoderTests covering all SIMD/scalar paths, with boundary and round-trip tests for ASCII, Hungarian, CJK, emoji, and mixed content, ensuring correctness and BCL compatibility.
Refactor: extract UTF-8 transcoder to Utf8Transcoder
Moved all UTF-8/UTF-16 encoding, decoding, and char counting logic from AcBinarySerializer/AcBinaryDeserializer into a new internal Utf8Transcoder class. Updated all call sites to use the new class. Removed redundant private methods from the original classes. Updated BINARY_TODO.md to clarify SIMD decode status and rationale for deferring AVX2 multi-byte SIMD path. No functional changes—pure refactor for maintainability and future SIMD work.
SIMD UTF-8 upgrades, i18n test data, MVC disabled
- Switch all test/benchmark data to Hungarian UTF-8 strings for i18n coverage
- Add AVX-512BW, Vector256, and Vector128 SIMD paths for UTF-8/UTF-16 encode/decode (ASCII and multi-byte) in binary serializer/deserializer
- Update WireMode docs for encoding guidance per workload/host
- Block-comment and disable MVC formatters and Microsoft.AspNetCore.App reference due to .NET 10 Hybrid client conflict; update docs to reflect temporary state
- Update appsettings: replace WaitForFlush with FlushPolicy
- Revise BINARY_TODO.md for SIMD transcoder progress and next steps
Add docs for AcBinary MVC formatters and pipeline updates
Comprehensive documentation for new ASP.NET Core MVC formatters supporting AcBinary, including registration, media type, request/response flow, error handling, and future plans. Updated project and topic docs to reference MVC formatters and folder structure. Added performance planning entry for StreamPipeWriter congestion fallback. Expanded markerless schema lane rationale and updated architecture docs to reflect MVC formatter integration. Improved navigation and layering documentation.
Remove PipeReader APIs from AcBinaryDeserializer
Refactored to remove all PipeReader-based async deserialization methods from AcBinaryDeserializer. Updated BINARY_TODO.md to clarify that draining PipeReader to AsyncPipeReaderInput is now a consumer responsibility. Refactored AcBinaryInputFormatter to inline the drain-loop and background deserialization, following new layering guidance. Updated comments and docs to reflect these changes.
AcBinary: Add ASCII string markers, doc optimizations
Enhanced string encoding with FixStrAscii/StringAscii markers for efficient ASCII handling, updated header flag base to 0xB0, and expanded documentation with marker-dispatch logic, performance results, and markerless schema lane plans.
AcBinary: ASCII string opt, Type-based API, MVC support
- Add ASCII-optimized string serialization/deserialization with new FixStrAscii/StringAscii markers for fast byte→char widening.
- Introduce non-generic Type-based Serialize/Deserialize overloads for runtime-typed scenarios (plugin, MVC, model binding).
- Add AcBinaryInputFormatter/OutputFormatter and AddAcBinaryFormatters extensions for ASP.NET Core MVC integration.
- Update project references and close ACCORE-BIN-T-N9G6 in docs.
Add issue/TODO for AcBinary default-value omission risk
Documented ACCORE-BIN-I-D9Y2 in BINARY_ISSUES.md, detailing the risk of silent data corruption when omitting default-valued properties if type defaults diverge between writer and reader. Added ACCORE-BIN-T-W7N5 in BINARY_TODO.md to track mitigation options, including a possible opt-out flag and required documentation/tests. Both entries are cross-referenced; decision on mitigation is deferred.
SIMD-accelerated UTF-8 encode/decode for AcBinary
- Added Vector256-based SIMD path for UTF-8 char counting in deserializer, replacing scalar loop for faster ASCII/multibyte handling.
- Introduced EncodeUtf8SinglePass in serializer: layered SIMD/DWORD/scalar UTF-16→UTF-8 encoding, bypassing Encoding.UTF8.GetBytes.
- Updated serializer to use new encoder for string writes.
- Expanded "fastestbyte" benchmark mode to compare both AcBinary (UTF-8/UTF-16) and MemoryPack strategies.
- Improved comments and docs to clarify new SIMD logic.
Disable ASCII fast paths; add FastestByte mode, plan tasks
Temporarily disable ASCII string fast paths in AcBinarySerializer and AcBinaryDeserializer to isolate and benchmark the custom UTF-8 encoder/decoder. Add "FastestByte" benchmark mode for focused AcBinary vs MemoryPack Byte[] comparison. Update BINARY_TODO.md with new technical tasks for .NET 11 SIMD decoder, sentinel-length encoding, ASCII marker-dispatch, and a custom UTF-8 encoder. These changes support staged optimization and future performance improvements.
Optimize string serialization: ASCII & UTF-8 fast paths
- Refactored AcBinarySerializer/Deserializer to use single-pass UTF-8 encoding and a custom allocation-free UTF-8 decoder, improving performance for both ASCII and non-ASCII strings.
- Expanded BinaryTypeCode with new ASCII string markers (FixStrAscii, StringAscii) and updated helpers for robust, branch-friendly string dispatch.
- Updated settings.local.json with new diagnostic and plugin management commands.
Optimize FastWire string (de)serialization and benchmarks
- Increased release benchmark iterations for more robust testing.
- Improved FastWire string deserialization with zero-copy UTF-16.
- Set FastWire and string caching options during context init/reset.
- Optimized FastWire string serialization for direct UTF-16 copy.
- Enhanced non-ASCII string fallback to use Utf8NoBom encoding.
- Refactored WriteFixStr for efficient ASCII and fallback handling.
Support FixStr for short non-ASCII strings (UTF-8)
Refactored WriteFixStrDirect to encode short non-ASCII strings as FixStr if their UTF-8 byte count is ≤31, not just ASCII. This improves efficiency for short international strings. Also ensured correct buffer sizing and direct UTF-8 encoding. No functional change to project file; property repositioned for clarity.
NativeAOT: full DAMs propagation, trimmer-safe serializers
- Propagate [DynamicallyAccessedMembers] from all public Serialize<T>/Deserialize<T> APIs through all type/property metadata and factories, centralizing requirements in TypeMetadataBase.RequiredMembers.
- Add [UnconditionalSuppressMessage] for known trimmer blind spots (polymorphism, inheritance, nested types) with detailed justifications.
- Update all internal delegate/factory signatures to preserve DAMs context.
- Annotate public APIs for AOT safety; document consumer requirements for SGen or rooted model assemblies.
- Update BINARY_FEATURES.md with NativeAOT/trimmer compatibility, guidance, and limitations.
- Adjust benchmark project for AOT/JIT parity and add i18n test data.
- No breaking API changes; SGen and Runtime paths remain, now fully AOT-compatible.
NativeAOT: fallback for delegates, exclude MessagePack
Added AYCODE_NATIVEAOT symbol for AOT builds and excluded MessagePack benchmarks from NativeAOT due to lack of AOT support. Updated AcSerializerCommon to use reflection-based delegates when dynamic code is unavailable, ensuring compatibility with both JIT and AOT. Added explanatory comments throughout.
Remove SegmentBufferReader; unify on AsyncPipeReaderInput
Migrates all SignalR chunked streaming receive logic to AsyncPipeReaderInput, fully removing SegmentBufferReader and SegmentBufferReaderInput from the codebase. Updates all references, deserialization paths, and documentation to reflect the new unified primitive. Marks ADR-0003 as accepted (partially executed), closes related TODOs, and clarifies protocol docs. Sets DoubleBuffered as the default FlushPolicy. No wire format or behavioral changes; all tests pass.
Switch to FlushPolicy enum for streaming flush control
Replaces the legacy bool waitForFlush with a new FlushPolicy enum (PerChunk, DoubleBuffered, Coalesced) across all binary streaming serialization APIs and SignalR protocol options. Updates all code, configuration, and documentation to use the new policy, clarifies memory/throughput trade-offs, and closes related TODOs. Stream-backed writers remain sequential; only parallel-capable Pipe-based writers honor the policy.
Move DrainFromAsync to tests, add in-memory benchmarks
- Moved AsyncPipeReaderInputExtensions.DrainFromAsync from the main framework to test-only assembly; no longer public API.
- Removed AcBinaryDeserializer.DeserializeFromPipeReaderAsync<T> from public API; tests now inline drain+deserialize logic.
- Added AcBinaryInMemoryPipeBenchmark and AcBinaryInMemoryRawByteArrayBenchmark to complete 2x2 transport × wire-format benchmark matrix.
- Refactored benchmark runner for interactive menu, settings, and CLI parsing.
- Expanded XML docs for AsyncPipeReaderInput and AsyncPipeWriterOutput to clarify push-pattern and real-world usage.
- Updated BINARY_ASYNCPIPE_TODO.md and related docs to reflect these changes.
Refactor pipe benchmarks to 2-task streaming model
Refactored AcBinaryNamedPipeBenchmark and AcBinaryNamedPipeRawByteArrayBenchmark to use a two-task (producer/consumer) streaming pipeline for deserialization, enabling true Ser↔Des overlap. Reduced BufferWriterChunkSize from 16K to 4K. Updated synchronization, cleanup, and comments to reflect the new architecture and improve performance comparison between chunked and raw byte[] modes.
Loretta