- Replace single String marker with StringLen8/16/32 (167–169) for explicit string length encoding (1/2/4 bytes)
- Update serialization/deserialization logic and marker checks for new scheme
- Rename charset suffixes: Latin1FixAscii → AsciiFix, add Latin1Fix (5-char Hungarian)
- Update menu, argument parsing, and references for new charset/test names and order
- Swap MaxDepthBehavior enum values: Throw=0, Truncate=1
- Update comments and docs for new marker and charset conventions
- Refactored string serialization for performance: ASCII-optimistic encode, single pass, and minimal shifting; extracted string interning logic to TryWriteInternedString for both runtime and SGen paths.
- Updated AcBinarySerializer buffer writes to use BufferAt helper, removing redundant bounds checks.
- Enhanced CLI argument parsing to support multiple args and charset selection; unknown args now emit warnings.
- Switched all test data generation from Hungarian to English.
- Benchmark report now includes .NET runtime version.
- Cached MinStringInternLength in AcBinaryDeserializer for performance.
- Minor BinaryTypeCode flag refactor and doc improvements.
- Added BINARY_ISSUES.md entry for FastWire string interning/ref handling desync bug.
Introduced a new "SGenOnly" build configuration across the solution, updating Directory.Build.props, AyCode.Core.targets, and the .sln file for full support. Centralized TargetFramework and build properties in AyCode.Core.targets, removing redundancy from project files. Updated code to recognize SGEN_ONLY at compile time. Added new Bash commands for file conversion and cleanup. No functional code changes outside build and configuration logic.
- Benchmark charset profiles are now length-consistent: all *Short = 40 chars, all *Long = 280 chars, across ASCII, Latin1, CJK BMP, Cyrillic, and Mixed.
- `CharsetSuffixes` was rewritten with new profiles and base-string repetition for compile-time constants.
- Menu/configuration updated for new profiles, selection logic, and improved descriptions.
- Docs updated to reflect new profiles, lengths, and serialization tier impacts.
- `StringSmall` deserialization split into `ReadStringSmallCompact` and `ReadStringSmallFastWire`; all call sites now dispatch by mode, clarifying the hot path.
- SGen codegen and runtime dispatch tables updated for the new decode split.
- Binary marker docs clarified: only Intern/Metadata/Polymorph features are wire-symmetric for reader case omission; RefHandling is not.
- Added `BINARY_STRICT_SGEN.md` planning doc for a SGen-only, attribute-required, AOT-friendly NuGet package.
- Introduce BDN-based runner (AcBinaryVsMemPackBenchmark) mirroring Console's FastestByte scenario; add BdnSummaryAdapter for unified result translation.
- Standardize output: both runners emit .log/.LLM/.output triplets; BDN-native artifacts go under Benchmark/BDN/.
- Simplify CLI: replace granular switches with --serializers; update help and usage.
- Remove legacy benchmark classes; focus on scenario-based approach.
- Rewrite README.md for both AyCode.Benchmark and Console to document dual-runner architecture, output conventions, and dependencies.
- Rotate BINARY_TODO.md; archive closed entries to BINARY_TODO_2026_04.md and BINARY_TODO_2026_05.md.
- Add BINARY_SGEN_OPTIMIZATION.md for SGen per-property emit optimization notes.
- Update comments and docstrings for clarity and maintainability; clarify BenchmarkResult iteration semantics for BDN rows.
- Removed Output.cs and migrated all reporting, formatting, and statistics logic to a new BenchmarkReportWriter static class.
- Extended ReportingContext with run-header fields for richer output metadata.
- Updated BenchmarkLoop to use BenchmarkReportWriter and the new ReportingContext.
- Centralized all output file generation (.log, .LLM, .output) and formatting helpers in BenchmarkReportWriter.
- Improved separation of concerns and unified output artifact naming and metadata.
Major refactor: all benchmark engine implementations, enums, and helpers moved from Console project to AyCode.Core.Benchmarks.Workloads.Scenarios for unified cross-runner use. BenchmarkResult and reporting logic moved to AyCode.Core.Benchmarks.Reporting. Attribute-flag aggregation centralized in BenchmarkOptions. Updated all usages, project references, and SGen codegen for ref-handling. Prepares codebase for shared reporting and future extensibility.
The "Test Type" (TypeName) field was removed from all benchmark output locations, including console summaries, formatted output, and markdown result files. No functional changes were made to the test model definitions; only a BOM was added to SharedTestOrderModels.cs. All other benchmark summary details remain unchanged.
Refactored the benchmark and test data infrastructure to use generic, type-safe, and multi-variant models. Introduced generic base classes for the test data hierarchy and factories, with closing-generic aliases for _All_True and _All_False families. Benchmarks now select the correct test data variant per serializer options, and all serializers are generic over the order type. Output and result reporting now include the CLR type name for clarity. Centralized string property handling and improved documentation throughout.
Refactor tests to use _All_True model types throughout
Replaced all usages of legacy test model types (e.g., TestOrder, TestOrderItem, SharedTag, etc.) with new, feature-complete _All_True variants across SignalR test infrastructure, data sources, and service handlers. Updated all generic constraints, method signatures, and test data to use the new types. Added SharedTestBaseModels.cs and SharedTestOrderModels.cs to define abstract bases and concrete _All_True models with full serialization attributes. This enables more thorough and realistic serialization/deserialization testing and future extensibility.
Refactor benchmark suite to use enums for config
Replaced string parameters for layer, opMode, and serializerMode with strongly-typed enums (BenchmarkLayer, BenchmarkOpMode, SerializerSelectionMode) across BenchmarkLoop, Menu, and Program. Updated CLI parsing and menu logic to use Enum.TryParse and return enums. Added XML docs for new enums. Improves type safety, code clarity, and maintainability.
Refactor benchmarks to use typed enums for engine/mode
Replaced string-based identifiers for serializer engine, I/O mode, and dispatch mode with strongly-typed enums (BenchmarkEngine, BenchmarkIoMode, BenchmarkDispatchMode). Added BenchmarkEnums.cs with ToDisplay() helpers for consistent output. Updated all benchmark implementations, DTOs, and output logic to use enums. Removed obsolete string constants from Configuration.cs. Merged allocation measurement methods in BenchmarkLoop.cs for clarity. Improves type safety, maintainability, and output consistency.
Refactor: move benchmark logic to BenchmarkLoop.cs
Moved all benchmark execution logic (RunBenchmark, RunBenchmarksForTestData, CreateSerializers) from Program.cs into a new static class BenchmarkLoop in BenchmarkLoop.cs. Program.cs now delegates benchmark runs to BenchmarkLoop, improving separation of concerns. No changes to benchmark functionality.
Moved all ISerializerBenchmark implementations for AcBinary and MemoryPack from Program.cs into dedicated files under Benchmarks/. Improves code organization and maintainability; no logic changes, only file structure refactor.
Refactor: extract serializer benchmarks to separate files
Moved AcBinary, MemoryPack, MessagePack, and SystemTextJson benchmark classes into dedicated files for clarity. Centralized options formatting and MemoryPack selection logic in a new BenchmarkOptions helper. Updated Program.cs to use these helpers and removed redundant inline implementations, improving code organization without changing benchmark logic.
Extract ISerializerBenchmark to its own file
Moved ISerializerBenchmark from Program.cs to a new ISerializerBenchmark.cs file under the AyCode.Core.Serializers.Console.Benchmarks namespace. Updated all benchmark classes in Program.cs to implement the interface from the new namespace and made them internal. Added the necessary using directive to Program.cs. Adjusted a PowerShell script in settings.local.json to ensure the new using is present. Removed the old interface definition from Program.cs.
Refactor: move benchmark loop logic to BenchmarkLoop.cs
Refactored all benchmark execution infrastructure from Program.cs into a new internal static class BenchmarkLoop. This includes timing, allocation measurement, progress reporting, GC helpers, MemoryPack setup validation, and test data filtering. Updated Program.cs and all serializer benchmarks to use the new class. Added serAllocPct reporting in Output.cs and a PowerShell script for automated refactoring. No functional changes to benchmark logic.
Refactor: split Program.cs into Menu, Output, DTO
Refactored the benchmark console app for modularity:
- Moved all menu logic to Menu.cs (main/settings menus)
- Moved all output/result formatting to Output.cs
- Extracted BenchmarkResult DTO to BenchmarkResult.cs
- Program.cs now only handles orchestration and the benchmark loop
- Moved GetCurrentCharsetName to Configuration.cs
- Removed obsolete Warmup methods from serializers
No functional changes; improves clarity and maintainability.
Refactor: centralize config/state in Configuration.cs
Moved all benchmark configuration, mutable state, and attribute-flag aggregation from Program.cs to a new Configuration.cs static class. Updated all references in Program.cs and related benchmark classes to use Configuration.<value>. Removed the "profiler" CLI mode and its code. Updated README.md to reflect these changes. This improves maintainability and keeps Program.cs focused on orchestration and UX, with no changes to benchmark logic.
Refactor output, allocation, and summary logic in Program
- Switched if/else and range checks to C# switch expressions for clarity.
- Improved console progress display with cleaner line updates.
- Added Thread.Sleep after JIT pre-warmup for stable benchmarking.
- Enhanced allocation measurement for serializer/deserializer setup.
- Made options and summary output conditional and more consistent.
- Standardized string outputs and comparison headers.
- Improved comments, XML docs, and code style for maintainability.
- No changes to core algorithms; all changes are quality-of-life and output improvements.
Phase-isolated Ser/Des warmup & GC in benchmarks
Refactored benchmark loop to perform separate warmup and measurement for serialization and deserialization phases, with forced GC.Collect at each phase boundary for heap and cache isolation. Added ForceGcCollect() and new WarmupSerialize/WarmupDeserialize interface methods (with defaults). Updated output, documentation, and per-phase iteration handling for improved accuracy and clarity. Added detailed comments explaining rationale and effects.
AcBinary: add framing doc, buffer growth fixes, doc updates
- Added `BINARY_WHYUSE.md` for architectural framing and value proposition
- Updated `BINARY_FEATURES.md` and `README.md` to reference and prioritize the new doc
- Documented AsyncPipeWriterOutput chunk-size limitation and workarounds in `BINARY_ASYNCPIPE_ISSUES.md`
- Refactored buffer growth logic in `AcBinarySerializer.BinarySerializationContext.cs` to validate capacity after grow and throw clear exceptions on under-provisioning; removed dead method
- Fixed chunk size alignment bug in `AsyncPipeWriterOutput.cs` to prevent buffer under-provisioning
- Added `AYCODE_NATIVEAOT` build config support in `Program.cs`
- Improved documentation clarity and error diagnostics for streaming/buffered serialization edge cases
Enable per-type property filter opt-out in AcBinary
Adds EnablePropertyFilterFeature to AcBinarySerializableAttribute, allowing types to opt out of property filter codegen and runtime checks. Updates source generator, metadata, and runtime logic to honor this flag. Removes UsePropertyFilter constant; emission is now attribute-driven. Also optimizes string serialization for non-ASCII cases and refactors deserializer byte reads for trusted single-segment fast paths. Backward compatible: property filter remains enabled by default.
Use ReadOnlySequence<byte> in benchmarks for deserialization
Updated all AcBinary and MemoryPack benchmark deserialization and round-trip verification methods to use ReadOnlySequence<byte> overloads instead of byte[] or ToArray(). This ensures benchmarks exercise the production-realistic deserialization path (e.g., for SignalR/Pipe consumers) and aligns buffer writer semantics across serializers. Added comments to clarify intent. No business logic was changed.
Benchmark stabilization & charset-param workload support
Major overhaul of the custom benchmark harness:
- Per-serializer warmup, GC isolation, pilot discard, and CPU pinning for stable, reproducible results
- Adaptive per-cell iteration targeting (~250ms/sample) and statistical reporting (min/max/stddev/CV)
- CLI/menu support for single-cell A/B runs
- Test data refactored to ASCII baselines with configurable charset suffix (6 presets), selectable via menu; charset recorded in all outputs
- Markdown/console output now includes per-op µs, inter-sample range, CV warnings, and iteration counts
- Documentation updated with rationale, methodology, and notes on reverted/experimental optimizations
Enables reliable, cross-charset, release-grade performance measurement for AcBinary.
SIMD Utf8Transcoder.GetUtf8ByteCount + test suite
Introduced SIMD-accelerated Utf8Transcoder.GetUtf8ByteCount for efficient UTF-8 byte counting, replacing all writer-side Encoding.UTF8.GetByteCount usages. Added 29 unit tests for correctness across ASCII, Hungarian, CJK, emoji, and boundary cases. Updated benchmarks to ensure FixStr is bypassed and wire mode is selectable. Documented implementation and dead-code review in BINARY_TODO.md. No public API changes.
SIMD-accelerated UTF-8 encode/decode for AcBinary
- Added Vector256-based SIMD path for UTF-8 char counting in deserializer, replacing scalar loop for faster ASCII/multibyte handling.
- Introduced EncodeUtf8SinglePass in serializer: layered SIMD/DWORD/scalar UTF-16→UTF-8 encoding, bypassing Encoding.UTF8.GetBytes.
- Updated serializer to use new encoder for string writes.
- Expanded "fastestbyte" benchmark mode to compare both AcBinary (UTF-8/UTF-16) and MemoryPack strategies.
- Improved comments and docs to clarify new SIMD logic.
Disable ASCII fast paths; add FastestByte mode, plan tasks
Temporarily disable ASCII string fast paths in AcBinarySerializer and AcBinaryDeserializer to isolate and benchmark the custom UTF-8 encoder/decoder. Add "FastestByte" benchmark mode for focused AcBinary vs MemoryPack Byte[] comparison. Update BINARY_TODO.md with new technical tasks for .NET 11 SIMD decoder, sentinel-length encoding, ASCII marker-dispatch, and a custom UTF-8 encoder. These changes support staged optimization and future performance improvements.
Optimize FastWire string (de)serialization and benchmarks
- Increased release benchmark iterations for more robust testing.
- Improved FastWire string deserialization with zero-copy UTF-16.
- Set FastWire and string caching options during context init/reset.
- Optimized FastWire string serialization for direct UTF-16 copy.
- Enhanced non-ASCII string fallback to use Utf8NoBom encoding.
- Refactored WriteFixStr for efficient ASCII and fallback handling.
Support FixStr for short non-ASCII strings (UTF-8)
Refactored WriteFixStrDirect to encode short non-ASCII strings as FixStr if their UTF-8 byte count is ≤31, not just ASCII. This improves efficiency for short international strings. Also ensured correct buffer sizing and direct UTF-8 encoding. No functional change to project file; property repositioned for clarity.
NativeAOT: full DAMs propagation, trimmer-safe serializers
- Propagate [DynamicallyAccessedMembers] from all public Serialize<T>/Deserialize<T> APIs through all type/property metadata and factories, centralizing requirements in TypeMetadataBase.RequiredMembers.
- Add [UnconditionalSuppressMessage] for known trimmer blind spots (polymorphism, inheritance, nested types) with detailed justifications.
- Update all internal delegate/factory signatures to preserve DAMs context.
- Annotate public APIs for AOT safety; document consumer requirements for SGen or rooted model assemblies.
- Update BINARY_FEATURES.md with NativeAOT/trimmer compatibility, guidance, and limitations.
- Adjust benchmark project for AOT/JIT parity and add i18n test data.
- No breaking API changes; SGen and Runtime paths remain, now fully AOT-compatible.
NativeAOT: fallback for delegates, exclude MessagePack
Added AYCODE_NATIVEAOT symbol for AOT builds and excluded MessagePack benchmarks from NativeAOT due to lack of AOT support. Updated AcSerializerCommon to use reflection-based delegates when dynamic code is unavailable, ensuring compatibility with both JIT and AOT. Added explanatory comments throughout.
Remove SegmentBufferReader; unify on AsyncPipeReaderInput
Migrates all SignalR chunked streaming receive logic to AsyncPipeReaderInput, fully removing SegmentBufferReader and SegmentBufferReaderInput from the codebase. Updates all references, deserialization paths, and documentation to reflect the new unified primitive. Marks ADR-0003 as accepted (partially executed), closes related TODOs, and clarifies protocol docs. Sets DoubleBuffered as the default FlushPolicy. No wire format or behavioral changes; all tests pass.
Switch to FlushPolicy enum for streaming flush control
Replaces the legacy bool waitForFlush with a new FlushPolicy enum (PerChunk, DoubleBuffered, Coalesced) across all binary streaming serialization APIs and SignalR protocol options. Updates all code, configuration, and documentation to use the new policy, clarifies memory/throughput trade-offs, and closes related TODOs. Stream-backed writers remain sequential; only parallel-capable Pipe-based writers honor the policy.
Move DrainFromAsync to tests, add in-memory benchmarks
- Moved AsyncPipeReaderInputExtensions.DrainFromAsync from the main framework to test-only assembly; no longer public API.
- Removed AcBinaryDeserializer.DeserializeFromPipeReaderAsync<T> from public API; tests now inline drain+deserialize logic.
- Added AcBinaryInMemoryPipeBenchmark and AcBinaryInMemoryRawByteArrayBenchmark to complete 2x2 transport × wire-format benchmark matrix.
- Refactored benchmark runner for interactive menu, settings, and CLI parsing.
- Expanded XML docs for AsyncPipeReaderInput and AsyncPipeWriterOutput to clarify push-pattern and real-world usage.
- Updated BINARY_ASYNCPIPE_TODO.md and related docs to reflect these changes.
Refactor pipe benchmarks to 2-task streaming model
Refactored AcBinaryNamedPipeBenchmark and AcBinaryNamedPipeRawByteArrayBenchmark to use a two-task (producer/consumer) streaming pipeline for deserialization, enabling true Ser↔Des overlap. Reduced BufferWriterChunkSize from 16K to 4K. Updated synchronization, cleanup, and comments to reflect the new architecture and improve performance comparison between chunked and raw byte[] modes.
Add raw NamedPipe benchmark & mux-mode AsyncPipe docs
- Add AcBinaryNamedPipeRawByteArrayBenchmark for raw NamedPipe (no chunking) to isolate kernel vs. AsyncPipe overhead
- Refactor progress reporting with in-place updates for all timed/allocation benchmarks
- Document [0xC8] marker as mux-mode direction; add ACCORE-BIN-T-M2X7 and ACCORE-BIN-I-C4N7 for multi-stream and single-consumer constraints
- Expand BINARY_WRITERS.md with parallel-flush regime analysis and allocation context
- Improve result comparison robustness for AsyncPipe-only runs
- Minor doc clarifications and explicit AsyncPipeReaderInput usage patterns
Modernize benchmarks, simplify attributes, doc cleanup
- Benchmark output now reports per-op µs and KB/op; added helpers for unit conversion and updated all output formats and headers.
- Split SetupAllocBytes into SetupSerializeAllocBytes and SetupDeserializeAllocBytes for finer allocation reporting.
- Simplified [AcBinarySerializable] usage in test models to single-argument form.
- Edited documentation for clarity, brevity, and consistency; improved navigation, updated technical details, and harmonized terminology across .md files.
Add AsyncPipe streaming mode, doc split, and test data tweaks
- Add AsyncPipe-only streaming mode to benchmark suite (CLI/menu)
- Aggregate and display AcBinarySerializableAttribute flags in options output
- Raise IId-ref and repeated-string share in all test data to ~20%
- Use explicit AcBinarySerializable(false, true, ...) on all test models
- Split streaming I/O issues/TODOs into BINARY_ASYNCPIPE_ISSUES.md and BINARY_ASYNCPIPE_TODO.md
- Update README and references for new streaming doc structure
- Minor code and doc cleanups for clarity and accuracy
Centralize pipe chunk size and fix buffer reset race
Centralized pipe chunk size config for all AcBinary pipe benchmarks, ensuring app-level and kernel buffer sizes stay in sync. Updated AsyncPipeReaderInput.MessageDone to atomically reset both _readPos and _writePos, preventing stale buffer reads. Improved comments and applied AggressiveOptimization to key methods. Adjusted AcquireChunk to ensure wire chunk fits exactly, avoiding kernel fragmentation. Updated related tests and documentation.
Refactor AcBinary NamedPipe to long-lived multi-message mode
Refactored the AcBinary NamedPipe benchmark to use a single long-lived AsyncPipeReaderInput in multi-message mode, with one background drain task and synchronous deserialization per message. Buffer recycling is now signaled by the consumer via a new MessageDone() method, called in the deserializer's finally block, preventing producer-consumer races. Added IsCompleted property to AsyncPipeReaderInput. Increased release-mode benchmark iteration counts. Updated documentation and comments to reflect the new architecture and rationale.
Refactor AcBinary streaming: multi-message protocol
- Renamed framing flags to multiMessage for clarity in AsyncPipeReaderInput/AsyncPipeWriterOutput.
- Multi-message mode ([202]=end-of-message) now auto-resets input for reuse; session end is explicit.
- Updated framing state machine, buffer cycling, and sentinel logic.
- Revised all serializer/deserializer entry points and tests for new protocol.
- Expanded docs and XML comments to detail wire format and protocol constraints.
- Updated benchmarks and tests for new streaming API and multi-message behavior.
- Documented protocol limits and added security issue/TODO for type-name deserialization in SignalR binary protocol.
Refactor benchmarks; clarify AcBinary doc warnings
Refactored serializer benchmark infra for richer, structured results and added fresh/reused buffer writer scenarios for AcBinary and MemoryPack. Disabled AcBinary SGen for all test models to ensure runtime/reflection-only benchmarks. Updated documentation to clarify and cross-link all silent corruption risks (hash collisions, MaxDepth, PropertyFilter), harmonized warnings, and referenced relevant issue IDs for traceability.
Refactor AcBinaryHubProtocol for thread safety
- Removed shared _currentHeaderContext; header context is now passed as a parameter through Parse* and ReadArguments/ReadSingleArgument methods, and stored per-binder for chunked messages.
- Updated AyCodeBinaryHubProtocol to use the new header context flow for type resolution and argument deserialization.
- Added concurrency tests to verify protocol instance safety under multi-threaded use and prevent state corruption or type resolution races.
- Improved documentation and comments to clarify the stateless, concurrency-safe design.
Benchmark: multi-sample median timing & EH inlining docs
Added BenchmarkSamples for multi-sample median timing in benchmarks, reducing variance and improving result stability. Updated output to show sample count. Refactored RunTimed to support multiple samples. Expanded documentation on JIT inlining barriers: clarified that EH regions (try/catch/finally/using) in hot-path and generated methods block inlining on .NET 9, and provided guidance for future generator features and stackalloc usage. Added audit requirements for EH and stackalloc in hot paths.
Moved BenchmarkTestDataProvider and TestDataSet to AyCode.Core.Tests.TestModels with public accessibility. Refactored dataset creation methods to accept a resetId parameter, allowing control over TestDataFactory ID resets. Improved code structure, formatting, and documentation for maintainability. The provider is now more flexible and easier to use in tests.
- Add TypeName to TestDataSet for clearer test scenario reporting
- Display serializer options in console and log outputs
- Extend BenchmarkResult with OptionsDescription
- Serializer benchmarks now provide detailed config summaries
- Log files now include test type and serializer options summary
- Generate .LLM Markdown results for LLM consumption and docs
- Reference .LLM results in BINARY_IMPLEMENTATION.md for visibility
Loretta