Refactored the benchmark and test data infrastructure to use generic, type-safe, and multi-variant models. Introduced generic base classes for the test data hierarchy and factories, with closing-generic aliases for _All_True and _All_False families. Benchmarks now select the correct test data variant per serializer options, and all serializers are generic over the order type. Output and result reporting now include the CLR type name for clarity. Centralized string property handling and improved documentation throughout.
Refactor tests to use _All_True model types throughout
Replaced all usages of legacy test model types (e.g., TestOrder, TestOrderItem, SharedTag, etc.) with new, feature-complete _All_True variants across SignalR test infrastructure, data sources, and service handlers. Updated all generic constraints, method signatures, and test data to use the new types. Added SharedTestBaseModels.cs and SharedTestOrderModels.cs to define abstract bases and concrete _All_True models with full serialization attributes. This enables more thorough and realistic serialization/deserialization testing and future extensibility.
Refactor: centralize config/state in Configuration.cs
Moved all benchmark configuration, mutable state, and attribute-flag aggregation from Program.cs to a new Configuration.cs static class. Updated all references in Program.cs and related benchmark classes to use Configuration.<value>. Removed the "profiler" CLI mode and its code. Updated README.md to reflect these changes. This improves maintainability and keeps Program.cs focused on orchestration and UX, with no changes to benchmark logic.
Benchmark stabilization & charset-param workload support
Major overhaul of the custom benchmark harness:
- Per-serializer warmup, GC isolation, pilot discard, and CPU pinning for stable, reproducible results
- Adaptive per-cell iteration targeting (~250ms/sample) and statistical reporting (min/max/stddev/CV)
- CLI/menu support for single-cell A/B runs
- Test data refactored to ASCII baselines with configurable charset suffix (6 presets), selectable via menu; charset recorded in all outputs
- Markdown/console output now includes per-op µs, inter-sample range, CV warnings, and iteration counts
- Documentation updated with rationale, methodology, and notes on reverted/experimental optimizations
Enables reliable, cross-charset, release-grade performance measurement for AcBinary.
SIMD Utf8Transcoder.GetUtf8ByteCount + test suite
Introduced SIMD-accelerated Utf8Transcoder.GetUtf8ByteCount for efficient UTF-8 byte counting, replacing all writer-side Encoding.UTF8.GetByteCount usages. Added 29 unit tests for correctness across ASCII, Hungarian, CJK, emoji, and boundary cases. Updated benchmarks to ensure FixStr is bypassed and wire mode is selectable. Documented implementation and dead-code review in BINARY_TODO.md. No public API changes.
SIMD UTF-8 upgrades, i18n test data, MVC disabled
- Switch all test/benchmark data to Hungarian UTF-8 strings for i18n coverage
- Add AVX-512BW, Vector256, and Vector128 SIMD paths for UTF-8/UTF-16 encode/decode (ASCII and multi-byte) in binary serializer/deserializer
- Update WireMode docs for encoding guidance per workload/host
- Block-comment and disable MVC formatters and Microsoft.AspNetCore.App reference due to .NET 10 Hybrid client conflict; update docs to reflect temporary state
- Update appsettings: replace WaitForFlush with FlushPolicy
- Revise BINARY_TODO.md for SIMD transcoder progress and next steps
NativeAOT: full DAMs propagation, trimmer-safe serializers
- Propagate [DynamicallyAccessedMembers] from all public Serialize<T>/Deserialize<T> APIs through all type/property metadata and factories, centralizing requirements in TypeMetadataBase.RequiredMembers.
- Add [UnconditionalSuppressMessage] for known trimmer blind spots (polymorphism, inheritance, nested types) with detailed justifications.
- Update all internal delegate/factory signatures to preserve DAMs context.
- Annotate public APIs for AOT safety; document consumer requirements for SGen or rooted model assemblies.
- Update BINARY_FEATURES.md with NativeAOT/trimmer compatibility, guidance, and limitations.
- Adjust benchmark project for AOT/JIT parity and add i18n test data.
- No breaking API changes; SGen and Runtime paths remain, now fully AOT-compatible.
Modernize benchmarks, simplify attributes, doc cleanup
- Benchmark output now reports per-op µs and KB/op; added helpers for unit conversion and updated all output formats and headers.
- Split SetupAllocBytes into SetupSerializeAllocBytes and SetupDeserializeAllocBytes for finer allocation reporting.
- Simplified [AcBinarySerializable] usage in test models to single-argument form.
- Edited documentation for clarity, brevity, and consistency; improved navigation, updated technical details, and harmonized terminology across .md files.
Add AsyncPipe streaming mode, doc split, and test data tweaks
- Add AsyncPipe-only streaming mode to benchmark suite (CLI/menu)
- Aggregate and display AcBinarySerializableAttribute flags in options output
- Raise IId-ref and repeated-string share in all test data to ~20%
- Use explicit AcBinarySerializable(false, true, ...) on all test models
- Split streaming I/O issues/TODOs into BINARY_ASYNCPIPE_ISSUES.md and BINARY_ASYNCPIPE_TODO.md
- Update README and references for new streaming doc structure
- Minor code and doc cleanups for clarity and accuracy
Refactor benchmarks; clarify AcBinary doc warnings
Refactored serializer benchmark infra for richer, structured results and added fresh/reused buffer writer scenarios for AcBinary and MemoryPack. Disabled AcBinary SGen for all test models to ensure runtime/reflection-only benchmarks. Updated documentation to clarify and cross-link all silent corruption risks (hash collisions, MaxDepth, PropertyFilter), harmonized warnings, and referenced relevant issue IDs for traceability.
Moved BenchmarkTestDataProvider and TestDataSet to AyCode.Core.Tests.TestModels with public accessibility. Refactored dataset creation methods to accept a resetId parameter, allowing control over TestDataFactory ID resets. Improved code structure, formatting, and documentation for maintainability. The provider is now more flexible and easier to use in tests.
- Add BINARY_FORMAT.md: full AcBinary wire format spec (markers, encoding, options, protocol, interactions)
- Reference BINARY_FORMAT.md from GLOSSARY.md, Binaries/README.md, and Serializers/Binaries/README.md; add new glossary terms
- Clarify and expand config options tables to match new doc
- Add/clarify LLM maintenance rules: always sync .md files with code, auto-fix discrepancies
- Update root README.md: AyCode.Core targets .NET 9, not 10; stress doc/code sync
- Add code reuse and doc sync conventions to copilot-instructions.md and CONVENTIONS.md
- Add docs/ folder and BINARY_FORMAT.md to solution as Solution Items
- Minor clarifications and cross-links in ARCHITECTURE.md and other docs
- Introduced `.github/copilot-instructions.md` as the single source of truth for domain rules, conventions, and pitfalls in each solution.
- Added `CLAUDE.md` to guide Claude to read domain rules, glossary, and README before code generation.
- Updated all solution and project `README.md` files to document project purpose, structure, key files, and LLM context (Copilot/Claude/Cursor).
- Added or revised `docs/ARCHITECTURE.md`, `docs/CONVENTIONS.md`, and `docs/GLOSSARY.md` to clarify dependency graphs, naming, patterns, and terminology.
- For FruitBankHybridApp, added `docs/SCHEMA.md` (Toon format) and expanded the glossary with business/measurement terms and common traps.
- Updated all subfolder READMEs to list key files, conventions, and LLM maintenance notes.
- Ensured all documentation is cross-referenced, up-to-date, and includes explicit instructions for LLMs to keep docs in sync with code and avoid suggesting removal/rollback as a solution.
- Standardized documentation and onboarding for maintainability and LLM/code quality across all solutions.
Major refactor of binary serialization codegen and runtime:
- Added property writer bridge methods for markerless/metadata paths
- Centralized object marker logic via new bridge methods
- Simplified SGen output: single bridge call replaces branching
- FixObj slot markers now supported in serialization/deserialization
- Refactored collection/dictionary element serialization
- Removed redundant WritePropertyMarkerless method
- Improved tests: use BinaryTypeCode constants, FixObj parsing
- Added InternalsVisibleTo for test project access
- Annotated TestSimpleClass for SGen support
Reduces generated code size, improves maintainability, and ensures correct handling of new binary format features.
Introduce new test models for circular refs, update tests to stress reference handling, and enhance deserializer to support ObjectRefFirst/WithMetadataRefFirst type codes. Fix intern cache index assignment, track generated readers in TypeMetadataWrapper, and disable UseGeneratedCode by default. Update benchmarks for reliability and diagnostics. These changes strengthen reference resolution, circular ref support, and performance.
- Refined collection kind mapping: IList<T>/IReadOnlyList<T> now treated as "IndexedCollection" for codegen, distinct from List<T>
- Generated code uses CollectionsMarshal.AsSpan for List<T> iteration, improving performance
- Updated generated read/write logic for collections to match new distinctions
- Added System.Runtime.InteropServices to generated code for span support
- Increased test iteration count in Program.cs for more robust benchmarks
- Enabled source-generated binary serialization for all test models by setting [AcBinarySerializable(true)]
Introduced source-generated binary readers for all serializable types, alongside writers. Added IGeneratedBinaryReader interface and registry for fast lookup in deserializer. Generated readers eliminate runtime overhead by directly reading properties, bypassing wrapper and delegate calls. Updated test models to disable metadata for markerless mode. Integrated fast path in deserializer for improved performance.
- Set `[AcBinarySerializable(true)]` on all SharedTestModels types
- Default `UseMetadata` to true for property hash footer
- Reset slotted ID maps and cache indices in serializer context
- Reduce test iterations for DEBUG builds to speed up runs
- Add debug comments in context `Clear()` method
AcBinarySourceGenerator now reads feature flags from [AcBinarySerializable], enabling selective code generation for ID tracking, reference handling, and string interning. Property ordering is always alphabetical, removing "Id"-first sorting for IId types. Reference tracking code is only emitted when features are enabled. TypeMetadataBase and AcBinarySerializer runtime logic now respect these flags. Default options updated: ReferenceHandlingMode is All, UseMetadata is false. Test models explicitly disable all features. Comments and code structure improved for clarity.
- Integrated MemoryPack as a serializer in the benchmark suite
- Added [MemoryPackable] and [MemoryPackIgnore] to test models
- Enabled AcBinary source generation by default
- Updated benchmark app to include MemoryPack and focus on key serializers
- Added MemoryPack NuGet references to projects
- Refactored AcBinarySerializer.WriteString flag handling
Introduce [AcStringIntern] for selective string interning on properties. Update serializer to use a per-property eligibility flag, improving efficiency and control over which string fields participate in interning during binary serialization. Update test models and internal context accordingly.
- Moved test data creation to BenchmarkTestDataProvider.cs and removed from Program.cs for better organization.
- Added [MessagePackObject]/[Key] attributes to all test models for explicit MessagePack serialization.
- Updated MessagePack benchmark to use MessagePackSerializerOptions.Standard.
- Improved AcBinaryDeserializer string cache with ASCII byte match to prevent hash collision bugs.
- Optimized AcBinarySerializer/Deserializer for string property handling and non-primitive writes.
- Set AcBinarySerializerOptions.UseMetadata default to true for safer deserialization.
- Test data now controls IId shared ref % for realistic deduplication benchmarks; display names include IId ref ratio.
- Added deep-level clearing of IId refs for realistic object graphs.
- Pallet, Measurement, and Point models now support shared IId refs.
- TestDataFactory passes shared refs to all hierarchy levels.
- Refactored TypeMetadataWrapper for type-specific Id getters, identity maps, and registration—removes hot path type checks/switches.
- AcBinary deserializer now uses new typed methods for reference tracking and registration.
- SerializationContextBase uses pre-cast Id getters for zero-overhead tracking.
- Reduced quick benchmark warmup iterations for faster startup.
- Improves performance, clarity, and maintainability of reference handling and benchmarks.
Introduce Roslyn Source Generator for AcBinary serialization, generating optimized Serialize/Deserialize methods for types marked with [AcBinarySerializable]. Integrate generator as analyzer in core, tests, and benchmarks. Update metadata to detect generated serializers. Add benchmarks and integration tests to validate performance and correctness. Update project files and internals visibility for testing. Existing runtime serialization remains as fallback.
- Changed TestStatus enum to use non-sequential values (5, 10, 20, ...)
- Updated related tests and deserialization logic for new enum values
- Changed PropertySkip marker from 253 to 191 to avoid TinyInt conflicts
- PropertySkip now only written for null references, not empty values
- Improved handling of skipped enum and nullable properties in deserializer
- Enhanced compiled property setter for nullable types
- Added comprehensive int serialization tests, including edge cases
- Fixed namespace casing and added missing using directive
- Introduced a high-performance, reusable "chain" API for JSON deserialization and object population, enabling parsed JSON to be reused for multiple deserializations without reparsing. Exposed via new extension methods and interfaces (`IDeserializeChain<T>`, `IPopulateChain`).
- Added comprehensive infrastructure for serializing and deserializing .NET Expression trees to a universal DTO (`AcExpressionNode`), with full round-trip support and robust handling of constants, closures, and queryable expressions.
- Centralized all property accessor and expression utilities in `AcSerializerCommon` to avoid duplication and improve maintainability.
- Enhanced both JSON and binary serializers to support Expression and IQueryable types, with automatic conversion to/from `AcExpressionNode`.
- Refactored type metadata and property accessor logic for performance and code reuse.
- Added extensive unit tests for the new chain API and expression serialization, and reorganized test models and namespaces for clarity.
- Improved logging, error handling, and test infrastructure.
- Misc: Added settings for local builds, updated project files, and cleaned up obsolete code.
- Introduced AcJsonSerializer/AcJsonDeserializer in AyCode.Core.Serializers.Jsons, optimized for IId<T> reference and circular reference handling.
- Added AcJsonSerializerOptions/AcSerializerOptions for configurable reference handling and max depth.
- Implemented fast-path streaming (Utf8JsonReader/Writer) with fallback to DOM for reference scenarios.
- Added type metadata/property accessor caching for performance.
- Provided robust object/collection population with merge semantics for IId<T> collections.
- Added AcJsonDeserializationException for detailed error reporting.
- Implemented UnifiedMergeContractResolver for Newtonsoft.Json, supporting JsonNoMergeCollectionAttribute to control merge behavior.
- Added IdAwareCollectionMergeConverter<TItem, TId> for merging IId<T> collections by ID.
- Included helpers for ID extraction and semantic ID generation.
- Added DeepPopulateWithMerge extension for deep merging.
- Optimized with frozen dictionaries, pre-encoded property names, and context pooling.
- Ensured compatibility with both System.Text.Json and Newtonsoft.Json.
- Fix: Ensure property names and skipped strings are interned during binary deserialization, preventing interned string index mismatches.
- Add: Extensive regression tests for string interning edge cases, including property mismatch (server has more properties than client), deeply nested objects, repeated/unique strings, and empty/null handling.
- Add: New test DTOs and infrastructure for property mismatch scenarios.
- Update: SignalR test helpers and services to support both JSON and Binary serialization in all tests.
- Improve: SignalRRequestModelPool now initializes models on Get().
- These changes address production bugs and ensure robust, consistent string interning and property skipping in AcBinarySerializer.
Add regression tests for string-stored DateTime values in GenericAttribute-like collections, ensuring exact value preservation in Binary serialization. Fix AcBinaryDeserializer to only intern strings >= 4 chars, matching serializer logic and preventing intern table index mismatches. Refactor SignalR tests to run under both JSON and Binary modes, and add comprehensive tests for edge cases and production bug scenarios. Clean up test code and comments for clarity.
Introduces SignalResponseBinaryMessage for efficient binary serialization of response data alongside existing JSON support. Adds utilities to detect serializer type and updates both server and client logic to handle JSON or binary formats automatically. Refactors response creation, logging, and deserialization for consistency and performance. Updates benchmarks and ensures all MessagePack operations use ContractlessStandardResolver.Options. Improves robustness and backward compatibility in client callback handling.
- Introduce AcBinarySerializerTests with full coverage for primitives, objects, collections, merge/populate, and size comparisons
- Add AcBinarySerializer class stub as a placeholder for implementation
- Extend serialization extension methods with binary helpers (ToBinary, BinaryTo, BinaryCloneTo, etc.)
- Update test models to ignore parent references for all major serializers ([IgnoreMember], [BsonIgnore])
- Refactor benchmarks: split into minimal, simple, complex, and MessagePack comparison; add command-line switches and improved size reporting
- Optimize AcJsonDeserializer with fast UTF-8 property lookup and direct primitive setting
- Add MessagePack and MongoDB.Bson dependencies to test and benchmark projects
- Add (accidentally) a summary of less commands as a documentation artifact
Major overhaul of JSON serialization/deserialization:
- Added AcJsonDeserializationException for clearer error reporting.
- Robust JSON validation, type checking, and double-serialization detection.
- Fast-path primitive (de)serialization for all common .NET types.
- Direct support for Dictionary<TKey, TValue> and improved collection handling.
- Enhanced $id/$ref reference resolution and in-place merging.
- Optimized property metadata caching and filtering.
- Comprehensive error handling with detailed messages.
- Extensive new tests for primitives, collections, references, and error cases.
- Benchmarks for serialization, deserialization, and merge scenarios.
SignalR infrastructure improvements:
- Refactored AcSignalRClientBase/AcWebSignalRHubBase for testability (virtual methods, test constructors).
- Added SignalRRequestModelPool for efficient request/response tracking.
- Improved parameter deserialization and IdMessage handling.
- New tags (PingTag, EchoTag) for SignalR messaging.
New test and benchmark infrastructure for SignalR:
- Shared test models and data factories for DTOs, primitives, and complex graphs.
- In-memory, dependency-free SignalR client/hub for round-trip testing.
- Exhaustive test suites for message processing, method invocation, and edge cases (including prior production bugs with Task<T>).
- Benchmarks for SignalR serialization, deserialization, and round-trip performance.
Other changes:
- Improved TaskHelper.WaitTo/WaitToAsync with more accurate polling and cancellation support.
- ExtensionMethods.InvokeMethod now properly unwraps Task/ValueTask results.
- General code cleanup, improved comments, and removal of obsolete code.
- Project references updated for shared test/benchmark infrastructure.
These changes deliver a robust, high-performance, and fully tested JSON/SignalR (de)serialization system, ready for production and advanced testing scenarios.
Loretta