Commit Graph

426 Commits

Author SHA1 Message Date
Loretta deffb77de4 Add full UseMetadata support to SGen with inline hashes
Implements inline type metadata emission in the source generator, matching runtime TypeMetadataBase. Computes FNV-1a hashes for type and property names, stores them in generated code, and emits metadata when UseMetadata is enabled. Adds per-type slot allocation and tracking for first/repeated metadata writes. Removes runtime fallback for UseMetadata, ensuring all logic is handled inline. Updates property filtering/order to match runtime, and optimizes Int32/Int64 skip logic. Thread-safe slot allocation is used for metadata tracking.
2026-02-18 18:43:21 +01:00
Loretta e2269d3ecf Refine AcBinary gen: metadata, ref tracking alignment
Improve generated serialization code to match runtime behavior for metadata emission and reference tracking. Markerless types now respect UseMetadata, ensuring type markers are written when required. Ref tracking guards for IId and non-IId types are unified to match scan pass logic. Generated writers are always used when available. Default UseMetadata is now true for consistent metadata output.
2026-02-18 13:07:26 +01:00
Loretta d40e40a45a Support nested types in source gen; improve prop filtering
- Generate writers for nested types using flat class names (Outer_Inner_Leaf) to ensure uniqueness and validity.
- Apply property filters in generated code for all non-markerless properties, matching runtime behavior.
- Emit skip labels for each property in generated code for correct control flow.
- Remove PropertyFilter check from IsDirectObjectWrite; generated code now handles filtering.
- Change default ReferenceHandlingMode to All.
- Make BinaryPropertyFilterContext constructor public.
- Increase release warmup iterations in Program.cs from 3000 to 5000.
2026-02-17 21:07:19 +01:00
Loretta 418d9f839a Disable FastWire mode; enforce compact encoding only
All FastWire-related code paths, fields, and options have been commented out in both AcBinarySerializer and AcBinaryDeserializer. This removes support for fixed-width integer and UTF-16 encoding, forcing the use of compact VarInt and UTF-8 encoding exclusively. The WireMode option is also commented out, so FastWire can no longer be selected. This change reduces output size and simplifies the codebase, but may impact serialization/deserialization speed for some scenarios.
2026-02-17 15:25:51 +01:00
Loretta 98d7a27245 Add WireMode for fast/compact binary serialization
Introduces a WireMode enum to select between Compact (VarInt + UTF-8) and Fast (fixed-width + UTF-16) wire formats for binary serialization. Updates AcBinarySerializerOptions to include a WireMode property (default: Fast). Serialization and deserialization logic now conditionally uses fixed-width or variable-length encoding for integers and strings based on the selected mode, enabling a tradeoff between output size and performance.
2026-02-17 09:53:15 +01:00
Loretta b244d9219a Add fast-path support for more primitive collections
Extend serializer/deserializer to efficiently handle arrays and lists of short, ushort, uint, ulong, sbyte, char, DateTimeOffset, and TimeSpan. Improve enum collection handling and support ICollection-based collections. Update source generator to distinguish between indexable and counted collections, generating optimal serialization code for each. These changes enhance performance and coverage for a broader range of collection types.
2026-02-16 21:26:51 +01:00
Loretta 7977feb36a Optimize collection serialization for complex element types
Enhance AcBinarySourceGenerator to emit direct write loops for List<T>/T[] where T is a complex type with a generated writer. This bypasses per-element runtime dispatch, improving serialization performance. PropInfo now tracks collection element metadata, enabling this optimization. Falls back to generic handling for other cases.
2026-02-16 19:37:02 +01:00
Loretta 9973b6be12 Update string serialization to always write length prefix
- Always write length prefix before ASCII and UTF-8 strings in AcBinarySerializer for consistency and correctness.
- Increase release warmup iterations from 2000 to 3000 in Program.cs.
2026-02-16 18:13:14 +01:00
Loretta 11e71336b0 Direct object write mode for binary serialization generator
Introduces direct object write mode in source-generated binary serialization, allowing generated writers to bypass the runtime WriteObject pipeline for eligible complex properties. Adds detection of generated writer presence and IId<T> implementation, inlines reference tracking logic, and replaces .GetType() with typeof() for type safety. String interning attribute detection and precomputed interning flags ensure safe cursor alignment. Expands PropInfo structure and enables fast path for all generated writers. Improves serialization performance, correctness, and reduces runtime overhead. Adds new web fetch domains to settings.
2026-02-16 15:06:01 +01:00
Loretta 03f5809e8a Optimize complex object serialization in source gen
Add WriteObjectGenerated to AcBinarySerializer for direct, efficient serialization of complex objects in source-generated code. Update AcBinarySourceGenerator to use this method, including proper handling of nullable properties, to reduce runtime overhead and avoid unnecessary type dispatch.
2026-02-16 13:33:15 +01:00
Loretta 7284856dda Unify and optimize primitive array/list serialization
Refactor AcBinarySerializer to use ReadOnlySpan<T> for bulk writing of primitive arrays and List<T>, replacing multiple specialized methods with a single TryWritePrimitiveCollection. This improves efficiency and reduces code duplication. Change default string interning mode to Attribute (opt-in). Update generated code path to allow reference tracking but not string interning. Adjust benchmarks to test correct serializer options. Reorder options for clarity.
2026-02-16 13:18:13 +01:00
Loretta dcd44cf705 Add MemoryPack benchmarks and model support
- Integrated MemoryPack as a serializer in the benchmark suite
- Added [MemoryPackable] and [MemoryPackIgnore] to test models
- Enabled AcBinary source generation by default
- Updated benchmark app to include MemoryPack and focus on key serializers
- Added MemoryPack NuGet references to projects
- Refactored AcBinarySerializer.WriteString flag handling
2026-02-16 07:59:24 +01:00
Loretta e30efff56c Enable property-level string interning via attribute
Introduce [AcStringIntern] for selective string interning on properties. Update serializer to use a per-property eligibility flag, improving efficiency and control over which string fields participate in interning during binary serialization. Update test models and internal context accordingly.
2026-02-15 19:01:21 +01:00
Loretta 6f88306e54 Optimize serializer with write plan for interning & refs
Implement write plan mechanism for string interning and IId object reference tracking. Scan pass now builds pre-computed WriteDuplicateEntry instructions, eliminating hot path IdentityMap lookups and redundant getter calls in the write pass. Update BinarySerializationContext, tracking visit indices and managing write plan array. Refactor ScanInternString and TryTrack methods to record visit indices and build write instructions for all duplicate occurrences. Update write pass logic to consume write plan entries. Add debug validation for scan/write pass order. Update benchmarks and test harness. Set UseGeneratedCode default to false. Improves performance for scenarios with interning and reference tracking.
2026-02-15 17:28:06 +01:00
Loretta 1af939ac4d Support markerless serialization for bool and enum types
Extend markerless serialization to Boolean and Enum types in AcBinarySourceGenerator and generated writers. Updated IsMarkerlessType and EmitMarkerlessWrite to handle these types directly. Refactored generated code to write DateTime, int, enum, and decimal properties without type markers or property skip checks. Simplified OnStringInterned debug callback in AcBinarySerializer. These changes reduce serialization overhead and improve performance for supported types.
2026-02-15 10:57:09 +01:00
Loretta 58cf9578c7 Optimize bool/enum serialization and ASCII string decoding
Improves binary serializer/deserializer performance by adding fast-path handling for boolean and enum properties, mapping their type codes directly for efficient read/write. Introduces a fast ASCII-only string decoding path for short strings, bypassing UTF8 overhead. Refactors array/list population to reduce redundant marker reads. Also applies aggressive inlining to core populate logic for further speedup.
2026-02-15 10:37:12 +01:00
Loretta e50dca93fa Refactor deserializer marker handling; add UseGeneratedCode opt
Refactored AcBinaryDeserializer to read the type code marker byte only once per property, eliminating redundant PeekByte/ReadByte calls and improving efficiency. Updated all property population branches to use the already-consumed type code. Adjusted handling of nested complex objects to rewind the marker byte when needed. Modified TryReadAndSetTypedValue to assume the marker is already consumed, removing unnecessary reads. Exception messages now report the actual type code read.

Added UseGeneratedCode option (default true) to AcBinarySerializerOptions and exposed it in the serialization context. The generated code fast path is now gated by this option, allowing users to enable or disable source-generated serialization. These changes improve deserialization performance, code clarity, and configurability.
2026-02-15 09:50:16 +01:00
Loretta 12b3244aa3 Refactor WriteValueGenerated for clarity and efficiency
Update WriteValueGenerated to require non-null values and call WriteValueNonPrimitive directly, clarifying its use for complex types in generated writers. Improve XML docs to reflect new behavior and intent.
2026-02-14 21:04:31 +01:00
Loretta 4ef65ee501 Generate IGeneratedBinaryWriter for fast serialization
Refactor source generator to emit per-type IGeneratedBinaryWriter classes for [AcBinarySerializable] types, with auto-registration at startup. Integrate generated writers into AcBinarySerializer for direct, delegate-free property writing, bypassing the runtime property loop when possible. Add registry, bridge methods, and update TypeMetadataWrapper for fast lookup. Expand tests to verify generated writers and round-trip correctness. This enables major serialization performance gains and reduces code size for supported types.
2026-02-14 20:50:38 +01:00
Loretta 896f720109 Add JIT disassembly benchmark and improve string interning
Introduce JitDisassemblyBenchmark for analyzing JIT-generated x64 assembly of AcBinarySerializer hot paths, accessible via --jitasm. Refactor string interning logic to support per-property and string collection interning, adding IsStringCollectionProperty and ScanStringCollection. Update ScanPass and WriteString for finer-grained control. Remove DEBUG-only CurrentPropertyPath in favor of a more robust property tracking approach. Update usage instructions and clean up related code.
2026-02-14 19:53:23 +01:00
Loretta bfab7c16b9 Improve string interning logic in AcBinarySerializer
- Respect both global and property-level [AcStringIntern] settings for string interning
- Add UseStringPropertyInterning method and flag-based caching in property accessors for fast runtime checks
- Update scan and write passes to use property-level interning decisions
- Introduce FilteredReferenceProperties for efficient scan filtering in TypeMetadataWrapper
- Refactor benchmarks to use correct serializer options
- Add TODOs and minor cleanups for clarity and future improvements
2026-02-14 11:07:26 +01:00
Loretta 7e7918e071 Optimize AcBinary hot paths with pre-cached type wrappers
Pre-caches TypeMetadataWrapper instances for complex properties,
eliminating repeated GetWrapper dictionary lookups in serialization
and deserialization. Adds ComplexPropertyIndex and ComplexPropertyCount
fields, and PropertyTypeWrappers array to TypeMetadataWrapper. Refactors
scan, write, and populate passes to use cached wrappers, improving
performance for deep and polymorphic object graphs. Updates benchmarks
to focus on FastMode variants. No breaking changes; internal efficiency
improved.
2026-02-14 01:41:54 +01:00
Loretta a0a6ac8ef4 Optimize AcBinary scan, string caching, and benchmarks
- Refactor collection scan to pre-cache element wrappers and optimize ScanItem for polymorphic types
- Add DisableStringCaching to deserializer; call it on first interned string marker
- Update benchmarks to restore default and no-ref variants, clarify string interning options
- Ensure property scanning respects property filters, skipping filtered properties
2026-02-13 22:58:07 +01:00
Loretta f84dcb773d Refactor Populate and add PopulateMerge for byte[] input
Refactored Populate<T>(byte[], T) to add null checks, early returns, and proper context management. Added PopulateMerge<T>(byte[], T) for merge semantics with IId collections, including merge mode and orphaned item removal.
2026-02-13 09:46:44 +01:00
Loretta 96409fe321 Refactor: generic binary input for deserialization
Refactored deserialization to use IBinaryInputBase abstraction, supporting both ArrayBinaryInput (byte[]) and SequenceBinaryInput (ReadOnlySequence<byte>). All context and methods are now generic over TInput, enabling zero-copy for arrays and true streaming for multi-segment sources. Internal logic (ReadValue, Populate, Skip, etc.) is specialized per input type, improving performance and flexibility. This enables future extensibility for other input sources and optimizes handling of large or segmented payloads.
2026-02-13 09:32:32 +01:00
Loretta 4c6342aa2b Refactor: unify deserialization context into single class
Eliminate BinaryDeserializationContextClass and merge all buffer, pooling, and cache logic into a single sealed BinaryDeserializationContext. All pooling and state management is now handled directly by the context, reducing allocations and indirection. All deserialization entry points and internal logic are updated to use the new context class and pool. String interning, metadata, and array pooling are now managed within the unified context, improving performance and maintainability.
2026-02-12 16:24:58 +01:00
Loretta 97e4315d12 Replace Newtonsoft.Json benchmark with AcBinary BufferWriter
Removed Newtonsoft.Json from benchmarks and codebase. Added AcBinaryBufferWriterBenchmark using ArrayBufferWriter and AcBinarySerializer's buffer writer API. Optimized WriteStringUtf8 for ASCII fast path. Improved ArrayBinaryOutput buffer reuse and memory management. Introduced Reset method to IBinaryOutputBase and implemented it in outputs. Streamlined serializer benchmarks to focus on AcBinary and System.Text.Json.
2026-02-11 17:29:19 +01:00
Loretta 991e8f6038 Refactor: move buffer/position to context for zero dispatch
Major serialization pipeline refactor: all hot-path buffer and position management is now owned by BinarySerializationContext<TOutput>, with all write methods inlined for zero virtual dispatch. TOutput (now struct, IBinaryOutputBase) handles only cold-path buffer management (Initialize, Grow, GetTotalPosition). ArrayBinaryOutput and BufferWriterBinaryOutput are simplified to buffer managers. IBinaryOutput and BinaryOutputBase are removed. All serialization logic now uses context write methods (~130 call sites updated). This yields significant performance gains by eliminating virtual/interface calls on the serialization hot path.
2026-02-11 13:02:24 +01:00
Loretta 270f1b8265 Refactor: make AcBinarySerializer fully generic on output
Major internal refactor: AcBinarySerializer and BinarySerializationContext are now generic on TOutput : BinaryOutputBase, enabling JIT devirtualization and eliminating virtual dispatch in hot serialization loops. All serialization logic (WriteValue, WriteObject, WriteArray, etc.) is now generic on TOutput and delegates buffer operations to the output instance (ArrayBinaryOutput or BufferWriterBinaryOutput). Context pooling is now per output type. All buffer management is moved to output classes. The public API is unchanged, but the internal architecture is now fully generic and ready for further JIT optimizations. Also disables the source generator and sets UseMetadata=false by default.
2026-02-10 20:24:47 +01:00
Loretta 0bde311aa1 Add high-performance binary output abstraction
Introduce ArrayBinaryOutput, BufferWriterBinaryOutput, BinaryOutputBase, and IBinaryOutput for flexible, efficient serialization to pooled arrays and streaming buffers. Refactor AcBinaryHubProtocol to use new output for SignalR. Make UseMetadata runtime-settable in AcBinarySerializerOptions. Update QuickBenchmark for new options. Enables allocation-free, extensible binary serialization infrastructure.
2026-02-10 07:12:11 +01:00
Loretta 97b7813633 Markerless serialization for value types (UseMetadata=false)
Introduced markerless serialization/deserialization for non-nullable value type properties when UseMetadata is false, eliminating type marker bytes for int, long, double, etc. Added ExpectedTypeCode to property accessors/setters to enable this optimization. Refactored property loops in serializer/deserializer for performance and clarity. Default UseMetadata is now false. Improves speed and reduces stream size for common value types while maintaining compatibility for complex types.
2026-02-09 08:38:47 +01:00
Loretta b38fd480d8 Refactor test data, MessagePack, and serializer logic
- Moved test data creation to BenchmarkTestDataProvider.cs and removed from Program.cs for better organization.
- Added [MessagePackObject]/[Key] attributes to all test models for explicit MessagePack serialization.
- Updated MessagePack benchmark to use MessagePackSerializerOptions.Standard.
- Improved AcBinaryDeserializer string cache with ASCII byte match to prevent hash collision bugs.
- Optimized AcBinarySerializer/Deserializer for string property handling and non-primitive writes.
- Set AcBinarySerializerOptions.UseMetadata default to true for safer deserialization.
2026-02-08 10:25:23 +01:00
Loretta b37d873792 Optimize serialization: precompute type metadata, remove caches
Refactored serialization for performance:
- Precompute type metadata (primitive, collection, element info) in TypeMetadataBase
- Remove runtime type caches from JsonUtilities
- Rewrite primitive/collection checks to use direct logic or metadata
- Update scan pass and serialization hot path to use wrappers/metadata
- Improve buffer management (halve oversized buffers)
- Increase profiler warmup iterations, comment out deserialization in hot path
- Clean up code and clarify documentation/comments

Reduces runtime overhead and memory usage, streamlines hot path execution.
2026-02-08 08:13:34 +01:00
Loretta 5a174ced4c Refactor: add pooled context for zero-alloc deserialization
Refactored binary deserialization to use a pooled BinaryDeserializationContextClass, eliminating per-call heap allocations and enabling cache reuse for string and intern caches. Introduced DeserializationContextClassPool for efficient context management. Updated all deserialization entry points to use the pool with proper disposal. Added efficient ReadOnlySequence<byte> support. Changed AcBinarySerializerOptions.UseMetadata default to false. These changes reduce GC pressure and improve performance, especially for high-throughput and WASM scenarios.
2026-02-07 18:21:10 +01:00
Loretta c1dc203dad Refactor header writing; add PooledBufferWriter & IBufferWriter support
Refactored AcBinarySerializer to write the binary header directly after scanning for duplicates, removing the placeholder and patching logic for improved performance and simplicity. Introduced a high-performance, ArrayPool-backed PooledBufferWriter for efficient buffer management and pooling. Added a new Serialize<T> overload supporting direct serialization to IBufferWriter<byte>, enabling zero-copy and high-throughput scenarios. These changes streamline serialization, reduce memory copying, and enhance extensibility.
2026-02-07 15:26:39 +01:00
Loretta 6b7f4bf44f Set SignalR client log level to Warning
Changed minimum log level from Trace to Warning in SignalR client configuration to reduce log verbosity and record only warnings, errors, and critical logs.
2026-02-06 19:00:57 +01:00
Loretta 9b4fa1159a Optimize cache index assignment during scan pass
Refactored AcBinarySerializer to assign cache indices immediately upon detecting duplicates during the scan pass, eliminating the need for a separate post-processing step. Updated TryTrack methods to take a ref nextCacheIndex for inline assignment. Removed AssignCacheIndicesInOrder and related code, simplified string interning, and made RegisterMetadataType static. This reduces allocations and improves performance by making cache index assignment a single-pass operation.
2026-02-06 15:48:48 +01:00
Loretta e5d4b1091f Two-pass serialization for string/object interning
Refactor binary serializer to use a true two-pass process for string interning and object reference tracking. Adds a scan pass to identify duplicates and assigns cache indices deterministically in first-occurrence order. Updates wire format to write explicit cache indices after *First markers. Refactors InternEntry, removes marker rewriting, and updates deserializer to match new format. Improves performance, correctness, and robustness for complex object graphs with shared references and repeated strings.
2026-02-06 09:55:28 +01:00
Loretta a87dc37b8b Add ReferenceProperties for efficient reference tracking
Introduce a lazily-computed ReferenceProperties array to AcBinarySerializer, containing only complex and string properties. This enables efficient iteration and reference tracking during serialization by filtering relevant properties on first access.
2026-02-05 17:11:24 +01:00
Loretta e8a0d36e43 Switch to marker-based interned value serialization
Refactor interned string/object tracking from footer-based to marker-based format.
- Serializer rewrites type code markers for first occurrences (StringInternFirst, ObjectRefFirst, ObjectWithMetadataRefFirst).
- Header now stores VarUInt cache count instead of footer position.
- Deserializer registers interned values sequentially as markers are encountered, eliminating footer parsing.
- Simplified registration logic and updated BinaryTypeCode constants.
- Improves cache locality, format compactness, and performance.
2026-02-05 08:03:44 +01:00
Loretta 097c1e8efe Refactor deserialization property cache construction
Move CacheMap building to dedicated method for efficiency.
Remove incremental cache logic and SourceHashes field.
Simplify property population and update documentation.
Improves performance and code clarity.
2026-02-05 07:12:08 +01:00
Loretta cd3d65b5f4 Refactor UseMetadata to support inline metadata entries
Refactored deserialization logic to use inline metadata for UseMetadata mode, replacing footer-based property hash lookup. Context now tracks inline metadata entries and builds cache maps incrementally per source type. Unified property population for both metadata modes. Updated TypeMetadataWrapper to manage cache map and source hashes per context. Improved robustness for runtime source type changes and streamlined object skipping logic. Updated method signatures to use BinaryPropertySetterBase abstraction.
2026-02-04 20:29:08 +01:00
Loretta 1410ee71f0 Migrate UseMetadata to inline format, remove metadata footer
Refactored AcBinarySerializer to write type property metadata inline
after the ObjectWithMetadata marker, eliminating the need for a
separate metadata footer section. Updated serialization, deserialization,
and diagnostic test logic to support the new inline metadata format.
Also updated settings.local.json to allow "Bash(git stash:*)" commands.
2026-02-04 16:04:53 +01:00
Loretta 18370879ec Add pure managed LZ4 compression to serializers
Implemented LZ4 compression/decompression in pure managed code, compatible with all platforms including WASM. Added new helpers (`Lz4`, `Lz4Compressor`, `Lz4Decompressor`) and a `Lz4CompressionMode` enum. Updated `AcBinarySerializerOptions` to support compression, and modified all relevant serializer methods to apply LZ4 when enabled. Benchmarks and buffer handling updated to support zero-allocation compression. No native dependencies required.
2026-02-04 14:36:16 +01:00
Loretta 3da902b575 Enable cross-type deserialization via property hashes
Introduce UseMetadata mode with FNV-1a property name hashing.
Write per-type property hashes to metadata footer for robust
property matching during deserialization. Remove legacy property
name table logic. Add ObjectWithMetadata marker and cachemap
logic for nested objects. Enable duplicate hash detection and
make UseMetadata default. Improves schema evolution support.
2026-02-04 09:38:49 +01:00
Loretta b7cb6256a0 Refactor list factory to support capacity for deserialization
Update list factory cache and GetOrCreateListFactory to accept a capacity parameter, enabling preallocation of lists during deserialization. Adjust deserializer code to pass collection size where available, improving performance and memory usage. Also, pre-size dictionaries on creation and add Bash(find:*) to allowed commands in settings.local.json.
2026-02-02 19:02:17 +01:00
Loretta bc62488965 Pool arrays for dup data and intern cache in deserializer
Reduce allocations by pooling int[] and object?[] arrays used for duplicate data and interned references during binary deserialization. Arrays are now rented and reused via ArrayPool<T>, with logic to clear or return them as appropriate. This improves performance and reduces GC pressure in steady-state scenarios.
2026-02-02 10:11:45 +01:00
Loretta 9b151fd6cf Unify string interning and object reference tracking
Major refactor: merges string interning and object reference tracking (IId/Non-IId) into a unified position-based cache and footer in binary serialization. Wire format now uses cache indices for all references; hashcode/Id prefixes removed. Serialization and deserialization logic simplified, improving performance and maintainability. Legacy code paths and redundant dictionary lookups eliminated.
2026-02-01 12:18:27 +01:00
Loretta 23af1fc98b Refactor string interning to use IdentityMap and InternEntry
Replaces Dictionary-based string interning in AcBinarySerializer.BinarySerializationContext with a new IdentityMap<string, InternEntry> approach. Introduces the InternEntry struct for efficient tracking of stream position and cache index. Updates all related logic and iteration to use the new IdentityMap API, improving performance, memory usage, and code clarity for interned string and identity tracking during serialization.
2026-02-01 10:52:53 +01:00
Loretta 1c41eba96e Refactor IdentityMap to be generic over key and value types
IdentityMap is now fully generic as IdentityMap<TKey, TValue>, enabling type-safe value storage and improved flexibility. All internal logic and method signatures are updated to use TKey and TValue. The small int optimization for value storage is removed, and the _useSmallInt flag is disabled by default. Legacy IIdentityMap code is deleted. TypeMetadataWrapper is updated to use the new generic IdentityMap signatures. This refactor improves type safety, eliminates boxing, and prepares the code for value-type scenarios.
2026-02-01 10:02:48 +01:00