Commit Graph

210 Commits

Author SHA1 Message Date
Loretta 762088caf7 Refactor and expand SignalR data source tests
Split SignalRDataSourceTests into multiple focused files and introduce an abstract base test class for comprehensive coverage. Add concrete test classes for all serializer and collection type combinations. Move data source implementations to separate files. Test coverage now includes edge cases, interface compliance, and advanced scenarios.
2025-12-15 19:28:20 +01:00
Loretta b8143e4897 Add FixStr encoding for short strings; SIMD bulk copy
Introduces FixStr encoding (type codes 34–65) for short ASCII/UTF8 strings up to 31 bytes, combining type and length in one byte for improved space and speed. Adds SIMD-optimized bulk copy methods for double, float, and Guid arrays. Updates deserializer to handle FixStr codes efficiently. Adjusts tiny int encoding range to free up FixStr space. Disables metadata and string interning in shallow copy options. Improves performance and reduces overhead for common serialization scenarios.
2025-12-15 17:21:18 +01:00
Loretta a832d8e86d Expand QuickBenchmark suite & add CLI scripts
Majorly enhanced QuickBenchmark.cs with new helper methods, standardized iteration count, and several comprehensive benchmarks comparing AcBinary (with/without reference handling) to MessagePack. Improved output formatting for clarity. Added RunQuickBenchmark.ps1 and .bat scripts for easy CLI execution and registered them as solution items. These changes make benchmarking more robust, readable, and user-friendly.
2025-12-15 12:00:03 +01:00
Loretta bc30a3aede Refactor: Add high-perf JSON serializer & merge support
- Introduced AcJsonSerializer/AcJsonDeserializer in AyCode.Core.Serializers.Jsons, optimized for IId<T> reference and circular reference handling.
- Added AcJsonSerializerOptions/AcSerializerOptions for configurable reference handling and max depth.
- Implemented fast-path streaming (Utf8JsonReader/Writer) with fallback to DOM for reference scenarios.
- Added type metadata/property accessor caching for performance.
- Provided robust object/collection population with merge semantics for IId<T> collections.
- Added AcJsonDeserializationException for detailed error reporting.
- Implemented UnifiedMergeContractResolver for Newtonsoft.Json, supporting JsonNoMergeCollectionAttribute to control merge behavior.
- Added IdAwareCollectionMergeConverter<TItem, TId> for merging IId<T> collections by ID.
- Included helpers for ID extraction and semantic ID generation.
- Added DeepPopulateWithMerge extension for deep merging.
- Optimized with frozen dictionaries, pre-encoded property names, and context pooling.
- Ensured compatibility with both System.Text.Json and Newtonsoft.Json.
2025-12-14 19:34:49 +01:00
Loretta b17c2df6c2 Add header-based string interning to AcBinarySerializer
- Support preloaded string intern table in binary header for efficient string interning and reduced output size.
- Add HeaderFlag_StringInternTable to BinaryTypeCode and update serializer/deserializer to handle intern tables.
- Simplify string interning logic: always intern eligible strings, remove candidate tracking.
- Refactor property name table and buffer management for clarity and efficiency.
- Remove obsolete interning/property name methods from serializer context.
- Add new output methods (ToArray, WriteTo, DetachResult) for buffer/result handling.
- Introduce QuickBenchmark.cs with benchmarks comparing AcBinarySerializer (with/without interning), JSON, and MessagePack, including repeated string scenarios.
2025-12-14 15:12:11 +01:00
Loretta 271f23d0f6 Enhance AcBinary: property filter, string interning, arrays
- Add property-level filtering via BinaryPropertyFilter delegate and context
- Improve string interning with new StringInternNew type code and promotion logic
- Optimize array and dictionary serialization for primitive types
- Expose strongly-typed property accessors for primitives and enums
- Add new benchmarks for serialization modes
- Refactor buffer pooling and cleanup code
- All new features are opt-in; maintains backward compatibility
2025-12-14 12:45:29 +01:00
Loretta 5601c0d3e2 Refactor serialization core, add pooled buffer support
Centralize serialization steps into a new SerializeCore method for consistency and maintainability. Rework property metadata registration to operate on types instead of instances, improving efficiency. Replace property index tracking with stack-allocated or pooled buffers to reduce allocations. Add SerializeToPooledBuffer and BinarySerializationResult for zero-copy serialization with proper buffer pooling and disposal. Simplify string writing logic and use GC.AllocateUninitializedArray for result arrays. Refactor and add helper methods for buffer management and metadata handling.
2025-12-14 10:20:07 +01:00
Loretta 3b5a895fbc Add BSON to benchmarks; optimize AcBinary deserializer
- Add BSON (MongoDB.Bson) as a serialization format in benchmarks, enabling direct comparison with AcBinary, MessagePack, and JSON.
- Update all AcBinary benchmarks to use options without reference handling for fair comparison.
- Show BSON results and ratios in benchmark output and size comparisons.
- Refactor AcBinaryDeserializer to use a fixed-size array for type reader dispatch, improving lookup speed and reducing allocations.
- Add a concurrent cache for type conversion info to optimize enum and nullable conversions.
- Use a cached UTF8Encoding instance for string decoding.
- Use FrozenDictionary for property lookup in BinaryDeserializeTypeMetadata.
- Remove legacy WithRef code paths and clean up formatting and comments.
- Improve error handling and fallback logic for BSON serialization/deserialization.
2025-12-14 04:47:16 +01:00
Loretta 74b4bbfd30 Add size_output.txt to .gitignore
Prevent size_output.txt from being tracked by Git by adding it to the .gitignore file. This helps keep the repository clean of generated or temporary files.
2025-12-14 04:46:54 +01:00
Loretta 2f1c00fd5c Remove MessagePack; unify SignalR serialization model
Major refactor to eliminate MessagePack from SignalR messaging.
All serialization now uses explicit binary methods (.ToBinary/.BinaryTo)
and Brotli-compressed JSON, managed via a new SignalRSerializationHelper.
Custom stream classes and MessagePack attributes are removed.
API is now consistent, type-safe, and easier to maintain.
Test code and all message handling updated to use the new model.
2025-12-14 01:45:17 +01:00
Loretta 489ef7486c Optimize JSON/Brotli serialization for zero-allocation
Major performance improvements for SignalR message transport:
- BrotliHelper now uses ArrayPool and stackalloc for zero-allocation compression/decompression; added span/IBufferWriter overloads and pooled buffer support.
- AcJsonDeserializer supports direct deserialization from UTF-8 spans, with fast path for no reference handling.
- SignalResponseDataMessage uses pooled buffers for Brotli decompression and zero-copy deserialization; implements IDisposable for buffer return.
- IdMessage serialization optimized for primitives and Guids to avoid unnecessary allocations.
- Added JsonTo<T> span/byte[] extension methods for zero-allocation deserialization.
- All changes are backward compatible and reduce GC pressure for high-throughput scenarios.
2025-12-13 23:23:16 +01:00
Loretta ac6735ebd8 Unify SignalR response serialization (JSON/Binary/Brotli)
Major refactor: all SignalR responses now use a single unified `SignalResponseDataMessage` type with binary payloads. JSON responses are Brotli-compressed for efficiency. Removed legacy JSON/Binary response types and MessagePack server-to-client serialization. Updated all serialization extensions for zero-allocation binary ops. Refactored client/server/data source/test code to use new message and serialization model. Improved deserialization robustness for primitives. Modernized and streamlined test suite.
2025-12-13 23:01:18 +01:00
Loretta 60238952d8 Rename BenchmarkSuite1 to AyCode.Benchmark project
Renamed the benchmark project from BenchmarkSuite1 to AyCode.Benchmark, updating the solution reference and moving all benchmark source files under the new project and namespace. No functional changes were made; this is a structural and naming reorganization for clarity and maintainability.
2025-12-13 10:11:39 +01:00
Loretta 9f1c31bd15 Centralize test/benchmark results; optimize deserializer
Introduce a unified Test_Benchmark_Results directory for all test, benchmark, and coverage artifacts, with MSBuild properties and MSTest runsettings for consistent output. Update .gitignore to exclude results. Refactor BenchmarkSuite1 to ensure all logs and artifacts are stored in versioned subfolders, and add logic for coverage file management.

Majorly optimize AcBinaryDeserializer: reuse existing nested objects and collections during deserialization, add PopulateListOptimized for in-place list updates, and use String.Create for efficient UTF8 decoding. Extend property metadata to track complex/collection types. Update MessagePack and BenchmarkDotNetDiagnosers package versions. Remove obsolete benchmark-report.html.
2025-12-13 09:59:18 +01:00
Loretta 056aae97a5 Optimize AcBinarySerializer: typed accessors & array writers
- Add typed property accessors to avoid boxing and speed up value type serialization
- Implement bulk array writers for primitives (int, long, double, etc.) for efficient, zero-copy serialization
- Add zero-copy IBufferWriter serialization and size estimation methods
- Refactor array/dictionary serialization for fast paths and memory efficiency
- Improve context pool memory management and reduce initial dictionary/set capacities
- Fix benchmark to avoid state accumulation between runs
- Downgrade MessagePack dependency for compatibility
2025-12-13 03:25:02 +01:00
Loretta f69b14c195 AcBinary: Major perf/memory optimizations & new benchmarks
- Zero-allocation hot paths for primitive (de)serialization using MemoryMarshal/Unsafe
- FrozenDictionary-based type dispatch for fast deserialization
- Optimized span-based UTF8 string handling, stackalloc for small strings
- Specialized fast-paths for primitive arrays (int, double, etc.)
- Binary header now uses flag-based format (48+) for metadata/ref handling
- Improved buffer management with ArrayPool and minimum size
- Property access via for-loops for better JIT and less overhead
- SignalR test infra supports full serializer options (WithRef/NoRef)
- Added comprehensive AcBinary vs MessagePack benchmarks (speed, memory, size)
- Added rich HTML benchmark report (benchmark-report.html)
- Updated JsonUtilities for new header detection
- Improved documentation and code comments throughout
2025-12-13 01:24:48 +01:00
Loretta 6faed09f9f Fix binary deserializer string interning and add regressions
- Fix: Ensure property names and skipped strings are interned during binary deserialization, preventing interned string index mismatches.
- Add: Extensive regression tests for string interning edge cases, including property mismatch (server has more properties than client), deeply nested objects, repeated/unique strings, and empty/null handling.
- Add: New test DTOs and infrastructure for property mismatch scenarios.
- Update: SignalR test helpers and services to support both JSON and Binary serialization in all tests.
- Improve: SignalRRequestModelPool now initializes models on Get().
- These changes address production bugs and ensure robust, consistent string interning and property skipping in AcBinarySerializer.
2025-12-13 00:12:21 +01:00
Loretta 1a9e760b68 Fix Binary serialization of string DateTimes in attributes
Add regression tests for string-stored DateTime values in GenericAttribute-like collections, ensuring exact value preservation in Binary serialization. Fix AcBinaryDeserializer to only intern strings >= 4 chars, matching serializer logic and preventing intern table index mismatches. Refactor SignalR tests to run under both JSON and Binary modes, and add comprehensive tests for edge cases and production bug scenarios. Clean up test code and comments for clarity.
2025-12-12 22:59:54 +01:00
Loretta 09a4604e52 Add binary serialization support for SignalR responses
Introduces SignalResponseBinaryMessage for efficient binary serialization of response data alongside existing JSON support. Adds utilities to detect serializer type and updates both server and client logic to handle JSON or binary formats automatically. Refactors response creation, logging, and deserialization for consistency and performance. Updates benchmarks and ensures all MessagePack operations use ContractlessStandardResolver.Options. Improves robustness and backward compatibility in client callback handling.
2025-12-12 21:40:48 +01:00
Loretta 2147d981db Refactor: new high-performance binary serializer/deserializer
Major overhaul of binary serialization:
- Rewrote AcBinarySerializer as a static, optimized, feature-rich serializer with VarInt encoding, string interning, property name tables, reference handling, and optional metadata.
- Added AcBinaryDeserializer with matching features, including merge/populate support and robust error handling.
- Introduced AcBinarySerializerOptions and AcSerializerOptions base class for unified serializer configuration (JSON/binary).
- Added generic extension methods for "any" serialization/deserialization based on options.
- Updated tests and benchmarks for new APIs; fixed null byte code and added DateTimeKind test.
- Fixed namespace typos and improved code style and documentation.
2025-12-12 21:03:39 +01:00
Loretta b9e83e2ef8 Add AcBinarySerializer tests, helpers, and benchmark updates
- Introduce AcBinarySerializerTests with full coverage for primitives, objects, collections, merge/populate, and size comparisons
- Add AcBinarySerializer class stub as a placeholder for implementation
- Extend serialization extension methods with binary helpers (ToBinary, BinaryTo, BinaryCloneTo, etc.)
- Update test models to ignore parent references for all major serializers ([IgnoreMember], [BsonIgnore])
- Refactor benchmarks: split into minimal, simple, complex, and MessagePack comparison; add command-line switches and improved size reporting
- Optimize AcJsonDeserializer with fast UTF-8 property lookup and direct primitive setting
- Add MessagePack and MongoDB.Bson dependencies to test and benchmark projects
- Add (accidentally) a summary of less commands as a documentation artifact
2025-12-12 20:06:00 +01:00
Loretta a945db9b09 High-perf streaming JSON (de)serialization, refactor
Major upgrade to AcJsonSerializer/AcJsonDeserializer:
- Add Utf8JsonReader/Writer fast-paths for streaming, allocation-free (de)serialization when reference handling is not needed, matching STJ performance.
- Populate/merge now uses streaming for in-place updates.
- Type metadata caches TypeCode, UnderlyingType, and uses frozen dictionaries for hot property lookup.
- Context pooling reduces allocations and GC pressure.
- Primitive (de)serialization uses TypeCode-based fast paths; improved enum, Guid, DateTime, etc. handling.
- Shared UTF8 buffer pool for efficient encoding/decoding.
- Pre-encoded property names and STJ writer for output.
- Improved validation, error handling, and double-serialization detection.
- Expanded benchmarks: small/medium/large, with/without refs, AyCode vs STJ vs Newtonsoft, grouped by scenario.
- General code modernization: aggressive inlining, ref params, ReferenceEquals, improved naming, and comments.
- Unified IId<T> and collection element detection; consistent $id/$ref handling.

Brings AyCode JSON (de)serializer to near-parity with STJ for non-ref scenarios, while retaining advanced features and improving maintainability and performance.
2025-12-12 16:23:54 +01:00
Loretta ad426feba4 Refactor JSON (de)serialization: options, depth, utilities
Major overhaul of JSON serialization/deserialization:
- Introduce AcJsonSerializerOptions for reference handling and max depth
- Centralize type checks, primitive/collection logic in JsonUtilities
- Add depth limiting to serializer/deserializer (MaxDepth support)
- Make $id/$ref reference handling optional via options
- Unify and simplify public API (ToJson, JsonTo, CloneTo, CopyTo, etc.)
- Improve primitive, enum, and collection handling and caching
- Refactor contract resolver and merge logic to use new utilities
- Remove redundant code, centralize string escaping/unescaping
- Update all tests and benchmarks to use new API and options
- Fix minor bugs and improve error handling and validation

This modernizes and unifies the JSON infrastructure for better performance, flexibility, and maintainability.
2025-12-12 11:30:55 +01:00
Loretta 8e7869b3da Improve JSON deserialization and observable collection handling
- Add batch update support for IAcObservableCollection during deserialization to suppress per-item notifications and fire a single reset event.
- Throw AcJsonDeserializationException with a clear message if a type lacks a parameterless constructor during deserialization.
- Enhance IsGenericCollectionType to recognize more collection types, including ObservableCollection<T> and custom IList<T> implementations.
- Improve array and collection deserialization logic to better handle target types and fall back to List<T> if needed.
- In AcObservableCollection, catch and ignore ObjectDisposedException in event handlers to prevent errors from disposed subscribers.
- Remove redundant batch update logic from AcSignalRDataSource; rely on deserializer's handling.
- Set SkipNegotiation = true in SignalR client options for WebSockets-only optimization and comment out automatic HubConnection.StartAsync() for more controlled connection management.
2025-12-11 23:46:30 +01:00
Loretta c29b3daa0e Refactor JSON/SignalR infra; add full test & benchmark suite
Major overhaul of JSON serialization/deserialization:
- Added AcJsonDeserializationException for clearer error reporting.
- Robust JSON validation, type checking, and double-serialization detection.
- Fast-path primitive (de)serialization for all common .NET types.
- Direct support for Dictionary<TKey, TValue> and improved collection handling.
- Enhanced $id/$ref reference resolution and in-place merging.
- Optimized property metadata caching and filtering.
- Comprehensive error handling with detailed messages.
- Extensive new tests for primitives, collections, references, and error cases.
- Benchmarks for serialization, deserialization, and merge scenarios.

SignalR infrastructure improvements:
- Refactored AcSignalRClientBase/AcWebSignalRHubBase for testability (virtual methods, test constructors).
- Added SignalRRequestModelPool for efficient request/response tracking.
- Improved parameter deserialization and IdMessage handling.
- New tags (PingTag, EchoTag) for SignalR messaging.

New test and benchmark infrastructure for SignalR:
- Shared test models and data factories for DTOs, primitives, and complex graphs.
- In-memory, dependency-free SignalR client/hub for round-trip testing.
- Exhaustive test suites for message processing, method invocation, and edge cases (including prior production bugs with Task<T>).
- Benchmarks for SignalR serialization, deserialization, and round-trip performance.

Other changes:
- Improved TaskHelper.WaitTo/WaitToAsync with more accurate polling and cancellation support.
- ExtensionMethods.InvokeMethod now properly unwraps Task/ValueTask results.
- General code cleanup, improved comments, and removal of obsolete code.
- Project references updated for shared test/benchmark infrastructure.

These changes deliver a robust, high-performance, and fully tested JSON/SignalR (de)serialization system, ready for production and advanced testing scenarios.
2025-12-11 21:25:50 +01:00
Loretta 5abff05031 lot of improvements and new mstests... 2025-12-11 21:24:32 +01:00
Loretta a0445e6d1e Improve JSON (de)serializer: WASM, SignalR, perf, tests
- Use compiled expression tree accessors for property get/set (AOT/WASM compatible, much faster than reflection)
- Add comprehensive WASM/AOT compatibility and SignalR parameter array tests
- Correctly handle $id/$ref for shared references; optimize reference resolution
- Always serialize empty collections as [], omit null collections
- Optimize primitive reading and type metadata caching
- Fix edge cases in array, primitive, and reference deserialization
- Ensure output matches Newtonsoft.Json for arrays and primitives
- Greatly expand test coverage for all major scenarios
2025-12-09 11:26:55 +01:00
Loretta f9dc9a65fb High-performance, thread-safe JSON and data source overhaul
- Introduced AcJsonSerializer/Deserializer: fast, reflection-free, streaming JSON with optimized $id/$ref handling and Base62 IDs.
- Default serialization now uses new serializers; falls back to Newtonsoft for complex cases.
- Extensive type/property caching for performance and thread safety.
- Refactored MergeContractResolver and collection merge logic; all merge/populate operations use centralized caches.
- AcObservableCollection and AcSignalRDataSource are now fully thread-safe and support batch operations.
- SignalResponseMessage<T> supports lazy deserialization and direct JSON access.
- Added comprehensive unit tests and benchmarks for serialization, deserialization, and collection operations.
- Updated .gitignore and solution files; refactored core classes for clarity and performance.
2025-12-09 03:24:51 +01:00
Loretta 166d97106d Enhance JSON handling and add hybrid reference support
- Updated all projects to use `Newtonsoft.Json` v13.0.3 for consistency.
- Introduced `HybridReferenceResolver` for semantic and numeric ID handling.
- Refactored `SerializeObjectExtensions` to support deep JSON merging.
- Simplified `IId<T>` interface by removing `IEquatable<T>` constraint.
- Improved `AcSignalRDataSource` with robust `AddRange` and `CopyTo` methods.
- Added `JsonExtensionTests` for deep hierarchy, reference, and edge cases.
- Implemented `UnifiedMergeContractResolver` for custom JSON behavior.
- Optimized type/property caching with `TypeCache` and `CachedPropertyInfo`.
- Enhanced SignalR integration to fix primitive array deserialization issues.
- Introduced `JsonNoMergeCollection` attribute for replace-only collections.
- Added test DTOs and `TestDataFactory` for real-world scenario simulations.
- Improved performance with `ConcurrentDictionary` and `ObjectPool`.
- Fixed `$id`/`$ref` handling for non-semantic references and arrays.
2025-12-08 15:50:48 +01:00
Loretta f3ec941774 microsoft packages update 2025-12-01 16:18:36 +01:00
Loretta da5ba340f7 ... 2025-11-26 10:14:56 +01:00
Loretta 2ecc7b1a7a improvements, fixes 2025-11-26 09:42:00 +01:00
Loretta 1131b5675b improvements, fixes, etc... 2025-11-24 08:26:55 +01:00
Loretta a67bd4f698 improvements 2025-11-21 16:29:05 +01:00
Loretta a724fce2f6 improvements 2025-11-21 07:20:35 +01:00
Loretta e0666027b3 improvements, fixes 2025-11-20 08:30:37 +01:00
Loretta dd5dc68862 TransportSendTimeout fix 2025-11-16 19:21:24 +01:00
Loretta 545b543abe fixes 2025-11-12 17:19:45 +01:00
Loretta bb3cc3c37c CollectionExtensions UpdateCollectionById<TId> 2025-11-07 14:10:30 +01:00
Loretta c1a707139c Implement IgnoreAndRenamePropertySerializerContractResolver 2025-10-31 13:51:39 +01:00
Loretta a24f0c1681 SignalR improvements; 2025-10-30 14:55:47 +01:00
Loretta 6f4f87ddd9 Logger improvements 2025-10-23 06:35:32 +02:00
Loretta ecdd922be2 AcLoggerBase get AppConfiguration fix 2025-10-22 06:46:30 +02:00
Loretta 0ee715ed97 improvements 2025-10-21 15:15:15 +02:00
Loretta a56f96903e nuget packages update 2025-10-20 16:45:31 +02:00
Loretta 981f86c701 improvements, fixes, etc... 2025-10-18 08:43:57 +02:00
Loretta f26f04eec5 fixes 2025-10-12 18:07:53 +02:00
Loretta e865a0535a CopyPublicValueTypeProperties fixes 2025-10-12 07:47:39 +02:00
Loretta 7609e94f18 PrpertyHelper; ICustomForeignKey 2025-10-11 17:51:58 +02:00
Loretta 9657e7449f CollectionExtensionsInt 2025-10-06 07:46:17 +02:00