Introduce diagnostic and test infrastructure for SignalR binary protocol serialization/deserialization, including:
- ProtocolRoundTripDiagnosticTest for isolated protocol byte inspection
- TestMultiSegmentProtocol to exercise multi-segment buffer parsing
- TestInvocationBinder for correct parameter type binding
- Updates to TestableSignalRClient2 and TestableSignalRHub2 to route all messages through protocol round-trip
- Enhanced SendMessageToClient to simulate real SignalR transport
- Clarified SequenceBinaryInput segment handling logic
- Made TryParseMessage virtual in AcBinaryHubProtocol for testability
These changes improve test coverage for cross-boundary and multi-segment scenarios in SignalR message handling.
- Rewrote SequenceBinaryInput for lazy TryGet iteration (no segment array allocation), zero-copy access to segment backing arrays, and efficient cross-boundary reads using a reusable ArrayPool scratch buffer.
- Added Release() to IBinaryInputBase; now always called after deserialization to return scratch buffer.
- BufferWriterChunkSize is now mutable; set to 4096 for SignalR protocol for better pipe alignment.
- Added and updated documentation: detailed input buffer lifecycle, cross-boundary handling, and new BINARY_ISSUES.md and SIGNALR_ISSUES.md for known limitations and planned optimizations.
- No breaking API changes; improves performance, memory usage, and diagnostics for multi-segment binary deserialization.
- Change OnReceiveMessage signature to use `object data` (was `SignalData`), enabling type-aware and raw byte[] payloads.
- Implement three-path argument deserialization: byte[] fast-path, IsRawBytesData, and eager typed deserialization via SignalDataType.
- Add SignalDataType and IsRawBytesData fields to SignalParams for protocol guidance.
- Write path now uses AcBinarySerializer zero-copy to pipe; byte[] uses fast-path.
- SequenceBinaryInput now dynamically sizes scratch buffer for large cross-segment reads.
- Deserializer now advances segments before throwing end-of-buffer, improving multi-segment support.
- Set client logging to Debug for better diagnostics.
- Update all docs and markdown to reflect new protocol, dispatch model, and field semantics.
- AyCodeBinaryHubProtocol is now an empty derived class for registration/future hooks; SignalData is no longer the primary payload type.
- SignalResponseDataMessage is now an internal DTO with RawResponseData as object? (typed or byte[]), and GetResponseData<T>() is a direct cast.
Implements a high-performance raw byte[] protocol path for SignalR DataSource GetAll/LoadDataSource, using a new IsRawBytesData flag in SignalParams. When enabled, the server pre-serializes response data and sends it as a byte array, which the protocol passes through without further (de)serialization. The client receives the raw bytes and deserializes as needed, avoiding double serialization/deserialization and improving performance for large payloads.
Adds SerializerType selection to DataSource, propagates SignalParams through hub and protocol layers, and updates client/server/test code to support the new path. Also includes diagnostics flags for binary serialization debugging and fixes for multi-segment buffer handling.
Major overhaul for SignalR response pipeline:
- All deserialization now uses byte[] (offset/length) for zero-copy, allocation-free operation; all span/memory overloads removed.
- SignalR protocol sends (signalParams, object) directly; SignalData envelope and related logic removed.
- Server sets SignalParams.SignalDataType so protocol deserializes to the correct runtime type on the client.
- SignalResponseDataMessage now only used for client request/response tracking and stream path; RawResponseData holds the actual object.
- All extension methods, helpers, and infrastructure updated to use new byte[]-based APIs.
- AcSignalRDataSource and all test/benchmark code updated for new object flow.
- Removes all diagnostics, logging, and error handling related to binary envelopes.
- Enables true zero-copy, type-safe, allocation-free SignalR response handling.
Major protocol refactor: all byte[] payloads in SignalR hub/client interfaces, plumbing, and DTOs are now wrapped in SignalData, a disposable, ArrayPool-backed type with Span access. Introduces AyCodeBinaryHubProtocol (derived from AcBinaryHubProtocol) to rent pooled buffers for SignalData on receive. All message signatures, diagnostics, and serialization logic updated. Documentation and tests revised to reflect SignalData usage. Enables zero-copy, low-GC, high-performance binary messaging for large payloads.
Major protocol/API change: replace SignalReceiveParams with SignalParams everywhere. SignalParams now carries packed method parameters as a single byte[] and provides SetParameterValues/GetParameterValues for type-safe packing/unpacking. All hub/client interfaces, method signatures, and dispatch logic updated. Legacy parameter serialization helpers removed; all parameter logic is encapsulated in SignalParams. Documentation and tests updated to reflect new wire format and flow. This unifies parameter handling, clarifies the protocol, and enables robust, extensible type-guided serialization. Breaking change.
Replaces legacy JSON-in-Binary parameter envelopes with a length-prefixed, per-parameter binary format for all client→server calls. Introduces SerializeParametersToBinary and DeserializeParametersFromBinary for type-safe, zero-copy parameter handling. Marks IdMessage, SignalPostJsonDataMessage, and related wrappers as obsolete. Updates all client CRUD/messaging helpers and server-side dispatch to use object[] binary serialization. Adds DataSerializerType to SignalReceiveParams for response format indication. Updates tests and documentation to reflect new protocol. BREAKING CHANGE: not compatible with previous JSON-in-Binary clients/servers.
Major protocol update: OnReceiveMessage now takes metadata (SignalReceiveParams) and payload (byte[]) as separate hub arguments, not a single envelope. Metadata is AcBinary-serialized; payload uses protocol fast-path. Updated all client/server code, interfaces, and docs. Added ISignalParams and SignalReceiveParams types. Improved AcBinaryHubProtocol diagnostics and made byte[] fast-path more robust. This enables clearer, more debuggable, and future-proof SignalR binary messaging.
Removed OnReceiveStreamMessage from both AcSignalRClientBase and IAcSignalRHubBase, eliminating client-to-server streaming support. Updated StreamAsync calls to use the explicit method name string. This change enforces server-to-client streaming only and cleans up related interface and method references.
- Cache wrapper/metadata in serialization bridge methods to avoid redundant GetType and GetWrapper calls, improving performance.
- Update source generator to combine null/depth checks and cache depthExceeded for collections.
- Add useAcBinaryProtocol option to AcSignalRClientBase, allowing binary protocol to be toggled via constructor and registered via DI.
- Update documentation to reflect new caching rules and performance improvements.
Refactored AcBinarySerializer to add a source-generated (SGen) root fast path, bypassing the full runtime dispatch chain for SGen-decorated types and improving performance. Introduced helper methods for context pooling and buffer management. All entry points now use these helpers for consistency. Added comprehensive SGen architecture documentation (BINARY_SGEN.md), updated all related docs to explain the hybrid model, bridge methods, and configuration. Clarified doc loading rules in copilot-instructions.md for strict doc-first enforcement.
- Add TypeName to TestDataSet for clearer test scenario reporting
- Display serializer options in console and log outputs
- Extend BenchmarkResult with OptionsDescription
- Serializer benchmarks now provide detailed config summaries
- Log files now include test type and serializer options summary
- Generate .LLM Markdown results for LLM consumption and docs
- Reference .LLM results in BINARY_IMPLEMENTATION.md for visibility
Major documentation overhaul for binary serialization and SignalR:
- Simplified and clarified AyCode.Core README, with direct links to new deep-dive docs (BINARY_IMPLEMENTATION.md, BINARY_WRITERS.md)
- Added BINARY_WRITERS.md: detailed design and rationale for ArrayBinaryOutput and BufferWriterBinaryOutput, chunk sizing, and buffer management
- Refined BINARY_IMPLEMENTATION.md: clearer buffer management, output strategies, and hot-path rules; references new writers doc
- Added SIGNALR_BINARY_PROTOCOL.md: full wire format, zero-copy pipeline, dual BWO pattern, and read path for custom SignalR protocol
- Updated SignalRs README and SIGNALR.md: clarified protocol, tag system, request/response flow, and technical debt
- Improved cross-linking and discoverability throughout
These changes make the technical documentation clearer, more maintainable, and easier to navigate for advanced contributors.
- Add BINARY_IMPLEMENTATION.md with internal architecture and perf details
- Link new implementation doc from all relevant documentation
- Change default ReferenceHandlingMode to OnlyId
- Change default UseStringInterning to All
- Add InternalsVisibleTo for Mango.Nop.Core
- Added "AI AGENT CORE PROTOCOL" to all copilot-instructions.md files: mandates [LOADED_DOCS] prefix, hard-gates tool usage, enforces no-re-read of .md files, and requires user consent for doc/code changes.
- Updated CLAUDE.md to require reading copilot-instructions.md first.
- Added topic-based doc separation and folder navigation rules.
- Changed doc sync: agent now passively detects discrepancies and asks before updating docs.
- Every code-modifying response must end with a [DOCUMENTATION CHECK] section.
- Centralized measurement system and domain traps in new FruitBank.Common/docs/GLOSSARY.md; updated references in FruitBankHybridApp GLOSSARY.md.
- Clarified schema and doc locations in FruitBankHybridApp README.md.
- Added hybrid execution model section to AyCode.Core BINARY_FEATURES.md.
- Removed unnecessary BeginUpdate/EndUpdate calls in MgGridBase.cs for layout persistence.
- Removed full Toon schema from plugin SCHEMA.md to avoid duplication.
- CLAUDE.md: reduced to single-line pointer to copilot-instructions.md (eliminates redundant auto-loaded content)
- copilot-instructions.md: added @repo name field, relative paths in own-dep-repos, "do not re-read .md files" rule, and explicit permission to navigate external repos
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Refined guidance on avoiding redundant code by focusing on the current context rather than broad searches. Updated .md file sync instructions to prioritize code as the source of truth, clarify when and how to update documentation, and outline review practices. Improved wording for clarity and alignment with actual workflows.
Added standardized metadata blocks to the top of each project and repo documentation file. These blocks define project type, and for repos, include layer and dependencies, improving clarity and enabling better tooling support.
Replaced Markdown links with plain code-style paths in all README.md files for consistency. Updated references to logging, SignalR, and dynamic method dispatch documentation. Clarified some documentation paths and improved consistency in context/architecture sections. No code changes—documentation only.
Replaced all Markdown link syntax in documentation with inline code formatting and consistent relative paths. Updated all cross-references and tables for clarity and uniformity. No code or logic changes; docs only.
- Move all major feature docs (logging, binary, SignalR, DataSource) into per-project `docs/` folders with dedicated Markdown files.
- Split monolithic docs into focused files: `BINARY_FORMAT.md`, `BINARY_FEATURES.md`, `BINARY_OPTIONS.md`, `LOGGING.md`, `LOGGING_SERVER.md`, `LOGGING_REMOTE.md`, `SIGNALR.md`, `SIGNALR_SERVER.md`, `SIGNALR_DATASOURCE.md`.
- Update all references in `README.md`, `copilot-instructions.md`, `GLOSSARY.md`, and `ARCHITECTURE.md` to point to the correct per-project doc.
- Add documentation tables to each project’s `README.md` and clarify folder structure.
- Update `.csproj` files to include `docs/**/*.md` for packaging.
- Remove obsolete/moved docs from the solution file.
- Ensure all technical debt warnings and cross-references are preserved and accurate.
- No code changes; documentation only.
- Added SIGNALR.md (transport) and SIGNALR_DATASOURCE.md (collection) as layered, comprehensive documentation; retired SIGNALR_ARCHITECTURE.md
- Updated all .md files and READMEs to reference new docs and clarify separation between transport and DataSource
- Clarified CRUD tag structure (5 independent tags), single-method tag-based dispatch, and JSON-in-Binary tech debt
- Added slot allocation and wire format clarifications to serialization docs
- Improved documentation layering, conventions, and critical warnings for future maintainers
Added comprehensive docs for SignalR tag-based dispatch (docs/SIGNALR_ARCHITECTURE.md), including message flow, tag system, dynamic method registry, and tech debt (JSON-in-Binary). Updated all related READMEs, glossaries, and conventions to reference this architecture and clarify grid infrastructure (MgGridBase, FruitBankGridBase) and external dependency locations (AyCode.Core, AyCode.Blazor, Mango.Nop Libraries, FruitBank Plugin). Synchronized solution items and copilot-instructions. Improves discoverability, enforces conventions, and clarifies tech debt for all developers.
- Add BINARY_FORMAT.md: full AcBinary wire format spec (markers, encoding, options, protocol, interactions)
- Reference BINARY_FORMAT.md from GLOSSARY.md, Binaries/README.md, and Serializers/Binaries/README.md; add new glossary terms
- Clarify and expand config options tables to match new doc
- Add/clarify LLM maintenance rules: always sync .md files with code, auto-fix discrepancies
- Update root README.md: AyCode.Core targets .NET 9, not 10; stress doc/code sync
- Add code reuse and doc sync conventions to copilot-instructions.md and CONVENTIONS.md
- Add docs/ folder and BINARY_FORMAT.md to solution as Solution Items
- Minor clarifications and cross-links in ARCHITECTURE.md and other docs
- Introduced `.github/copilot-instructions.md` as the single source of truth for domain rules, conventions, and pitfalls in each solution.
- Added `CLAUDE.md` to guide Claude to read domain rules, glossary, and README before code generation.
- Updated all solution and project `README.md` files to document project purpose, structure, key files, and LLM context (Copilot/Claude/Cursor).
- Added or revised `docs/ARCHITECTURE.md`, `docs/CONVENTIONS.md`, and `docs/GLOSSARY.md` to clarify dependency graphs, naming, patterns, and terminology.
- For FruitBankHybridApp, added `docs/SCHEMA.md` (Toon format) and expanded the glossary with business/measurement terms and common traps.
- Updated all subfolder READMEs to list key files, conventions, and LLM maintenance notes.
- Ensured all documentation is cross-referenced, up-to-date, and includes explicit instructions for LLMs to keep docs in sync with code and avoid suggesting removal/rollback as a solution.
- Standardized documentation and onboarding for maintainability and LLM/code quality across all solutions.
Added comprehensive README.md documentation to every project and subfolder in the solution. Each README describes the purpose, key files, structure, dependencies, and design patterns for its area. This improves codebase navigability and maintainability, and includes a maintenance note to keep docs in sync with future code changes.
Added detailed README files for the root AyCode.Core library and each major subfolder (Compression, Consts, Enums, Extensions, Helpers, Interfaces, Loggers). Each README outlines folder purpose, key files, APIs, and critical notes, with tables and architecture diagrams where relevant. Included explicit instructions for LLM-based maintenance. This greatly improves project documentation and developer onboarding.
Updated test and benchmark project dependencies to latest versions (BenchmarkDotNet, coverlet.collector, MSTest, etc.). Reformatted source generator project references for consistency. Commented out most AcBinary benchmark variants in Program.cs, leaving only FastMode+Default enabled. Significantly expanded README with detailed architecture, file descriptions, and configuration for each serializer subfolder. Added LLM maintenance notes to ensure documentation stays in sync with code changes.
Added HasScanWork and related methods to determine at compile time if a property requires scan work. EmitScanProp now skips code generation for properties proven to not need scan logic, improving efficiency and reducing unnecessary code output.
Major refactor of binary serialization codegen and runtime:
- Added property writer bridge methods for markerless/metadata paths
- Centralized object marker logic via new bridge methods
- Simplified SGen output: single bridge call replaces branching
- FixObj slot markers now supported in serialization/deserialization
- Refactored collection/dictionary element serialization
- Removed redundant WritePropertyMarkerless method
- Improved tests: use BinaryTypeCode constants, FixObj parsing
- Added InternalsVisibleTo for test project access
- Annotated TestSimpleClass for SGen support
Reduces generated code size, improves maintainability, and ensures correct handling of new binary format features.
Split WriteObjectProperties into markerless and metadata variants for clarity and performance. Adjust method inlining attributes to favor hot path optimization. Comment out WritePropertyValue and some AcBinaryBenchmark variants to streamline code and benchmarks. Improves maintainability and serialization efficiency.
Split hot/cold paths for polymorphic and reference-tracked object serialization. Introduce dedicated cold-path methods for polymorphic and IId reference handling, and new helpers for writing reference indices. Extract property writing loop for reuse. Simplify WriteObject and update property dispatch logic to reduce branching and clarify marker handling.
Refactored BinaryTypeCode to reserve 0..63 for FixObj slot indices, enabling direct array access for object wrappers. Introduced a new polymorphic type prefix system for properties whose runtime type differs from their declared type, with first/repeated occurrence markers and combined ref-tracking support. Unified wrapper slot caching for SGen and runtime types, improving performance and eliminating dictionary lookups in hot paths. Updated code generation, tests, and constants to use the new slot system. Added new settings and utility scripts. Overall, serialization is now faster, more robust, and extensible.
Changed default WireMode to Compact and string interning to Attribute in AcBinarySerializerOptions. Added Bash commands in settings.local.json for fetching and processing Cysharp/MemoryPack files via GitHub API and curl/python scripts.
Refactored property enumeration in AcBinarySourceGenerator to match runtime ordering and filtering using a new helper. Null checks for reference types are now unconditional in generated code. Changed default string interning mode to All. Added InternalsVisibleTo for FruitBank.Common. Writer attribute checks now only apply to source-defined types.
- Introduced IEntityComment interface for entity comments.
- Changed SignalR client message serialization to binary.
- Updated SignalResponseDataMessage docs for GZip (was Brotli).
- Refactored GetResponseData<T> for GZip decompression and improved error handling.
- Added necessary using statements for new interface and features.
- Changed default SignalR message serialization from JSON to binary.
- Updated AcSerializerType enum values: Binary=0, Json=1.
- Disabled string caching when intern table is present.
- Replaced Brotli with GZip for JSON compression in comments and logic.
- Refactored SignalResponseDataMessage deserialization for improved error handling.
Activate FastWire encoding for both serialization and deserialization, using UTF-16 and fixed-width lengths for strings. WireMode is now public and defaults to FastWire. Removes unused ObjectEnd marker, clarifying object end handling. This improves string (de)serialization speed at the cost of larger output.
Introduce ReadProperties to IGeneratedBinaryReader and generated readers, enabling property population into existing instances. Refactor ReadObject to delegate to ReadProperties. Update codegen for complex objects, collections, and dictionaries to use inline property deserialization. Improves efficiency and flexibility for parent-driven instance creation and cache management. Update documentation and comments to reflect new pattern.
Refactor serialization context to use precomputed boolean flags
(HasRefHandling, HasAllRefHandling, HasStringInterning) for faster
reference and string interning checks, replacing repeated enum
comparisons. Update source generator to emit code using these flags.
Add AcBinarySerializer.ScanOnly for isolated scan benchmarking.
Set MaxDepth in test options. Improves performance and maintainability.
Switch from caching next entry object to caching VisitIndex for write plan tracking in binary serialization. Updates TryConsumeWritePlanEntry and related logic for improved simplicity and performance. Comments revised to match new approach.
Pre-compute InternBit and reference handling flags in context to avoid repeated field access and shifting. Refactor all intern mode checks to use InternBit, improving performance and code clarity in generated code, property accessors, and scan/write passes. Optimize write plan entry access for faster serialization.
Refactored EmitReadString to use a switch for O(1) dispatch of string wire formats, improving performance and clarity. Updated comments to document the new approach and its benefits. Removed the UseGeneratedCode property from AcBinarySerializerOptions.
Optimized AcBinarySourceGenerator to emit direct code for dictionary property serialization and scanning. Added PropInfo fields for dictionary value type metadata, scan requirements, and FNV-1a hashes. Specialized code now handles string interning, complex value reference tracking, and metadata inline, reducing runtime overhead and improving performance. Fallback to runtime methods only for unsupported types.
Enable optimized codegen for Dictionary<TKey, TValue> and IDictionary<TKey, TValue> properties. The generator now analyzes key/value types and emits direct binary read loops for primitive, string, enum, and complex types with generated readers. PropInfo is extended with dictionary metadata. Improves performance and type safety by eliminating reflection and runtime type dispatch for supported dictionary types. Also refactors collection reading logic and updates dependency graph and scanning for dictionary value types.
Added compile-time cycle detection for [AcBinarySerializable] types in AcBinarySourceGenerator, reporting ACBIN001 warnings to guide reference handling. Increased depth increment for nested serialization to improve max depth enforcement. Refactored collection element depth checks for clarity. Updated tests to conditionally assign Parent and commented out a redundant assertion. Simplified slot bounds check in AcSerializerContextBase.
Detect circular references at compile time in source gen
Added DFS-based cycle detection to AcBinarySourceGenerator, reporting ACBIN001 warnings for circular reference chains among [AcBinarySerializable] types. Increased depth increment for nested serialization from +1 to +2 to improve handling of deep/circular structures. Adjusted null-check logic for collections to respect MaxDepth. Updated tests to conditionally set circular references and commented out assertions for Parent when not applicable. Minor slot bounds check fix in AcSerializerContextBase.
Changed AcBinarySerializerOptions defaults: UseMetadata is now false, UseGeneratedCode is now true. Improved IId reference tests by adding UseMetadata option, new null and reference identity assertions, and refactored assertion order for clarity. These changes ensure more robust test coverage and align default serializer behavior with expected usage.
Loretta
Adam