Adam
/
AyCode.Core
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Refactor: Split and expand project-level documentation 

- Move all major feature docs (logging, binary, SignalR, DataSource) into per-project `docs/` folders with dedicated Markdown files.
- Split monolithic docs into focused files: `BINARY_FORMAT.md`, `BINARY_FEATURES.md`, `BINARY_OPTIONS.md`, `LOGGING.md`, `LOGGING_SERVER.md`, `LOGGING_REMOTE.md`, `SIGNALR.md`, `SIGNALR_SERVER.md`, `SIGNALR_DATASOURCE.md`.
- Update all references in `README.md`, `copilot-instructions.md`, `GLOSSARY.md`, and `ARCHITECTURE.md` to point to the correct per-project doc.
- Add documentation tables to each project’s `README.md` and clarify folder structure.
- Update `.csproj` files to include `docs/**/*.md` for packaging.
- Remove obsolete/moved docs from the solution file.
- Ensure all technical debt warnings and cross-references are preserved and accurate.
- No code changes; documentation only.
This commit is contained in:
Loretta 2026-03-29 22:16:28 +02:00
parent 03d606164c
commit ffd537b5eb
31 changed files with 1014 additions and 589 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/copilot-instructions.md vendored
View File

@ -15,7 +15,7 @@
## SignalR ## SignalR
7. **Single-method transport** — all SignalR communication uses `OnReceiveMessage(tag, bytes, requestId)`. Tags are `int` constants resolved via `DynamicMethodRegistry`. Never add conventional hub methods. 7. **Single-method transport** — all SignalR communication uses `OnReceiveMessage(tag, bytes, requestId)`. Tags are `int` constants resolved via `DynamicMethodRegistry`. Never add conventional hub methods.
8. **AcSignalRDataSource** — generic `IList<T>` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `docs/SIGNALR_DATASOURCE.md`. Transport docs: `docs/SIGNALR.md`. 8. **AcSignalRDataSource** — generic `IList<T>` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. Transport docs: `AyCode.Services/docs/SIGNALR.md`.
9. **JSON-in-Binary tech debt** — client→server request parameters are currently JSON inside a Binary envelope (`SignalPostJsonDataMessage`). Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects. 9. **JSON-in-Binary tech debt** — client→server request parameters are currently JSON inside a Binary envelope (`SignalPostJsonDataMessage`). Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects.
## Critical Warnings ## Critical Warnings

4
AyCode.Core.Server/AyCode.Core.Server.csproj
View File

@ -15,4 +15,8 @@
<ProjectReference Include="..\AyCode.Core\AyCode.Core.csproj" /> <ProjectReference Include="..\AyCode.Core\AyCode.Core.csproj" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<None Include="docs\**\*.md" />
</ItemGroup>
</Project> </Project>

6
AyCode.Core.Server/Loggers/README.md
View File

@ -1,7 +1,9 @@
# Loggers # Loggers
Provides a singleton `GlobalLogger` for application-wide logging with multiple severity levels (Detail, Debug, Info, Warning, Suggest, Error) and support for pluggable log writers. Server-side singleton logger for static access across the application.
> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). For core logger and writer abstractions see [`AyCode.Core/Loggers/README.md`](../../AyCode.Core/Loggers/README.md).
## Key Files ## Key Files
- **`GlobalLogger.cs`** — Singleton static logger that delegates to `AcLoggerBase`. Supports category names, caller member tracking, and configurable `LogLevel` and `AppType`. - **`GlobalLogger.cs`** — Singleton static wrapper around an internal `AcGlobalLoggerBase` (sealed `AcLoggerBase` subclass). Provides static methods for all log levels (`Detail`, `Debug`, `Info`, `Warning`, `Suggest`, `Error`, `Write`). Default category: `"GLOBAL_LOGGER"`. Reads config from `appsettings.json` like any `AcLoggerBase`. Exposes `GetWriters` and `Writer<T>()` for accessing specific writer instances.

6
AyCode.Core.Server/README.md
View File

@ -2,6 +2,12 @@
Server-side extension of AyCode.Core. Provides server-specific implementations that build on the shared core library. Server-side extension of AyCode.Core. Provides server-specific implementations that build on the shared core library.
## Documentation
| Document | Topic |
|---|---|
| [LOGGING_SERVER.md](docs/LOGGING_SERVER.md) | GlobalLogger singleton, server-side logging |
## Folder Structure ## Folder Structure
| Folder | Purpose | | Folder | Purpose |

23
AyCode.Core.Server/docs/LOGGING_SERVER.md Normal file
View File

@ -0,0 +1,23 @@
# Server Logging
Server-side logging extensions. For core framework (base classes, configuration, LogLevel, ILogger bridge) see [`AyCode.Core/docs/LOGGING.md`](../../AyCode.Core/docs/LOGGING.md). For remote writers (HTTP, browser, SignalR) see [`AyCode.Services/docs/LOGGING_REMOTE.md`](../../AyCode.Services/docs/LOGGING_REMOTE.md).
## GlobalLogger
Server-side singleton for static access. Wraps an internal `AcGlobalLoggerBase` instance (sealed `AcLoggerBase` subclass):
```csharp
GlobalLogger.Info("Server started");
GlobalLogger.Error("Failed to process", ex, "MyCategory");
GlobalLogger.Writer<IAcConsoleLogWriter>()?.Suggest("hint");
```
Default category: `"GLOBAL_LOGGER"`. Reads config from `appsettings.json` like any other `AcLoggerBase` instance.
All static methods mirror the `IAcLogWriterBase` contract: `Detail`, `Debug`, `Info`, `Warning`, `Suggest`, `Error`, `Write`.
## Key Source Files
| Component | Path |
|-----------|------|
| GlobalLogger | `Loggers/GlobalLogger.cs` |

2
AyCode.Core.sln
View File

@ -60,10 +60,8 @@ EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{D4B2E9F1-A6C3-4F7E-8D5B-3E2A1C4F6B8D}" Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{D4B2E9F1-A6C3-4F7E-8D5B-3E2A1C4F6B8D}"
ProjectSection(SolutionItems) = preProject ProjectSection(SolutionItems) = preProject
docs\ARCHITECTURE.md = docs\ARCHITECTURE.md docs\ARCHITECTURE.md = docs\ARCHITECTURE.md
docs\BINARY_FORMAT.md = docs\BINARY_FORMAT.md
docs\CONVENTIONS.md = docs\CONVENTIONS.md docs\CONVENTIONS.md = docs\CONVENTIONS.md
docs\GLOSSARY.md = docs\GLOSSARY.md docs\GLOSSARY.md = docs\GLOSSARY.md
docs\SIGNALR_ARCHITECTURE.md = docs\SIGNALR_ARCHITECTURE.md
EndProjectSection EndProjectSection
EndProject EndProject
Global Global

4
AyCode.Core/AyCode.Core.csproj
View File

@ -24,6 +24,10 @@
<Folder Include="Expressions\" /> <Folder Include="Expressions\" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<None Include="docs\**\*.md" />
</ItemGroup>
<ItemGroup> <ItemGroup>
<InternalsVisibleTo Include="AyCode.Core.Serializers.Console" /> <InternalsVisibleTo Include="AyCode.Core.Serializers.Console" />
<InternalsVisibleTo Include="AyCode.Core.Tests" /> <InternalsVisibleTo Include="AyCode.Core.Tests" />

35
AyCode.Core/Loggers/README.md
View File

@ -1,34 +1,39 @@
# Loggers # Loggers
Custom logging framework with `Microsoft.Extensions.Logging` integration. Supports multiple output writers, colored console output, and structured log formatting. Custom logging framework with multi-writer fan-out and `Microsoft.Extensions.Logging` integration. This directory contains the **core** logger and writer abstractions — the defining layer for the logging system.
> For full architecture, configuration, all writers, and remote logging flow see [`docs/LOGGING.md`](../../docs/LOGGING.md).
## Architecture ## Architecture
``` ```
IAcLoggerBase (: IAcLogWriterBase, ILogger) IAcLoggerBase (: IAcLogWriterBase, ILogger)
└─ AcLoggerBase (abstract, delegates to writers) └─ AcLoggerBase (abstract, multi-writer fan-out, ILogger bridge)
└─ [concrete loggers per app] └─ [concrete loggers per consuming project]
IAcLogWriterBase IAcLogWriterBase
└─ AcLogWriterBase (abstract, config from appsettings) └─ AcLogWriterBase (abstract, per-writer config from appsettings)
└─ AcTextLogWriterBase (abstract, text formatting) ├─ AcTextLogWriterBase (abstract, text formatting)
└─ AcConsoleLogWriter (colored console output) │ └─ AcConsoleLogWriter (colored console output)
└─ [AcLogItemWriterBase<T> in AyCode.Entities — structured writers]
``` ```
**Two-level filtering:** Logger has a global `LogLevel` gate; each writer has its own `LogLevel`. Both must pass for a log entry to be written. See [`docs/LOGGING.md` → Design Overview](../../docs/LOGGING.md#design-overview).
## Key Files ## Key Files
### Logger Core ### Logger Core
- **`IAcLoggerBase.cs`** — Unified interface combining `IAcLogWriterBase` + `ILogger`. Exposes `GetWriters` and `Writer<T>()`. - **`IAcLoggerBase.cs`** — Unified interface combining `IAcLogWriterBase` + `ILogger`. Exposes `GetWriters` and `Writer<T>()`.
- **`AcLoggerBase.cs`** — Abstract logger implementing `ILogger`. Manages a list of `IAcLogWriterBase` writers. Maps MS `LogLevel` to `AcLogLevel`. Reads writer config from `appsettings.json` (`AyCode:Logger:LogWriters`). - **`AcLoggerBase.cs`** — Abstract logger implementing `ILogger`. Manages `List<IAcLogWriterBase>`. Reads config from `appsettings.json` (`AyCode:Logger:{AppType, LogLevel, LogWriters[]}`). Maps MS `LogLevel` to AC `LogLevel`. Fan-out dispatch to all writers. `[Conditional("DEBUG")]` variants for all methods.
- **`AcLoggerAdapter.cs`** — `AcLoggerProvider<TLogger>` implementing `ILoggerProvider` for DI integration. Extension methods: `AddAcLogger<T>()`, `UseOnlyAcLogger<T>()`. - **`AcLoggerAdapter.cs`** — `AcLoggerProvider<TLogger>` implementing `ILoggerProvider` with `ConcurrentDictionary<string, TLogger>` per-category cache. Extension methods: `AddAcLogger<T>()`, `UseOnlyAcLogger<T>()`.
### Writers ### Writers
- **`IAcLogWriterBase.cs`** — Writer contract: `Detail()`, `Debug()`, `Info()`, `Warning()`, `Suggest()`, `Error()`, `Write()`. - **`IAcLogWriterBase.cs`** — Writer contract: `Detail()`, `Debug()`, `Info()`, `Warning()`, `Suggest()`, `Error()`, plus `Write()` overloads and `Write(IAcLogItemClient)`.
- **`IAcLogWriterClientBase.cs`** — Marker interface for client-side writers. - **`IAcLogWriterClientBase.cs`** — Marker interface for client-side writers (no additional members).
- **`AcLogWriterBase.cs`** — Abstract base with config loading from appsettings. - **`AcLogWriterBase.cs`** — Abstract base. Own `LogLevel` loaded from appsettings by matching `AssemblyQualifiedName`. Named methods delegate to terminal `Write(AppType, LogLevel, text, caller, category, errorType, exMessage)`.
- **`AcTextLogWriterBase.cs`** — Abstract text formatter. Output format: `[TIME] [APP] [LEVEL] [CATEGORY->METHOD] [THREADID] TEXT [ERROR]`. - **`AcTextLogWriterBase.cs`** — Abstract text formatter. Format: `[HH:mm:ss.fff] [AppType[0]] [Level] [Category->Method] [ThreadId] Text [Error]`. Subclasses implement `WriteText(string, LogLevel)`.
- **`AcConsoleLogWriter.cs`** — Colored console writer (Gray=Trace, Cyan=Suggest, Yellow=Warning, Red=Error). Thread-safe via lock. - **`AcConsoleLogWriter.cs`** — Colored console writer. Thread-safe via `static lock`. Colors: Gray=≤Trace, White=Debug–Info, Cyan=Suggest, Yellow=Warning, Red=≥Error.
### Supporting ### Supporting
- **`IAcLogItemClient.cs`** — Structured log item DTO for remote transmission. - **`IAcLogItemClient.cs`** — Structured log item DTO interface for remote transmission (TimeStampUtc, AppType, LogLevel, ThreadId, CategoryName, CallerName, Text, Exception, ErrorType).
- **`LogLevel.cs`** — Byte enum: Detail(0), Trace(5), Debug(10), Info(15), Suggest(17), Warning(20), Error(25), Disabled(255). **Must match the database LogLevel table.** - **`LogLevel.cs`** — Byte enum: Detail(0), Trace(5), Debug(10), Info(15), Suggest(17), Warning(20), Error(25), Disabled(255). ⚠️ **Values synchronized with database `LogLevel` table — do NOT renumber.**

9
AyCode.Core/README.md
View File

@ -2,6 +2,15 @@
Core library for the AyCode platform. Targets .NET 9 (set in `AyCode.Core.targets`). Provides serialization (Binary, JSON, Toon), compression, logging, validation, and shared utilities. Core library for the AyCode platform. Targets .NET 9 (set in `AyCode.Core.targets`). Provides serialization (Binary, JSON, Toon), compression, logging, validation, and shared utilities.
## Documentation
| Document | Topic |
|---|---|
| [BINARY_FORMAT.md](docs/BINARY_FORMAT.md) | Binary wire format spec (stream layout, type markers) |
| [BINARY_FEATURES.md](docs/BINARY_FEATURES.md) | Binary features (interning, ref tracking, property ordering) |
| [BINARY_OPTIONS.md](docs/BINARY_OPTIONS.md) | Binary configuration options & presets |
| [LOGGING.md](docs/LOGGING.md) | Logging framework (hierarchy, config, ILogger bridge) |
## Folder Structure ## Folder Structure
| Folder | Purpose | README | | Folder | Purpose | README |

93
AyCode.Core/docs/BINARY_FEATURES.md Normal file
View File

@ -0,0 +1,93 @@
# AcBinary Features
Advanced serialization features built on top of the wire format. For core type markers and encoding see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). For configuration options and presets see [`BINARY_OPTIONS.md`](BINARY_OPTIONS.md).
## Compact Encoding Selection
The serializer applies compact encodings automatically:
| Data | Condition | Encoding | Savings |
|------|-----------|----------|---------|
| Integer | −16 ≤ v ≤ 47 | TinyInt (1 byte) | 2–5 bytes |
| String | ≤31 bytes, ASCII | FixStr (1+N bytes) | 1 byte (no length prefix) |
| Object | type index < 64 | FixObj (1 byte) | 1–5 bytes (no VarUInt index) |
| String | empty | StringEmpty (1 byte) | 1+ bytes |
| Bool | — | True/False (1 byte) | no payload |
## String Interning Protocol
Controls deduplication of repeated string values.
**Modes** (`StringInterningMode`):
- `None` — all strings inline, no overhead
- `Attribute` — only `[AcStringIntern]` properties interned (default)
- `All` — all strings within length limits interned
**Length limits:** `MinStringInternLength=4`, `MaxStringInternLength=64` (configurable).
**Wire protocol:**
1. Serializer pre-scans all eligible strings to build a plan (which strings repeat)
2. First occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]`
3. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]`
4. Single-occurrence strings: written as normal `String`/`FixStr` (no interning overhead)
## Reference Tracking
Prevents infinite loops and preserves object identity for repeated references.
**Modes** (`ReferenceHandlingMode`):
- `None` — no tracking (fastest, use when graph is a tree)
- `OnlyId` — track only `IId` objects (matched by ID value)
- `All` — track all reference types (two-phase scan required)
**Two-phase process:**
1. **Scan pass** (`ScanPass.cs`) — walks the object graph, detects multi-referenced objects and repeated strings. Builds a `WriteDuplicateEntry[]` array (the "write plan") containing `VisitIndex`, `CacheMapIndex`, `IsFirst`, and `Value` for each duplicate.
2. **Sort** — write plan entries are sorted by `VisitIndex` to match the write pass traversal order.
3. **Serialize pass** — consumes the sorted write plan via `TryConsumeWritePlanEntry()`. A cursor (`_nextWritePlanVisitIndex`) advances through the plan in O(1) — no dictionary lookups during serialization.
**Wire protocol:**
- First occurrence: `[ObjectRefFirst(70)] [VarUInt refCacheIndex] [object body...]`
- Subsequent: `[ObjectRef(65)] [VarUInt refCacheIndex]`
**Example — same object referenced twice:**
```
Input: { Users: [userA, userA] } (same instance)
Scan pass → WritePlan:
[{VisitIndex:2, CacheMapIndex:0, IsFirst:true},
{VisitIndex:3, CacheMapIndex:0, IsFirst:false}]
Wire output (Compact mode, ReferenceHandling=All):
[version=1] [flags=0x96] [VarUInt cacheCount=1] ← header
[FixObj(0)] ← root object
[Array(66)] [VarUInt(2)] ← Users array, 2 elements
[ObjectRefFirst(70)] [VarUInt(0)] [props...] ← userA, 1st occurrence
[ObjectRef(65)] [VarUInt(0)] ← userA, 2nd (2 bytes only)
```
## Property Ordering
Properties are serialized in a deterministic order defined by `TypeMetadataBase.GetUnfilteredProperties()`:
1. Walk the inheritance chain from **derived → base** (`currentType.BaseType` loop)
2. At each level, collect declared public instance properties
3. Sort **alphabetically** (`StringComparer.Ordinal`) within each level
4. Result: **base properties first, then derived, alphabetical within each level**
This order is stable across serializer/deserializer as long as the type hierarchy doesn't change.
### Cross-Type Deserialization (UseMetadata)
When `UseMetadata=true`, property name hashes (FNV-1a via `FnvHash.ComputeString`) are written per type, enabling schema evolution:
- **Serializer** writes property hashes in the metadata section (`ObjectWithMetadata(69)`)
- **Deserializer** builds an index mapping array (`GetIndexMapping()`) that maps source property indices to destination indices by matching FNV-1a hashes
- This allows deserialization even when source and destination types have different property sets or ordering
When `UseMetadata=false`, properties are matched by **positional index only** — source and destination must have identical property layouts.
**Edge cases:**
- **Hash collision** (`CheckDuplicatePropName=true`, default): throws `InvalidOperationException`. When `false`: collision silently ignored — risk of data corruption.
- **Source has unknown property** (not in destination): silently skipped via `SkipValue()`, no error.
- **Destination has extra property** (not in source): left at default value (new instance) or unchanged (populate mode).

167
AyCode.Core/docs/BINARY_FORMAT.md Normal file
View File

@ -0,0 +1,167 @@
# AcBinary Wire Format
Complete wire format specification for the AcBinary serializer. Source of truth: [`Serializers/Binaries/BinaryTypeCode.cs`](../Serializers/Binaries/BinaryTypeCode.cs).
> For advanced features (compact encoding, string interning, reference tracking, property ordering) see [`BINARY_FEATURES.md`](BINARY_FEATURES.md).
> For configuration options, presets, and option interactions see [`BINARY_OPTIONS.md`](BINARY_OPTIONS.md).
## Stream Layout
```
[version : 1 byte] [flags : 1 byte] [cacheCount : VarUInt?] [payload...]
```
- **version** — `FormatVersion = 1` (current).
- **flags** — See [Header Flags](#header-flags).
- **cacheCount** — Present only when `HeaderFlag_HasCacheCount` is set. Number of type wrapper slots used by serializer.
## Header Flags
The flags byte uses `0x90` (144) as base with bit flags in the lower nibble:
| Bit | Mask | Flag | Meaning |
|-----|------|------|---------|
| 0 | `0x01` | Metadata | Property hash metadata included (cross-type deserialization) |
| 1 | `0x02` | RefHandling_OnlyId | Reference tracking for `IId` objects only |
| 2 | `0x04` | RefHandling_All | Reference tracking for all objects (always combined with bit 1) |
| 3 | `0x08` | HasCacheCount | VarUInt cache count follows the flags byte |
**Reference handling modes:** None = `0x00`, OnlyId = `0x02`, All = `0x06` (bits 1+2).
## Variable-Length Encoding
### VarUInt (unsigned)
LEB128: 7 data bits per byte, MSB = continuation flag.
```
value < 128 → 1 byte [0xxxxxxx]
value < 16384 → 2 bytes [1xxxxxxx] [0xxxxxxx]
value < 2097152 → 3 bytes ...
(max 5 bytes for uint32)
```
### VarInt (signed)
ZigZag encoding maps signed to unsigned, then LEB128:
```
encode: (value << 1) ^ (value >> 31)
decode: (raw >> 1) ^ -(raw & 1)
```
Maps: `0 → 0`, `-1 → 1`, `1 → 2`, `-2 → 3`, etc.
### VarULong (unsigned 64-bit)
Same LEB128 encoding, max 10 bytes for uint64.
## Type Markers
All markers defined in `BinaryTypeCode.cs`. `SlotCount = 64`.
### FixObj (0–63)
Single-byte object type. The marker byte **is** the type slot index — no additional type identifier needed.
```
[FixObj(N)] [properties...]
```
**Slot allocation:** Slots 0–63 are reserved for runtime polymorphic types, assigned dynamically on first encounter during serialization. Source-generated (SGen) types receive slots starting at 64+ via `AllocateWrapperSlot()` (sequential, `Interlocked.Increment`). SGen slots are compile-time stable; runtime slots depend on serialization order.
### Complex Types (64–71)
| Code | Name | Wire format |
|------|------|-------------|
| 64 | Object | `[64] [VarUInt typeIndex] [properties...]` |
| 65 | ObjectRef | `[65] [VarUInt refCacheIndex]` |
| 66 | Array | `[66] [VarUInt count] [elements...]` |
| 67 | Dictionary | `[67] [VarUInt count] [key, value pairs...]` |
| 68 | ByteArray | `[68] [VarUInt length] [raw bytes]` |
| 69 | ObjectWithMetadata | `[69] [VarUInt typeIndex] [VarUInt hashCount] [hashes...] [properties...]` |
| 70 | ObjectRefFirst | `[70] [VarUInt refCacheIndex] [object body...]` |
| 71 | ObjectWithMetadataRefFirst | `[71] [VarUInt refCacheIndex] [metadata + properties...]` |
### Polymorphic Types (72–75)
Used when runtime type differs from declared property type and `UseMetadata=false`.
| Code | Name | Wire format |
|------|------|-------------|
| 72 | ObjectWithTypeName | `[72] [UTF8 typeName] [inner marker] [body...]` — prefix, inner Object/Array/Dict follows |
| 73 | ObjectWithTypeNameRefFirst | `[73] [UTF8 typeName] [VarUInt refCacheIndex] [properties...]` — combined, no inner marker |
| 74 | ObjectWithTypeIndex | `[74] [VarUInt typeIndex] [inner marker] [body...]` — prefix |
| 75 | ObjectWithTypeIndexRefFirst | `[75] [VarUInt typeIndex] [VarUInt refCacheIndex] [properties...]` — combined |
Second occurrence of a referenced polymorphic object uses plain `ObjectRef(65)` — no polymorphic prefix needed.
### Primitives (76–90)
| Code | Name | Wire format |
|------|------|-------------|
| 76 | Null | `[76]` — no payload |
| 77 | True | `[77]` — no payload |
| 78 | False | `[78]` — no payload |
| 79 | Int8 | `[79] [1 byte]` |
| 80 | UInt8 | `[80] [1 byte]` |
| 81 | Int16 | `[81] [VarInt]` |
| 82 | UInt16 | `[82] [VarUInt]` |
| 83 | Int32 | `[83] [VarInt]` |
| 84 | UInt32 | `[84] [VarUInt]` |
| 85 | Int64 | `[85] [VarLong]` |
| 86 | UInt64 | `[86] [VarULong]` |
| 87 | Float32 | `[87] [4 bytes IEEE 754]` |
| 88 | Float64 | `[88] [8 bytes IEEE 754]` |
| 89 | Decimal | `[89] [16 bytes]` |
| 90 | Char | `[90] [VarUInt]` |
### Strings (91–94)
| Code | Name | Wire format |
|------|------|-------------|
| 91 | String | `[91] [VarUInt byteLength] [UTF-8 bytes]` |
| 92 | StringInterned | `[92] [VarUInt cacheIndex]` — 2nd+ occurrence |
| 93 | StringEmpty | `[93]` — no payload |
| 94 | StringInternFirst | `[94] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]` — 1st occurrence |
### Date/Time (95–98)
| Code | Name | Wire format |
|------|------|-------------|
| 95 | DateTime | `[95] [8 bytes ticks]` |
| 96 | DateTimeOffset | `[96] [8 bytes ticks] [VarInt offsetMinutes]` |
| 97 | TimeSpan | `[97] [VarLong ticks]` |
| 98 | Guid | `[98] [16 bytes]` |
### Other Markers
| Code | Name | Wire format |
|------|------|-------------|
| 99 | Enum | `[99] [VarInt underlyingValue]` |
| 100 | MetadataHeader | Legacy: implies `RefHandling=true` + metadata present |
| 101 | NoMetadataHeader | Legacy: implies `RefHandling=true`, no metadata |
| 102 | PropertySkip | `[102]` — marks skipped property (default/null value) |
### FixStr (103–134)
Short ASCII strings encoded in a single marker byte + raw bytes (no length prefix):
```
[FixStrBase + byteLength] [ASCII bytes]
```
- Length range: 0–31 bytes (`FixStrBase=103`, `FixStrMax=134`)
- Saves 1 byte vs `String` marker + VarUInt length
- Falls back to `String(91)` if content is non-ASCII
### TinyInt (192–255)
Single-byte integer encoding for small values:
```
value = marker - 192 - 16 (range: -16 to 47)
marker = value + 16 + 192 (64 values total)
```
Saves 2+ bytes vs `Int32(83)` + VarInt for frequently occurring small integers.

154
AyCode.Core/docs/BINARY_OPTIONS.md Normal file
View File

@ -0,0 +1,154 @@
# AcBinary Configuration
Configuration options, presets, and option interactions for `AcBinarySerializerOptions`. For wire format see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). For features (interning, ref tracking, property ordering) see [`BINARY_FEATURES.md`](BINARY_FEATURES.md).
## WireMode
| Value | Integers | Strings | Output size | Speed |
|-------|----------|---------|-------------|-------|
| `Compact` (default) | VarInt/VarUInt (1–5 bytes) | UTF-8 with speculative ASCII fast path | Smaller | Slightly slower |
| `Fast` | Fixed-width raw bytes (4/8 bytes) | UTF-16 memcpy (`charCount * 2` bytes) | Larger | Fastest encode/decode |
**Format difference for strings:**
- Compact: `[VarUInt byteLength] [UTF-8 bytes]` — speculative ASCII (1 pass if all ASCII, rewind+UTF-8 fallback otherwise)
- Fast: `[VarUInt charCount] [raw UTF-16 bytes]` — zero-encoding memcpy
**Code branch:** `context.FastWire` flag set at `context.Reset()`. Checked in `WriteStringUtf8()` and integer write methods. FixStr optimization is skipped in Fast mode (UTF-8 specific).
## ReferenceHandling
| Value | Tracked objects | Scan pass | Header flags | Wire markers |
|-------|----------------|-----------|--------------|-------------|
| `None` | Nothing | Skipped | `0x00` | Standard object markers only |
| `OnlyId` | `IId` objects only (by ID value) | Partial | `0x02` | `ObjectRefFirst(70)` + `ObjectRef(65)` |
| `All` (default) | All reference types | Full graph walk | `0x06` | `ObjectRefFirst(70)` + `ObjectRef(65)` |
**Format impact:** When enabled, multi-referenced objects are written once with `ObjectRefFirst(70) + VarUInt(refCacheIndex)` on first encounter, then replaced by `ObjectRef(65) + VarUInt(refCacheIndex)` on subsequent encounters. Header `HasCacheCount` flag is set and cache count written.
**Interaction with `ThrowOnCircularReference` (default: `true`):**
- `true` + ref handling enabled: all objects tracked for cycle detection, throws `InvalidOperationException` on circular reference
- `false` + ref handling enabled: only IId types tracked for deduplication, non-IId circular refs silently truncated at `MaxDepth`
## UseMetadata
| Value | Wire markers | Property matching | Overhead |
|-------|-------------|-------------------|----------|
| `false` (default) | `FixObj`/`Object` | Positional index only — types must match | None |
| `true` | `ObjectWithMetadata(69)` / `ObjectWithMetadataRefFirst(71)` | FNV-1a property name hashes | 4 bytes per property per type |
**Format impact:** When enabled, each type's first occurrence writes `[VarUInt hashCount] [FNV-1a hash × N]` before properties. Deserializer uses hashes to build source→destination index mapping, enabling cross-type deserialization (different property sets/ordering).
**Code branch:** `context.UseMetadata` controls whether `ObjectWithMetadata(69)` or plain `Object(64)` markers are used. When `false`, `IsDirectObjectWrite=true` allows source-generated writers to bypass `WriteObject` entirely and inline property writes.
**Related:** `CheckDuplicatePropName` (default: `true`) — throws if FNV-1a hash collision detected between property names of the same type. Disable in production for performance.
## UseStringInterning
| Value | Eligible strings | Scan overhead | Wire markers |
|-------|-----------------|---------------|-------------|
| `None` | Nothing | None | `String(91)` / `FixStr` only |
| `Attribute` (default) | Properties with `[AcStringIntern(true)]` | Scans marked properties | `StringInternFirst(94)` + `StringInterned(92)` |
| `All` | All strings within length limits | Scans all strings | `StringInternFirst(94)` + `StringInterned(92)` |
**Length limits:** `MinStringInternLength` (default: 4) and `MaxStringInternLength` (default: 64, 0=unlimited). Strings outside this range are always written inline.
**Format impact:** Interned strings on first occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [string data]`. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]` (1–2 bytes vs full string). Single-occurrence strings are never interned — no overhead for unique strings.
**Code branch:** `context.StringInternEligible` flag set per-property before `WriteString`. Scan pass builds a `WriteDuplicateEntry[]` plan; write pass consumes it via cursor.
## MaxDepth
| Value | Behavior |
|-------|----------|
| `255` (default) | Effectively unlimited nesting |
| `0` | Root level only — nested objects/collections written as `Null(76)` |
| `N` | Objects deeper than N levels written as `Null(76)` |
**Format impact:** Depth-exceeded values appear as `Null(76)` in the stream — indistinguishable from actual null values. No special marker.
**Code branch:** Checked at entry of every object/collection write: `if (depth > MaxDepth) { WriteByte(Null); return; }`.
## UseCompression
| Value | Method | Granularity | Memory |
|-------|--------|-------------|--------|
| `None` (default) | No compression | — | — |
| `Block` | LZ4 single block | Entire payload | Full buffer in memory |
| `BlockArray` | LZ4 chunked | 64KB chunks | Streaming-friendly, lower peak memory |
**Format impact:** Compression is applied **post-serialization** as a transparent wrapper — the inner wire format is unchanged. Both modes are pure managed C# (WASM-compatible, no native dependencies).
**Code branch:** Applied in `AcBinarySerializer.Serialize()` after the serialization context produces the raw buffer: `if (UseCompression != None) Lz4.Compress(buffer, mode)`. Decompression is automatic on deserialize.
## PropertyFilter
Optional delegate `BinaryPropertyFilter?` (default: `null`). When set, invoked for each property to decide inclusion.
```
delegate bool BinaryPropertyFilter(in BinaryPropertyFilterContext context);
```
**BinaryPropertyFilterContext fields:** `DeclaringType`, `PropertyName`, `PropertyType`, `Instance` (null during metadata phase), `IsMetadataPhase`, `GetValue()` (lazy).
**Format impact:** Excluded properties are completely absent from the stream — no marker, no placeholder. The deserializer must use `UseMetadata=true` or identical filter to correctly match property indices.
**Code branch:** `context.HasPropertyFilter` checked in `ShouldSerializeProperty()`. Called twice: once during metadata registration (`Instance=null`), once during write phase.
## PropertyMapper
Optional delegate `PropertyMapperDelegate?` (default: `null`) for cross-type deserialization property remapping.
```
delegate PropertyInfo? PropertyMapperDelegate(PropertyInfo sourceProperty, Type destinationType);
```
**Purpose:** Maps properties between different class hierarchies (renamed properties, external DTOs). Result is cached — zero overhead on same-type operations (`Deserialize<T>`).
## WASM Options
| Option | Default | Purpose |
|--------|---------|---------|
| `IsWasm` | `OperatingSystem.IsBrowser()` | Auto-detect WASM environment |
| `UseStringCaching` | follows `IsWasm` | Cache short strings during deserialization to reduce GC pressure |
| `MaxCachedStringLength` | 64 | Max string length to cache |
**Format impact:** None — these are deserialization-only optimizations. When `UseStringCaching=true`, the deserializer maintains an intern cache for strings ≤ `MaxCachedStringLength` chars. Disabled automatically when `StringInternFirst` marker is encountered (interning takes precedence).
## Other Options
| Option | Type | Default | Purpose |
|--------|------|---------|---------|
| `UseGeneratedCode` | bool | `true` | Use source-generated writers/readers when available |
| `InitialBufferCapacity` | int | 4096 | Starting buffer size (bytes) for serialization output |
| `RemoveOrphanedItems` | bool | `false` | During `PopulateMerge`: remove destination collection items with no matching source ID |
| `UseAsync` | bool | `false` | Async context pool return via ThreadPool. Auto-disabled in WASM and when `ReferenceHandling=None` |
| `MaxContextPoolSize` | int | 8 | Max serialization contexts kept in pool |
## Presets
| Preset | WireMode | Metadata | StringInterning | RefHandling | MaxDepth | Compression | Other |
|--------|----------|----------|-----------------|-------------|----------|-------------|-------|
| `Default` | Compact | false | Attribute | All | 255 | None | — |
| `FastMode` | Compact | false | None | None | 255 | None | No scan pass |
| `ShallowCopy` | Compact | false | None | None | **0** | None | Root level only |
| `WasmOptimized` | Compact | false | Attribute | All | 255 | None | +StringCaching |
| `WithoutReferenceHandling` | Compact | false | Attribute | **None** | 255 | None | No scan pass |
| `WithoutMetadata` | Compact | **false** | Attribute | All | 255 | None | — |
**Performance implication of presets:**
- `Default` / `WasmOptimized` — two-phase (scan + serialize) due to `ReferenceHandling=All`
- `FastMode` / `ShallowCopy` — single-phase (no scan pass) since both interning and refs are disabled
- The scan pass adds ~20-30% overhead; disable it when the object graph is a simple tree
## Option Interactions
Key interdependencies that affect which code branches execute:
| Combination | Effect |
|-------------|--------|
| `ReferenceHandling=None` + `UseStringInterning=None` | **No scan pass** — fastest path, single-phase serialization |
| `ReferenceHandling=All` + `UseMetadata=true` | Uses `ObjectWithMetadataRefFirst(71)` marker — combined ref + metadata |
| `UseMetadata=false` + `UseGeneratedCode=true` | `IsDirectObjectWrite=true` — generated code inlines property writes, bypasses `WriteObject` |
| `UseMetadata=true` + `PropertyFilter` set | Filter invoked twice (metadata phase + write phase); filter results must be stable |
| `WireMode=Fast` + `UseStringInterning!=None` | Interned strings still use the fast string path (UTF-16 for first occurrence, VarUInt index for subsequent) |
| `UseCompression!=None` + any other option | Compression is orthogonal — applied post-serialization, inner format unchanged |

243
AyCode.Core/docs/LOGGING.md Normal file
View File

@ -0,0 +1,243 @@
# Logging
Custom logging framework with multi-writer fan-out and `Microsoft.Extensions.Logging` integration. Source: `Loggers/` in this project.
> For server-side GlobalLogger see [`AyCode.Core.Server/docs/LOGGING_SERVER.md`](../../AyCode.Core.Server/docs/LOGGING_SERVER.md).
> For remote writers (HTTP, browser console, SignalR) see [`AyCode.Services/docs/LOGGING_REMOTE.md`](../../AyCode.Services/docs/LOGGING_REMOTE.md).
## Design Overview
A logger holds a list of writers. Every log call fans out to all writers that pass the level filter. Two independent level gates control what gets written:
```
logger.Info("msg")
│
├─ Gate 1: Logger.LogLevel <= Info? ← global minimum
│ NO → discard
│ YES ↓
├─ Writer[0].Write(...)
│ └─ Gate 2: Writer.LogLevel <= Info? ← per-writer minimum
│ NO → discard
│ YES → write to output
├─ Writer[1].Write(...)
│ └─ Gate 2: ...
└─ Writer[N].Write(...)
└─ Gate 2: ...
```
## Class Hierarchy
```
IAcLogWriterBase (writer contract)
└─ AcLogWriterBase (abstract, config from appsettings)
├─ AcTextLogWriterBase (abstract, text formatting)
│ ├─ AcConsoleLogWriter (colored console output)
│ └─ AcBrowserConsoleLogWriter (AyCode.Services — Blazor JSInterop console)
└─ AcLogItemWriterBase<TLogItem> (AyCode.Entities — abstract, structured → ThreadPool + Mutex)
├─ AcDbLogItemWriter<TCtx,TItem> (AyCode.Database — EF Core database writer)
├─ AcHttpClientLogItemWriter<T> (AyCode.Services — HTTP POST as JSON)
└─ AcSignaRClientLogItemWriter (AyCode.Services — SignalR hub transport)
IAcLoggerBase (: IAcLogWriterBase, ILogger)
└─ AcLoggerBase (abstract, multi-writer fan-out)
└─ AcGlobalLoggerBase (sealed, used by GlobalLogger singleton)
└─ [concrete loggers in consuming projects]
IAcLogWriterClientBase (: IAcLogWriterBase) (marker for client-side writers)
```
**Two writer branches:**
| Branch | Base class | Output | Concurrency |
|--------|-----------|--------|-------------|
| **Text** | `AcTextLogWriterBase` | Formatted string → `WriteText(string, LogLevel)` | Depends on subclass (Console uses `lock`) |
| **Structured** | `AcLogItemWriterBase<TLogItem>` | `TLogItem` object → `WriteLogItemCallback(TLogItem)` | `TaskHelper.RunOnThreadPool` + `Mutex` |
## LogLevel
```csharp
public enum LogLevel : byte
{
Detail = 0,
Trace = 5,
Debug = 10,
Info = 15,
Suggest = 17,
Warning = 20,
Error = 25,
Disabled = 255,
}
```
⚠️ **Values are synchronized with the database `LogLevel` table.** Do NOT renumber.
Comparison is `<=`: a logger/writer with `LogLevel = Info` will process `Info`, `Suggest`, `Warning`, `Error` (anything ≥ 15).
## Configuration
All config lives under `AyCode:Logger` in `appsettings.json`:
```json
{
"AyCode": {
"Logger": {
"AppType": "Server",
"LogLevel": "Debug",
"LogWriters": [
{
"LogWriterType": "MyApp.Loggers.MyConsoleLogWriter, MyApp",
"LogLevel": "Info"
},
{
"LogWriterType": "MyApp.Loggers.MyDbLogWriter, MyApp",
"LogLevel": "Warning"
}
]
}
}
}
```
| Key | Type | Purpose |
|-----|------|---------|
| `AppType` | `AppType` enum | Identifies the application (Server, Web, etc.). Stamped on every log entry. |
| `LogLevel` | `LogLevel` enum | Global minimum — the logger's own gate. |
| `LogWriters[]` | array | One entry per writer. Each has `LogWriterType` (AssemblyQualifiedName) and `LogLevel` (per-writer gate). |
### Writer Instantiation
`AcLoggerBase` constructor iterates `LogWriters[]` and calls:
```csharp
Activator.CreateInstance(logWriterType, AppType, logWriterLogLevel, CategoryName)
```
Each writer's constructor signature must accept `(AppType, LogLevel, string?)`.
### Writer Config Self-Lookup
`AcLogWriterBase` also reads its own `LogLevel` from config by matching its `AssemblyQualifiedName` against the `LogWriterType` entries. This means a writer instantiated manually (not via config) can still pick up config values if a matching entry exists.
## Core Components
### AcLoggerBase
Abstract logger implementing both `IAcLogWriterBase` and `ILogger`. Central responsibilities:
1. **Writer management** — `List<IAcLogWriterBase> LogWriters`, `GetWriters`, `Writer<T>()`
2. **Fan-out dispatch** — Named methods (`Detail`, `Debug`, `Info`, `Warning`, `Suggest`, `Error`) check `LogLevel` then `ForEach` writers
3. **Write overloads** — Terminal: `Write(AppType, LogLevel, text, caller, category, errorType, exMessage)`
4. **Write(IAcLogItemClient)** — Accepts structured log items (from remote clients)
5. **ILogger bridge** — `Log<TState>`, `IsEnabled`, `BeginScope` (no-op `NullScope`)
6. **[Conditional("DEBUG")]** — Every named method has a `*Conditional` variant stripped from Release builds
**Constructors:**
| Signature | Behavior |
|-----------|----------|
| `(string? categoryName)` | Reads config from `appsettings.json`, creates writers via `Activator.CreateInstance` |
| `(string? categoryName, params IAcLogWriterBase[])` | Reads `AppType` + `LogLevel` from config, uses provided writers |
| `(AppType, LogLevel, string?, params IAcLogWriterBase[])` | Fully manual, no config reading |
### AcLogWriterBase
Abstract writer base. Each writer has its own `LogLevel` and `AppType`. Named methods (`Detail`, `Debug`, etc.) delegate to the terminal `Write(AppType, LogLevel, text, caller, category, errorType, exMessage)` which subclasses override.
The base `Write` throws `NotImplementedException` — subclasses **must** override either the 7-parameter `Write` (for text writers) or `Write(IAcLogItemClient)` (for structured writers), or both.
### AcTextLogWriterBase
Text formatting base for human-readable output. Overrides `Write` to format via `GetDiagnosticText` then calls abstract `WriteText(string, LogLevel)`.
**Output format:**
```
[HH:mm:ss.fff] [S] [Level] [Category->Method] [ThreadId] Text
[ERROR_TYPE]: exception details...
```
- Timestamp: `DateTime.UtcNow.ToLocalTime()` (local display, UTC storage)
- `[S]` — First character of `AppType` (e.g., `S`=Server, `W`=Web)
- Level: Left-padded to 9 chars
- Category→Method: Left-padded to 54 chars
- Error text on new line only if `errorType` is non-empty
### AcConsoleLogWriter
Colored console output. Thread-safe via `static readonly object ForWriterLock`.
| LogLevel | Color |
|----------|-------|
| ≤ Trace | Gray |
| Debug – Info | White (default) |
| Suggest | Cyan |
| Warning | Yellow |
| ≥ Error | Red (with extra newlines) |
## Microsoft.Extensions.Logging Bridge
### ILogger Bridge
`AcLoggerBase` implements `ILogger` directly. The `Log<TState>` method maps MS log levels to AC methods:
| Microsoft.Extensions.Logging.LogLevel | AyCode LogLevel | Method called |
|---------------------------------------|-----------------|---------------|
| Trace | Detail | `Detail()` |
| Debug | Debug | `Debug()` |
| Information | Info | `Info()` |
| Warning | Warning | `Warning()` |
| Error | Error | `Error()` |
| Critical | Error | `Error("[CRITICAL] ...")` |
| None | Disabled | — (ignored) |
`BeginScope` returns a no-op `NullScope` (scopes not supported).
**ShortenCategoryNames** (default: `true`): When MS logging provides fully-qualified type names as categories (e.g., `Microsoft.AspNetCore.SignalR.HubConnectionHandler`), shortens to just the class name (`HubConnectionHandler`).
### ILoggerProvider
`AcLoggerProvider<TLogger>` implements `ILoggerProvider` with a `ConcurrentDictionary<string, TLogger>` per-category cache. Factory function provided at registration.
**Extension methods:**
```csharp
// Add alongside default providers
builder.Logging.AddAcLogger<MyLogger>(categoryName => new MyLogger(categoryName));
// Replace all providers with only AcLogger
builder.Logging.UseOnlyAcLogger<MyLogger>(categoryName => new MyLogger(categoryName));
```
## Log Item Entity Hierarchy
Log items flow across projects as a chain of types:
```
IAcLogItemClient (AyCode.Core — DTO interface)
└─ AcLogItemClient (AyCode.Entities — [MessagePackObject])
└─ AcLogItem (AyCode.Entities.Server — [Table("LogItem")])
```
**IAcLogItemClient Fields:** TimeStampUtc, AppType, LogLevel, ThreadId, CategoryName, CallerName, Text, ErrorType, Exception.
## Patterns
### [Conditional("DEBUG")] Pattern
Every named log method has a `*Conditional` counterpart (e.g., `InfoConditional`, `ErrorConditional`) decorated with `[Conditional("DEBUG")]`. These are completely stripped from Release builds by the compiler. Both `AcLoggerBase` and `AcLogWriterBase` provide these. Use them for development-only diagnostics that should have zero cost in production.
### CallerMemberName Auto-Capture
All named log methods use `[CallerMemberName] string? memberName = null`. The compiler auto-fills the calling method name. For `ILogger.Log<TState>` calls (from MS logging), the `EventId.Name` is used as member name if available, otherwise defaults to `"Log"`.
## Key Source Files
| Component | Path |
|-----------|------|
| Logger base | `Loggers/AcLoggerBase.cs` |
| Writer base | `Loggers/AcLogWriterBase.cs` |
| Text writer base | `Loggers/AcTextLogWriterBase.cs` |
| Console writer | `Loggers/AcConsoleLogWriter.cs` |
| ILoggerProvider | `Loggers/AcLoggerAdapter.cs` |
| LogLevel enum | `Loggers/LogLevel.cs` |
| Log item DTO interface | `Loggers/IAcLogItemClient.cs` |
| Client marker | `Loggers/IAcLogWriterClientBase.cs` |

8
AyCode.Database/DbContexts/Loggers/README.md
View File

@ -1,8 +1,10 @@
# DbContexts / Loggers # DbContexts / Loggers
Logger-specific EF Core DbContext with NoTracking query behavior for read performance. Logger-specific EF Core DbContext with `NoTracking` query behavior for read performance. Used by `AcDbLogItemWriter` to persist log items.
> For full logging architecture see [`docs/LOGGING.md`](../../../../docs/LOGGING.md).
## Key Files ## Key Files
- **`IAcLoggerDbContextBase.cs`** — Interface for log item DbSet. - **`IAcLoggerDbContextBase.cs`** — Interface extending `IAcLogItemDbSetBase<TLogItem>` for log item DbSet.
- **`AcLoggerDbContextBase.cs`** — Implementation with NoTracking default, optimized for log reads. - **`AcLoggerDbContextBase.cs`** — Generic implementation (`AcDbContextBase` subclass). Configures `QueryTrackingBehavior.NoTracking` for read performance. Exposes `DbSet<TLogItem> LogItems`.

6
AyCode.Database/DbSets/Loggers/README.md
View File

@ -1,7 +1,9 @@
# DbSets / Loggers # DbSets / Loggers
Log item DbSet interface. Log item DbSet interface for EF Core log storage.
> For full logging architecture see [`docs/LOGGING.md`](../../../../docs/LOGGING.md).
## Key Files ## Key Files
- **`IAcLogItemDbSetBase.cs`** — Interface for LogItem DbSet. - **`IAcLogItemDbSetBase.cs`** — Interface exposing `DbSet<TLogItem>` for log items. Implemented by `AcLoggerDbContextBase`.

8
AyCode.Entities.Server/LogItems/README.md
View File

@ -1,8 +1,10 @@
# LogItems # LogItems
Server-side log item entity and interface, extending the client-side `AcLogItemClient` with a database-mapped identity and header reference. Server-side log item entity and interface, extending the client-side `AcLogItemClient` with database-mapped identity and header reference.
> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). For client-side entity see [`AyCode.Entities/LogItems/README.md`](../../AyCode.Entities/LogItems/README.md).
## Key Files ## Key Files
- **`IAcLogItem.cs`** — Interface extending `IAcLogItemClient` and `IEntityInt`, adding a `LogHeaderId` property. - **`IAcLogItem.cs`** — Interface extending `IAcLogItemClient` + `IEntityInt`, adding `LogHeaderId` property.
- **`AcLogItem.cs`** — Entity class mapped to the `LogItem` database table with MessagePack serialization. Provides `Id` (auto-generated) and `LogHeaderId`. - **`AcLogItem.cs`** — `[Table("LogItem")]` entity extending `AcLogItemClient`. MessagePack keys: `Id` (Key 55, auto-generated PK), `LogHeaderId` (Key 60). Used by `AcDbLogItemWriter` for EF Core persistence.

6
AyCode.Entities/LogItems/README.md
View File

@ -1,7 +1,9 @@
# LogItems # LogItems
Client-side log item entity used for structured logging. Serialized with MessagePack for efficient transport. Client-side log item entity used for structured logging. Serialized with MessagePack for efficient transport over SignalR.
> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md).
## Key Files ## Key Files
- **`AcLogItemClient.cs`** — Implements `IAcLogItemClient` with fields for `AppType`, `LogLevel`, `ThreadId`, `CategoryName`, `CallerName`, `Text`, and exception details. Uses `[MessagePackObject]` serialization with explicit key indices. - **`AcLogItemClient.cs`** — Implements `IAcLogItemClient`. `[MessagePackObject]` with explicit key indices (5, 15, 20, 25, 30, 35, 40, 45, 50). Fields: `TimeStampUtc`, `AppType`, `LogLevel`, `ThreadId`, `CategoryName`, `CallerName`, `Text`, `ErrorType`, `Exception`. Extended by `AcLogItem` on the server side.

4
AyCode.Services.Server/AyCode.Services.Server.csproj
View File

@ -33,4 +33,8 @@
<Folder Include="Emails\" /> <Folder Include="Emails\" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<None Include="docs\**\*.md" />
</ItemGroup>
</Project> </Project>

7
AyCode.Services.Server/README.md
View File

@ -2,6 +2,13 @@
Server-side service implementations: JWT authentication, SendGrid email delivery, SignalR hub infrastructure with binary protocol, session management, and change-tracked data sources. Server-side service implementations: JWT authentication, SendGrid email delivery, SignalR hub infrastructure with binary protocol, session management, and change-tracked data sources.
## Documentation
| Document | Topic |
|---|---|
| [SIGNALR_SERVER.md](docs/SIGNALR_SERVER.md) | Server-side SignalR hub (dispatch, session, broadcast) |
| [SIGNALR_DATASOURCE.md](docs/SIGNALR_DATASOURCE.md) | Real-time DataSource with CRUD & change tracking |
## Folder Structure ## Folder Structure
| Folder | Purpose | | Folder | Purpose |

43
docs/SIGNALR_DATASOURCE.md → AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md
View File

@ -1,8 +1,9 @@
# SignalR DataSource # SignalR DataSource
Change-tracked real-time collection built on top of the SignalR transport layer. Source: `AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs`. Change-tracked real-time collection built on top of the SignalR transport layer. Source: `SignalRs/AcSignalRDataSource.cs` in this project.
> For the underlying transport (tag system, wire protocol, dispatch) see [`SIGNALR.md`](SIGNALR.md). > For the underlying transport (tag system, wire protocol, client base) see [`AyCode.Services/docs/SIGNALR.md`](../../AyCode.Services/docs/SIGNALR.md).
> For server hub infrastructure see [`SIGNALR_SERVER.md`](SIGNALR_SERVER.md).
## Overview ## Overview
@ -43,7 +44,6 @@ public sealed class SignalRCrudTags(
**Usage (consuming project):** **Usage (consuming project):**
```csharp ```csharp
// Tags are defined as independent constants in the project's tag class
public abstract class MyProjectTags : AcSignalRTags public abstract class MyProjectTags : AcSignalRTags
{ {
public const int OrderGetAll = 300; public const int OrderGetAll = 300;
@ -51,10 +51,8 @@ public abstract class MyProjectTags : AcSignalRTags
public const int OrderAdd = 302; public const int OrderAdd = 302;
public const int OrderUpdate = 303; public const int OrderUpdate = 303;
public const int OrderRemove = 304; public const int OrderRemove = 304;
// Tags don't have to be sequential — they just happen to be here
} }
// Construct with 5 explicit tags
var crudTags = new SignalRCrudTags( var crudTags = new SignalRCrudTags(
MyProjectTags.OrderGetAll, MyProjectTags.OrderGetAll,
MyProjectTags.OrderGetItem, MyProjectTags.OrderGetItem,
@ -69,21 +67,12 @@ var crudTags = new SignalRCrudTags(
## Data Loading ## Data Loading
```csharp ```csharp
// Awaits full response via sync-wait transport pattern (polls up to 60s) await dataSource.LoadDataSource(); // sync-wait transport (polls up to 60s)
await dataSource.LoadDataSource(); await dataSource.LoadDataSourceAsync(); // async callback path
// Async callback — requests SignalResponseDataMessage directly to avoid double deserialization
await dataSource.LoadDataSourceAsync();
// Load from raw response bytes (public — usable when response is already available)
await dataSource.LoadDataSourceFromResponseData(responseData, serializerType); await dataSource.LoadDataSourceFromResponseData(responseData, serializerType);
await dataSource.LoadItem(id); // single item by ID
// Single item by ID — updates or adds to InnerList
await dataSource.LoadItem(id);
``` ```
All load methods are `async Task`. `LoadDataSource` uses the sync-wait transport pattern (`GetAllAsync<T>` → polls for response), while `LoadDataSourceAsync` uses the async callback path (avoids deserializing `ResponseData` into a typed object — works directly with `byte[]`).
**Binary deserialization paths:** **Binary deserialization paths:**
- `AcObservableCollection<T>`: `BeginUpdate()` → `BinaryToMerge()` → `EndUpdate()` — single batched UI notification. - `AcObservableCollection<T>`: `BeginUpdate()` → `BinaryToMerge()` → `EndUpdate()` — single batched UI notification.
- `List<T>`: `BinaryTo(InnerList)` — direct populate. - `List<T>`: `BinaryTo(InnerList)` — direct populate.
@ -120,14 +109,12 @@ Each operation has an optional `autoSave` parameter — if true, immediately cal
| Event | Signature | Fires when | | Event | Signature | Fires when |
|-------|-----------|------------| |-------|-----------|------------|
| `OnDataSourceItemChanged` | `Func<ItemChangedEventArgs<T>, Task>?` | After each item is saved or loaded (carries item + TrackingState) | | `OnDataSourceItemChanged` | `Func<ItemChangedEventArgs<T>, Task>?` | After each item is saved or loaded |
| `OnDataSourceLoaded` | `Func<Task>?` | After `LoadDataSource` / `LoadDataSourceAsync` completes | | `OnDataSourceLoaded` | `Func<Task>?` | After `LoadDataSource` / `LoadDataSourceAsync` completes |
| `OnSyncingStateChanged` | `Action<bool>?` | On 0→1 (true) and 1→0 (false) sync transitions | | `OnSyncingStateChanged` | `Action<bool>?` | On 0→1 (true) and 1→0 (false) sync transitions |
## SaveChanges ## SaveChanges
Two variants with different transport patterns:
| Method | Returns | Transport pattern | Use case | | Method | Returns | Transport pattern | Use case |
|--------|---------|-------------------|----------| |--------|---------|-------------------|----------|
| `SaveChanges()` | `List<TrackingItem>` (remaining failures) | Sync-wait (`ContinueWith`) | When caller needs to know what failed | | `SaveChanges()` | `List<TrackingItem>` (remaining failures) | Sync-wait (`ContinueWith`) | When caller needs to know what failed |
@ -144,8 +131,6 @@ Both follow the same flow:
EndSync() EndSync()
``` ```
**SaveItem(item, trackingState):** saves a single item (same flow). **SaveItem(id):** looks up tracking item by ID, then saves.
**Rollback:** `TryRollbackItem(id)` restores `OriginalValue` to `InnerList`. For `TrackingState.Add`: removes item entirely. For `Remove`: re-adds `OriginalValue`. Manual `Rollback()` reverts all tracked changes at once. **Rollback:** `TryRollbackItem(id)` restores `OriginalValue` to `InnerList`. For `TrackingState.Add`: removes item entirely. For `Remove`: re-adds `OriginalValue`. Manual `Rollback()` reverts all tracked changes at once.
## Sync State ## Sync State
@ -162,17 +147,14 @@ UI binds to `IsSyncing` to show loading indicators. The counter supports nested
## Working Reference List ## Working Reference List
Allows an external list to become the DataSource's inner storage:
```csharp ```csharp
dataSource.SetWorkingReferenceList(externalList); dataSource.SetWorkingReferenceList(externalList);
// Now dataSource operates directly on externalList — same reference, no copy // Now dataSource operates directly on externalList — same reference, no copy
// Read back the current inner list reference
TIList innerList = dataSource.GetReferenceInnerList(); TIList innerList = dataSource.GetReferenceInnerList();
``` ```
This is useful when the UI already has a bound collection and you want the DataSource to manage it in-place. `HasWorkingReferenceList` indicates whether an external list has been set. Useful when the UI already has a bound collection and you want the DataSource to manage it in-place.
## Locking Strategy ## Locking Strategy
@ -185,7 +167,7 @@ This is useful when the UI already has a bound collection and you want the DataS
## Relationship to Transport ## Relationship to Transport
The DataSource is a **consumer** of the SignalR transport, not part of it. The flow: The DataSource is a **consumer** of the SignalR transport, not part of it:
``` ```
DataSource.SaveChanges() DataSource.SaveChanges()
@ -195,14 +177,13 @@ DataSource.SaveChanges()
→ Server method with [SignalR(tag)] ← tag dispatch → Server method with [SignalR(tag)] ← tag dispatch
``` ```
Projects can also call the transport directly without DataSource — see [`SIGNALR.md` § How Projects Use Tags](SIGNALR.md#how-projects-use-tags). Projects can also call the transport directly without DataSource — see [`AyCode.Services/docs/SIGNALR.md`](../../AyCode.Services/docs/SIGNALR.md).
## Key Source Files ## Key Source Files
| Component | Path | | Component | Path |
|-----------|------| |-----------|------|
| DataSource | `AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs` | | DataSource | `SignalRs/AcSignalRDataSource.cs` |
| Tracking helpers | `SignalRs/TrackingItemHelpers.cs` |
| CRUD tags | `AyCode.Services/SignalRs/SignalRCrudTags.cs` | | CRUD tags | `AyCode.Services/SignalRs/SignalRCrudTags.cs` |
| Tracking helpers | `AyCode.Services.Server/SignalRs/TrackingItemHelpers.cs` |
| Transport client | `AyCode.Services/SignalRs/AcSignalRClientBase.cs` | | Transport client | `AyCode.Services/SignalRs/AcSignalRClientBase.cs` |
| Transport doc | [`docs/SIGNALR.md`](SIGNALR.md) |

104
AyCode.Services.Server/docs/SIGNALR_SERVER.md Normal file
View File

@ -0,0 +1,104 @@
# SignalR Server
Server-side SignalR hub infrastructure: method dispatch, session management, broadcast, and diagnostics. Source: `SignalRs/` in this project.
> For client-side transport (tags, wire protocol, client base) see [`AyCode.Services/docs/SIGNALR.md`](../../AyCode.Services/docs/SIGNALR.md).
> For the DataSource collection see [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md).
## Server Processing
```
6. OnReceiveMessage(tag, bytes, requestId)
7. DynamicMethodRegistry.GetMethodByMessageTag(tag) ← ConcurrentDictionary lookup
8. DeserializeParameters(bytes):
├─ DeserializeFromBinary<SignalPostJsonMessage>() ← unwrap Binary envelope
├─ IdMessage format? → parse each Ids[i] as JSON per parameter type
└─ Complex object? → json.JsonTo(paramType) ⚠️ tech debt: JSON parse
9. MethodInfo.InvokeMethod(instance, params) ← unwraps Task/ValueTask
10. CreateResponseMessage(tag, Success, result) ← pure Binary serialization
11. ResponseToCaller(tag, message, requestId)
12. If SendToOtherClientType != None:
└─ SendMessageToOthers(sendToOtherClientTag, result) ← uses sendToOtherClientTag, not messageTag
```
## Dynamic Method Dispatch
See also: [`AyCode.Models.Server/DynamicMethods/README.md`](../../AyCode.Models.Server/DynamicMethods/README.md)
### Server-Side Lookup
```
1. OnReceiveMessage(tag=100, bytes, requestId)
2. DynamicMethodRegistry.GetMethodByMessageTag(100)
├─ Check static ConcurrentDictionary<int, (Type, AcMethodInfoModel)?> cache
├─ Hit? → find instance of cached Type from registered instances
├─ Miss? → scan all registered instances' methods for [SignalR(100)]
│ cache the result (including negative = null)
└─ Return (instance, methodInfoModel) or null
3. AcMethodInfoModel contains:
├─ MethodInfo (the method to invoke)
├─ SignalRAttribute (tag, sendToOtherClientTag, sendToOtherClientType)
└─ ParamInfos[] (ParameterInfo for deserialization)
```
The `DynamicMethodRegistry` uses a static `ConcurrentDictionary` for the global tag→method cache.
### Registration
The hub registers service instances during initialization:
```csharp
DynamicMethodRegistry.Register(myService); // scans [SignalR(tag)] methods lazily
DynamicMethodRegistry.Register(anotherService);
```
Reflection runs lazily per tag on first request, then results are cached statically.
## Session Management
`AcSessionService<TSessionItem, TSessionItemId>` tracks connected clients:
```csharp
ConcurrentDictionary<TSessionItemId, TSessionItem> Sessions
```
`IAcSessionItem<TSessionItemId>` requires `SessionId` property. Used for targeting messages to specific users/connections.
## Broadcast Service
`AcSignalRSendToClientService<THub, TTags, TLogger>` provides server-push methods:
| Method | Target |
|--------|--------|
| `SendMessageToAllClients` | All connected |
| `SendMessageToConnection(connectionId)` | Single connection |
| `SendMessageToUser(userId)` | User (all connections) |
| `SendMessageToUsers(userIds)` | Multiple users |
All messages wrapped in `SignalResponseDataMessage` → binary serialized → `OnReceiveMessage`.
## Hub Events
- `OnConnectedAsync()` — log connection
- `OnDisconnectedAsync(exception)` — log disconnection, cleanup session
## Diagnostics
Enable with `AcWebSignalRHubBase.EnableBinaryDiagnostics = true`.
Logs: hex dump (500 byte sample), header parsing (version, marker), property count + names via VarUInt reading.
`SignalResponseDataMessage.DiagnosticLogger` — per-response logging: target type info, property list, inheritance chain, hex dump.
## Key Source Files
| Component | Path |
|-----------|------|
| Hub base | `SignalRs/AcWebSignalRHubBase.cs` |
| Session service | `SignalRs/AcSessionService.cs` |
| Broadcast service | `SignalRs/AcSignalRSendToClientService.cs` |
| Logger hub | `SignalRs/AcLoggerSignalRHub.cs` |
| Tracking helpers | `SignalRs/TrackingItemHelpers.cs` |
| Dynamic dispatch | `AyCode.Models.Server/DynamicMethods/AcDynamicMethodRegistry.cs` |

4
AyCode.Services/AyCode.Services.csproj
View File

@ -18,4 +18,8 @@
<ProjectReference Include="..\AyCode.Models\AyCode.Models.csproj" /> <ProjectReference Include="..\AyCode.Models\AyCode.Models.csproj" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<None Include="docs\**\*.md" />
</ItemGroup>
</Project> </Project>

12
AyCode.Services/Loggers/README.md
View File

@ -1,11 +1,11 @@
# Loggers # Loggers
Remote log writers for sending log items over HTTP, SignalR, or to the browser console. Remote log writers for sending log items over HTTP, SignalR, or to the browser console. All three implement `IAcLogWriterClientBase`.
> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). For core logger and writer abstractions see [`AyCode.Core/Loggers/README.md`](../../AyCode.Core/Loggers/README.md).
## Key Files ## Key Files
- **`AcHttpClientLogItemWriter.cs`** — Abstract generic writer extending `AcLogItemWriterBase<TLogItem>`. Sends log items via HTTP POST as JSON. Manages its own `HttpClient` lifecycle. - **`AcHttpClientLogItemWriter.cs`** — Abstract structured writer extending `AcLogItemWriterBase<TLogItem>`. Sends log items via `HttpClient.PostAsJsonAsync()` (fire-and-forget). Manages its own `HttpClient` + `HttpClientHandler`. HTTP/2 default.
- **`AcBrowserConsoleLogWriter.cs`** — Browser console writer via JS interop (`IJSRuntime`). Maps `LogLevel` to console methods (log, warn, error). Used in Blazor apps. - **`AcBrowserConsoleLogWriter.cs`** — Blazor browser console writer extending `AcTextLogWriterBase` (text branch). Uses `IJSRuntime.InvokeVoidAsync()` to call `console.info` / `console.warn` / `console.error` based on `LogLevel`.
- **`AcSignaRClientLogItemWriter.cs`** — Sends `AcLogItemClient` items to a SignalR hub. Manages `HubConnection` lifecycle with `StartConnection()`/`StopConnection()`. - **`AcSignaRClientLogItemWriter.cs`** — SignalR log transport writer extending `AcLogItemWriterBase<AcLogItemClient>`. Sends structured log items to `AcLoggerSignalRHub` via `HubConnection.SendAsync("AddLogItem", logItem)`. Manages connection lifecycle (`StartConnection`/`StopConnection`). Converts `TimeStampUtc` to UTC before sending (SignalR doesn't transmit `DateTime.Kind`).
All three implement `IAcLogWriterClientBase`.

7
AyCode.Services/README.md
View File

@ -2,6 +2,13 @@
Shared service implementations: SignalR communication (custom binary protocol), login services, and remote log writers. Shared service implementations: SignalR communication (custom binary protocol), login services, and remote log writers.
## Documentation
| Document | Topic |
|---|---|
| [SIGNALR.md](docs/SIGNALR.md) | Client-side SignalR transport (tags, wire protocol, req/resp flow) |
| [LOGGING_REMOTE.md](docs/LOGGING_REMOTE.md) | Remote log writers (HTTP, browser console, SignalR) |
## Folder Structure ## Folder Structure
| Folder | Purpose | | Folder | Purpose |

69
AyCode.Services/docs/LOGGING_REMOTE.md Normal file
View File

@ -0,0 +1,69 @@
# Remote Log Writers
Client-side log writers that send log data to remote endpoints. Source: `Loggers/` in this project. For core logging framework see [`AyCode.Core/docs/LOGGING.md`](../../AyCode.Core/docs/LOGGING.md). For server-side GlobalLogger see [`AyCode.Core.Server/docs/LOGGING_SERVER.md`](../../AyCode.Core.Server/docs/LOGGING_SERVER.md).
## AcBrowserConsoleLogWriter
Blazor browser console writer. Extends `AcTextLogWriterBase` (text branch, not structured). Uses `IJSRuntime` to invoke browser console methods:
| LogLevel | JS method |
|----------|-----------|
| Detail – Suggest | `console.info` |
| Warning | `console.warn` |
| Error | `console.error` |
Implements `IAcLogWriterClientBase` (client-side marker).
## AcHttpClientLogItemWriter\<TLogItem\>
HTTP POST writer. Extends `AcLogItemWriterBase<TLogItem>` (structured branch). Manages its own `HttpClient` + `HttpClientHandler`. Sends log items as JSON:
```csharp
_httpClient.PostAsJsonAsync(_url, logItem).Forget();
```
Note: `Forget()` — fire-and-forget, no await on the HTTP response. HTTP/2 by default.
## AcSignaRClientLogItemWriter
SignalR log transport writer. Extends `AcLogItemWriterBase<AcLogItemClient>`. Sends items to a dedicated logger hub. Manages its own `HubConnection`:
```csharp
_hubConnection.SendAsync("AddLogItem", logItem).Forget();
```
**UTC conversion note:** SignalR doesn't transmit `DateTime.Kind`. The writer calls `logItem.TimeStampUtc.ToUniversalTime()` before sending to ensure the server receives actual UTC.
Connection lifecycle: `StartConnection()` waits up to 10s for `Connected` state. `StopConnection()` stops and disposes the hub.
## Remote Logging Flow
Client applications can send log items to a server via SignalR:
```
Client App Server
────────── ──────
AcSignaRClientLogItemWriter AcLoggerSignalRHub<TLogger>
│ │
│ logItem.ToUniversalTime() │
│ StartConnection() │
├──SendAsync("AddLogItem", item)──► │
│ ├─ logItem.TimeStampUtc = UtcNow
│ ├─ _logger.Write(logItem)
│ │ ├─ Console writer
│ │ ├─ DB writer
│ │ └─ ... (all server writers)
```
Note: `AcLoggerSignalRHub` overrides the client's `TimeStampUtc` with `DateTime.UtcNow` on the server side for authoritative timestamps. The hub is a simple `Hub` (not `AcWebSignalRHubBase`) — it does NOT use tag-based dispatch.
## Key Source Files
| Component | Path |
|-----------|------|
| Browser writer | `Loggers/AcBrowserConsoleLogWriter.cs` |
| HTTP writer | `Loggers/AcHttpClientLogItemWriter.cs` |
| SignalR writer | `Loggers/AcSignaRClientLogItemWriter.cs` |
| Structured writer base | `AyCode.Entities/AcLogItemWriterBase.cs` |
| DB writer | `AyCode.Database/AcDbLogItemWriter.cs` |
| Logger hub (server) | `AyCode.Services.Server/SignalRs/AcLoggerSignalRHub.cs` |

120
docs/SIGNALR.md → AyCode.Services/docs/SIGNALR.md
View File

@ -1,8 +1,9 @@
# SignalR Communication # SignalR Client
The transport layer for AyCode real-time communication. Source: `AyCode.Services/SignalRs/` (client), `AyCode.Services.Server/SignalRs/` (server). Client-side SignalR transport with custom binary protocol and tag-based dispatch. Source: `SignalRs/` in this project.
> For the change-tracked collection built on top of this transport, see [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). > For server-side hub, session, broadcast see [`AyCode.Services.Server/docs/SIGNALR_SERVER.md`](../../AyCode.Services.Server/docs/SIGNALR_SERVER.md).
> For the DataSource collection see [`AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`](../../AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md).
## Design Overview ## Design Overview
@ -13,7 +14,7 @@ Client ──OnReceiveMessage(tag, bytes, requestId)──► Server
Client ◄──OnReceiveMessage(tag, bytes, requestId)── Server Client ◄──OnReceiveMessage(tag, bytes, requestId)── Server
``` ```
`OnReceiveMessage` is the only transport channel — all calls go through it. The `tag` (int) determines **which server method to invoke**. The `DynamicMethodRegistry` maps the tag to a concrete `[SignalR(tag)]`-annotated method at runtime. Projects call any endpoint by specifying the tag; the server dispatches accordingly. `OnReceiveMessage` is the only transport channel — all calls go through it. The `tag` (int) determines **which server method to invoke**. Projects call any endpoint by specifying the tag; the server dispatches accordingly.
``` ```
Client side: Server side: Client side: Server side:
@ -48,7 +49,6 @@ public abstract class MyProjectTags : AcSignalRTags
public const int GetOrders = 100; public const int GetOrders = 100;
public const int GetOrderById = 101; public const int GetOrderById = 101;
public const int SaveOrder = 102; public const int SaveOrder = 102;
// ... project-specific tags
} }
``` ```
@ -64,9 +64,6 @@ Three attribute levels:
// Server method: tag + optional broadcast behavior // Server method: tag + optional broadcast behavior
[SignalR(messageTag: 100, sendToOtherClientTag: 100, sendToOtherClientType: SendToClientType.Others)] [SignalR(messageTag: 100, sendToOtherClientTag: 100, sendToOtherClientType: SendToClientType.Others)]
// messageTag: incoming tag this method handles
// sendToOtherClientTag: tag used when broadcasting result to other clients (can differ from messageTag)
// sendToOtherClientType: who receives the broadcast
// Client receive: marks method for server→client dispatch // Client receive: marks method for server→client dispatch
[SignalRSendToClient(100)] [SignalRSendToClient(100)]
@ -79,7 +76,6 @@ Three attribute levels:
Projects call the SignalR transport directly — not only through DataSource: Projects call the SignalR transport directly — not only through DataSource:
```csharp ```csharp
// Direct call: project code calls any endpoint by tag
var orders = await signalRClient.PostAsync<List<Order>>(MyTags.GetOrders, companyId); var orders = await signalRClient.PostAsync<List<Order>>(MyTags.GetOrders, companyId);
var order = await signalRClient.GetByIdAsync<Order>(MyTags.GetOrderById, orderId); var order = await signalRClient.GetByIdAsync<Order>(MyTags.GetOrderById, orderId);
await signalRClient.PostDataAsync(MyTags.SaveOrder, order); await signalRClient.PostDataAsync(MyTags.SaveOrder, order);
@ -91,42 +87,7 @@ await signalRClient.PostDataAsync(MyTags.SaveOrder, order, async response => { .
await signalRClient.PostDataAsync(MyTags.SaveOrder, order, response => { ... }); await signalRClient.PostDataAsync(MyTags.SaveOrder, order, response => { ... });
``` ```
The `AcSignalRClientBase` CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are all generic transport methods that work with any tag — they are not tied to DataSource. The CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are all generic transport methods that work with any tag — they are not tied to DataSource.
## Dynamic Method Dispatch
See also: [`AyCode.Models.Server/DynamicMethods/README.md`](../AyCode.Models.Server/DynamicMethods/README.md)
### Server-Side Lookup
```
1. OnReceiveMessage(tag=100, bytes, requestId)
2. DynamicMethodRegistry.GetMethodByMessageTag(100)
├─ Check static ConcurrentDictionary<int, (Type, AcMethodInfoModel)?> cache
├─ Hit? → find instance of cached Type from registered instances
├─ Miss? → scan all registered instances' methods for [SignalR(100)]
│ cache the result (including negative = null)
└─ Return (instance, methodInfoModel) or null
3. AcMethodInfoModel contains:
├─ MethodInfo (the method to invoke)
├─ SignalRAttribute (tag, sendToOtherClientTag, sendToOtherClientType)
└─ ParamInfos[] (ParameterInfo for deserialization)
```
The `DynamicMethodRegistry` uses a static `ConcurrentDictionary` for the global tag→method cache. Note: `AcDynamicMethodCallModel` uses a `FrozenDictionary` per-type cache internally, but the registry's own lookup path does direct method scanning with `ConcurrentDictionary` caching.
### Registration
The hub registers service instances during initialization:
```csharp
DynamicMethodRegistry.Register(myService); // scans [SignalR(tag)] methods lazily
DynamicMethodRegistry.Register(anotherService);
```
Reflection runs lazily per tag on first request, then results are cached statically.
## Wire Protocol ## Wire Protocol
@ -170,22 +131,6 @@ Arguments serialized individually with VarUInt length prefix. Direct write to `I
5. AcBinaryHubProtocol frames it on the wire 5. AcBinaryHubProtocol frames it on the wire
``` ```
### Server Processing
```
6. OnReceiveMessage(tag, bytes, requestId)
7. DynamicMethodRegistry.GetMethodByMessageTag(tag) ← ConcurrentDictionary lookup
8. DeserializeParameters(bytes):
├─ DeserializeFromBinary<SignalPostJsonMessage>() ← unwrap Binary envelope
├─ IdMessage format? → parse each Ids[i] as JSON per parameter type
└─ Complex object? → json.JsonTo(paramType) ⚠️ tech debt: JSON parse
9. MethodInfo.InvokeMethod(instance, params) ← unwraps Task/ValueTask
10. CreateResponseMessage(tag, Success, result) ← pure Binary serialization
11. ResponseToCaller(tag, message, requestId)
12. If SendToOtherClientType != None:
└─ SendMessageToOthers(sendToOtherClientTag, result) ← uses sendToOtherClientTag, not messageTag
```
### Server → Client ### Server → Client
``` ```
@ -206,11 +151,11 @@ Arguments serialized individually with VarUInt length prefix. Direct write to `I
└─ Consuming project overrides to handle server push └─ Consuming project overrides to handle server push
``` ```
**Broadcast receive:** When the server broadcasts to `Others`/`All`, receiving clients have no matching `requestId`. The message falls through to `MessageReceived(int messageTag, byte[] messageBytes)` — an abstract method the consuming project overrides to handle unsolicited server pushes (e.g., refresh UI, update local state). **Broadcast receive:** When the server broadcasts to `Others`/`All`, receiving clients have no matching `requestId`. The message falls through to `MessageReceived(int messageTag, byte[] messageBytes)` — an abstract method the consuming project overrides.
**Request model pooling:** `SignalRRequestModel` instances are managed via `SignalRRequestModelPool` (ObjectPool<T> + IResettable) to avoid allocations per request. **Request model pooling:** `SignalRRequestModel` instances are managed via `SignalRRequestModelPool` (ObjectPool<T> + IResettable) to avoid allocations per request.
### Response Patterns ## Response Patterns
| Pattern | Method | Blocking | | Pattern | Method | Blocking |
|---------|--------|----------| |---------|--------|----------|
@ -218,29 +163,6 @@ Arguments serialized individually with VarUInt length prefix. Direct write to `I
| Async callback | `PostDataAsync(tag, data, async msg => {...})` | No, `Func<SignalResponseDataMessage, Task>` | | Async callback | `PostDataAsync(tag, data, async msg => {...})` | No, `Func<SignalResponseDataMessage, Task>` |
| Fire-and-forget | `PostDataAsync(tag, data, msg => {...})` | No, `Action<SignalResponseDataMessage>` | | Fire-and-forget | `PostDataAsync(tag, data, msg => {...})` | No, `Action<SignalResponseDataMessage>` |
## Session Management
`AcSessionService<TSessionItem, TSessionItemId>` tracks connected clients:
```csharp
ConcurrentDictionary<TSessionItemId, TSessionItem> Sessions
```
`IAcSessionItem<TSessionItemId>` requires `SessionId` property. Used for targeting messages to specific users/connections.
## Broadcast Service
`AcSignalRSendToClientService<THub, TTags, TLogger>` provides server-push methods:
| Method | Target |
|--------|--------|
| `SendMessageToAllClients` | All connected |
| `SendMessageToConnection(connectionId)` | Single connection |
| `SendMessageToUser(userId)` | User (all connections) |
| `SendMessageToUsers(userIds)` | Multiple users |
All messages wrapped in `SignalResponseDataMessage` → binary serialized → `OnReceiveMessage`.
## Connection Lifecycle ## Connection Lifecycle
**Client configuration:** **Client configuration:**
@ -249,18 +171,12 @@ All messages wrapped in `SignalResponseDataMessage` → binary serialized → `O
- Reconnection: `WithAutomaticReconnect()` + `WithStatefulReconnect()` - Reconnection: `WithAutomaticReconnect()` + `WithStatefulReconnect()`
- Keepalive: 60s interval, 180s server timeout - Keepalive: 60s interval, 180s server timeout
**Hub events:**
- `OnConnectedAsync()` — log connection
- `OnDisconnectedAsync(exception)` — log disconnection, cleanup session
## Diagnostics ## Diagnostics
Enable with `AcSignalRClientBase.EnableBinaryDiagnostics = true` or `AcWebSignalRHubBase.EnableBinaryDiagnostics = true`. Enable with `AcSignalRClientBase.EnableBinaryDiagnostics = true`.
Logs: hex dump (500 byte sample), header parsing (version, marker), property count + names via VarUInt reading. Logs: hex dump (500 byte sample), header parsing (version, marker), property count + names via VarUInt reading.
`SignalResponseDataMessage.DiagnosticLogger` — per-response logging: target type info, property list, inheritance chain, hex dump.
## Known Technical Debt ## Known Technical Debt
**JSON-in-Binary request parameters:** Client→server requests currently wrap parameters in JSON inside the binary envelope (`SignalPostJsonDataMessage`). This adds an unnecessary serialization round-trip. Responses are already pure binary. Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects. **JSON-in-Binary request parameters:** Client→server requests currently wrap parameters in JSON inside the binary envelope (`SignalPostJsonDataMessage`). This adds an unnecessary serialization round-trip. Responses are already pure binary. Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects.
@ -269,13 +185,11 @@ Logs: hex dump (500 byte sample), header parsing (version, marker), property cou
| Component | Path | | Component | Path |
|-----------|------| |-----------|------|
| Hub base | `AyCode.Services.Server/SignalRs/AcWebSignalRHubBase.cs` | | Client base | `SignalRs/AcSignalRClientBase.cs` |
| Client base | `AyCode.Services/SignalRs/AcSignalRClientBase.cs` | | Binary protocol | `SignalRs/AcBinaryHubProtocol.cs` |
| Binary protocol | `AyCode.Services/SignalRs/AcBinaryHubProtocol.cs` | | Tag attributes | `SignalRs/SignalMessageTagAttribute.cs` |
| Message types | `AyCode.Services/SignalRs/IAcSignalRHubClient.cs` | | Base tags | `SignalRs/AcSignalRTags.cs` |
| Serialization | `AyCode.Services/SignalRs/SignalRSerializationHelper.cs` | | CRUD tags | `SignalRs/SignalRCrudTags.cs` |
| Tag attributes | `AyCode.Services/SignalRs/SignalMessageTagAttribute.cs` | | SendToClientType | `SignalRs/SendToClientType.cs` |
| Base tags | `AyCode.Services/SignalRs/AcSignalRTags.cs` | | Message types | `SignalRs/IAcSignalRHubClient.cs` |
| Session service | `AyCode.Services.Server/SignalRs/AcSessionService.cs` | | Serialization | `SignalRs/SignalRSerializationHelper.cs` |
| Broadcast service | `AyCode.Services.Server/SignalRs/AcSignalRSendToClientService.cs` |
| Dynamic dispatch | `AyCode.Models.Server/DynamicMethods/AcDynamicMethodRegistry.cs` |

17
README.md
View File

@ -12,7 +12,22 @@ Domain rules and key pitfalls live in a single file: [`.github/copilot-instructi
| Claude Code | ✅ `CLAUDE.md` → references above | None | | Claude Code | ✅ `CLAUDE.md` → references above | None |
| Cursor / Windsurf | ✅ `README.md` | Read `copilot-instructions.md` via @file | | Cursor / Windsurf | ✅ `README.md` | Read `copilot-instructions.md` via @file |
Detailed docs: [`docs/`](docs/) — GLOSSARY.md, ARCHITECTURE.md, CONVENTIONS.md Solution-level docs in [`docs/`](docs/):
| Document | Topic |
|---|---|
| [GLOSSARY.md](docs/GLOSSARY.md) | Core terminology reference |
| [ARCHITECTURE.md](docs/ARCHITECTURE.md) | Solution layers, dependency rules |
| [CONVENTIONS.md](docs/CONVENTIONS.md) | Coding conventions |
Project-level docs — each project's `docs/` folder documents the code it defines:
| Project | Documents |
|---|---|
| [`AyCode.Core/docs/`](AyCode.Core/docs/) | [BINARY_FORMAT](AyCode.Core/docs/BINARY_FORMAT.md), [BINARY_FEATURES](AyCode.Core/docs/BINARY_FEATURES.md), [BINARY_OPTIONS](AyCode.Core/docs/BINARY_OPTIONS.md), [LOGGING](AyCode.Core/docs/LOGGING.md) |
| [`AyCode.Core.Server/docs/`](AyCode.Core.Server/docs/) | [LOGGING_SERVER](AyCode.Core.Server/docs/LOGGING_SERVER.md) |
| [`AyCode.Services/docs/`](AyCode.Services/docs/) | [SIGNALR](AyCode.Services/docs/SIGNALR.md), [LOGGING_REMOTE](AyCode.Services/docs/LOGGING_REMOTE.md) |
| [`AyCode.Services.Server/docs/`](AyCode.Services.Server/docs/) | [SIGNALR_SERVER](AyCode.Services.Server/docs/SIGNALR_SERVER.md), [SIGNALR_DATASOURCE](AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md) |
## Solution Structure ## Solution Structure

2
docs/ARCHITECTURE.md
View File

@ -36,7 +36,7 @@ AyCode.Services ← AyCode.Services.Server
- **AyCode.Services.Server** — Server-side: SignalR hub with custom binary protocol, email (SendGrid), JWT auth. - **AyCode.Services.Server** — Server-side: SignalR hub with custom binary protocol, email (SendGrid), JWT auth.
- **AyCode.Models.Server/DynamicMethods** — Reflection-based tag→method dispatch used by the SignalR hub. - **AyCode.Models.Server/DynamicMethods** — Reflection-based tag→method dispatch used by the SignalR hub.
> **SignalR Dispatch:** Both directions use a single method `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)` with integer tag-based routing instead of standard Hub methods. See [`SIGNALR.md`](SIGNALR.md) for full details. > **SignalR Dispatch:** Both directions use a single method `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)` with integer tag-based routing instead of standard Hub methods. See [`AyCode.Services/docs/SIGNALR.md`](../AyCode.Services/docs/SIGNALR.md) for full details.
### Server Extensions ### Server Extensions
- **AyCode.Core.Server**, **AyCode.Interfaces.Server**, **AyCode.Entities.Server**, **AyCode.Models.Server** — Server-only additions that don't belong in shared code. - **AyCode.Core.Server**, **AyCode.Interfaces.Server**, **AyCode.Entities.Server**, **AyCode.Models.Server** — Server-only additions that don't belong in shared code.

409
docs/BINARY_FORMAT.md
View File

@ -1,409 +0,0 @@
# AcBinary Wire Format
Complete wire format specification for the AcBinary serializer. Source of truth: [`AyCode.Core/Serializers/Binaries/BinaryTypeCode.cs`](../AyCode.Core/Serializers/Binaries/BinaryTypeCode.cs).
## Stream Layout
```
[version : 1 byte] [flags : 1 byte] [cacheCount : VarUInt?] [payload...]
```
- **version** — `FormatVersion = 1` (current).
- **flags** — See [Header Flags](#header-flags).
- **cacheCount** — Present only when `HeaderFlag_HasCacheCount` is set. Number of type wrapper slots used by serializer.
## Header Flags
The flags byte uses `0x90` (144) as base with bit flags in the lower nibble:
| Bit | Mask | Flag | Meaning |
|-----|------|------|---------|
| 0 | `0x01` | Metadata | Property hash metadata included (cross-type deserialization) |
| 1 | `0x02` | RefHandling_OnlyId | Reference tracking for `IId` objects only |
| 2 | `0x04` | RefHandling_All | Reference tracking for all objects (always combined with bit 1) |
| 3 | `0x08` | HasCacheCount | VarUInt cache count follows the flags byte |
**Reference handling modes:** None = `0x00`, OnlyId = `0x02`, All = `0x06` (bits 1+2).
## Variable-Length Encoding
### VarUInt (unsigned)
LEB128: 7 data bits per byte, MSB = continuation flag.
```
value < 128 → 1 byte [0xxxxxxx]
value < 16384 → 2 bytes [1xxxxxxx] [0xxxxxxx]
value < 2097152 → 3 bytes ...
(max 5 bytes for uint32)
```
### VarInt (signed)
ZigZag encoding maps signed to unsigned, then LEB128:
```
encode: (value << 1) ^ (value >> 31)
decode: (raw >> 1) ^ -(raw & 1)
```
Maps: `0 → 0`, `-1 → 1`, `1 → 2`, `-2 → 3`, etc.
### VarULong (unsigned 64-bit)
Same LEB128 encoding, max 10 bytes for uint64.
## Type Markers
All markers defined in `BinaryTypeCode.cs`. `SlotCount = 64`.
### FixObj (0–63)
Single-byte object type. The marker byte **is** the type slot index — no additional type identifier needed.
```
[FixObj(N)] [properties...]
```
**Slot allocation:** Slots 0–63 are reserved for runtime polymorphic types, assigned dynamically on first encounter during serialization. Source-generated (SGen) types receive slots starting at 64+ via `AllocateWrapperSlot()` (sequential, `Interlocked.Increment`). SGen slots are compile-time stable; runtime slots depend on serialization order.
### Complex Types (64–71)
| Code | Name | Wire format |
|------|------|-------------|
| 64 | Object | `[64] [VarUInt typeIndex] [properties...]` |
| 65 | ObjectRef | `[65] [VarUInt refCacheIndex]` |
| 66 | Array | `[66] [VarUInt count] [elements...]` |
| 67 | Dictionary | `[67] [VarUInt count] [key, value pairs...]` |
| 68 | ByteArray | `[68] [VarUInt length] [raw bytes]` |
| 69 | ObjectWithMetadata | `[69] [VarUInt typeIndex] [VarUInt hashCount] [hashes...] [properties...]` |
| 70 | ObjectRefFirst | `[70] [VarUInt refCacheIndex] [object body...]` |
| 71 | ObjectWithMetadataRefFirst | `[71] [VarUInt refCacheIndex] [metadata + properties...]` |
### Polymorphic Types (72–75)
Used when runtime type differs from declared property type and `UseMetadata=false`.
| Code | Name | Wire format |
|------|------|-------------|
| 72 | ObjectWithTypeName | `[72] [UTF8 typeName] [inner marker] [body...]` — prefix, inner Object/Array/Dict follows |
| 73 | ObjectWithTypeNameRefFirst | `[73] [UTF8 typeName] [VarUInt refCacheIndex] [properties...]` — combined, no inner marker |
| 74 | ObjectWithTypeIndex | `[74] [VarUInt typeIndex] [inner marker] [body...]` — prefix |
| 75 | ObjectWithTypeIndexRefFirst | `[75] [VarUInt typeIndex] [VarUInt refCacheIndex] [properties...]` — combined |
Second occurrence of a referenced polymorphic object uses plain `ObjectRef(65)` — no polymorphic prefix needed.
### Primitives (76–90)
| Code | Name | Wire format |
|------|------|-------------|
| 76 | Null | `[76]` — no payload |
| 77 | True | `[77]` — no payload |
| 78 | False | `[78]` — no payload |
| 79 | Int8 | `[79] [1 byte]` |
| 80 | UInt8 | `[80] [1 byte]` |
| 81 | Int16 | `[81] [VarInt]` |
| 82 | UInt16 | `[82] [VarUInt]` |
| 83 | Int32 | `[83] [VarInt]` |
| 84 | UInt32 | `[84] [VarUInt]` |
| 85 | Int64 | `[85] [VarLong]` |
| 86 | UInt64 | `[86] [VarULong]` |
| 87 | Float32 | `[87] [4 bytes IEEE 754]` |
| 88 | Float64 | `[88] [8 bytes IEEE 754]` |
| 89 | Decimal | `[89] [16 bytes]` |
| 90 | Char | `[90] [VarUInt]` |
### Strings (91–94)
| Code | Name | Wire format |
|------|------|-------------|
| 91 | String | `[91] [VarUInt byteLength] [UTF-8 bytes]` |
| 92 | StringInterned | `[92] [VarUInt cacheIndex]` — 2nd+ occurrence |
| 93 | StringEmpty | `[93]` — no payload |
| 94 | StringInternFirst | `[94] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]` — 1st occurrence |
### Date/Time (95–98)
| Code | Name | Wire format |
|------|------|-------------|
| 95 | DateTime | `[95] [8 bytes ticks]` |
| 96 | DateTimeOffset | `[96] [8 bytes ticks] [VarInt offsetMinutes]` |
| 97 | TimeSpan | `[97] [VarLong ticks]` |
| 98 | Guid | `[98] [16 bytes]` |
### Other Markers
| Code | Name | Wire format |
|------|------|-------------|
| 99 | Enum | `[99] [VarInt underlyingValue]` |
| 100 | MetadataHeader | Legacy: implies `RefHandling=true` + metadata present |
| 101 | NoMetadataHeader | Legacy: implies `RefHandling=true`, no metadata |
| 102 | PropertySkip | `[102]` — marks skipped property (default/null value) |
### FixStr (103–134)
Short ASCII strings encoded in a single marker byte + raw bytes (no length prefix):
```
[FixStrBase + byteLength] [ASCII bytes]
```
- Length range: 0–31 bytes (`FixStrBase=103`, `FixStrMax=134`)
- Saves 1 byte vs `String` marker + VarUInt length
- Falls back to `String(91)` if content is non-ASCII
### TinyInt (192–255)
Single-byte integer encoding for small values:
```
value = marker - 192 - 16 (range: -16 to 47)
marker = value + 16 + 192 (64 values total)
```
Saves 2+ bytes vs `Int32(83)` + VarInt for frequently occurring small integers.
## Compact Encoding Selection
The serializer applies compact encodings automatically:
| Data | Condition | Encoding | Savings |
|------|-----------|----------|---------|
| Integer | −16 ≤ v ≤ 47 | TinyInt (1 byte) | 2–5 bytes |
| String | ≤31 bytes, ASCII | FixStr (1+N bytes) | 1 byte (no length prefix) |
| Object | type index < 64 | FixObj (1 byte) | 1–5 bytes (no VarUInt index) |
| String | empty | StringEmpty (1 byte) | 1+ bytes |
| Bool | — | True/False (1 byte) | no payload |
## String Interning Protocol
Controls deduplication of repeated string values.
**Modes** (`StringInterningMode`):
- `None` — all strings inline, no overhead
- `Attribute` — only `[AcStringIntern]` properties interned (default)
- `All` — all strings within length limits interned
**Length limits:** `MinStringInternLength=4`, `MaxStringInternLength=64` (configurable).
**Wire protocol:**
1. Serializer pre-scans all eligible strings to build a plan (which strings repeat)
2. First occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]`
3. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]`
4. Single-occurrence strings: written as normal `String`/`FixStr` (no interning overhead)
## Reference Tracking
Prevents infinite loops and preserves object identity for repeated references.
**Modes** (`ReferenceHandlingMode`):
- `None` — no tracking (fastest, use when graph is a tree)
- `OnlyId` — track only `IId` objects (matched by ID value)
- `All` — track all reference types (two-phase scan required)
**Two-phase process:**
1. **Scan pass** (`ScanPass.cs`) — walks the object graph, detects multi-referenced objects and repeated strings. Builds a `WriteDuplicateEntry[]` array (the "write plan") containing `VisitIndex`, `CacheMapIndex`, `IsFirst`, and `Value` for each duplicate.
2. **Sort** — write plan entries are sorted by `VisitIndex` to match the write pass traversal order.
3. **Serialize pass** — consumes the sorted write plan via `TryConsumeWritePlanEntry()`. A cursor (`_nextWritePlanVisitIndex`) advances through the plan in O(1) — no dictionary lookups during serialization.
**Wire protocol:**
- First occurrence: `[ObjectRefFirst(70)] [VarUInt refCacheIndex] [object body...]`
- Subsequent: `[ObjectRef(65)] [VarUInt refCacheIndex]`
**Example — same object referenced twice:**
```
Input: { Users: [userA, userA] } (same instance)
Scan pass → WritePlan:
[{VisitIndex:2, CacheMapIndex:0, IsFirst:true},
{VisitIndex:3, CacheMapIndex:0, IsFirst:false}]
Wire output (Compact mode, ReferenceHandling=All):
[version=1] [flags=0x96] [VarUInt cacheCount=1] ← header
[FixObj(0)] ← root object
[Array(66)] [VarUInt(2)] ← Users array, 2 elements
[ObjectRefFirst(70)] [VarUInt(0)] [props...] ← userA, 1st occurrence
[ObjectRef(65)] [VarUInt(0)] ← userA, 2nd (2 bytes only)
```
## Property Ordering
Properties are serialized in a deterministic order defined by `TypeMetadataBase.GetUnfilteredProperties()`:
1. Walk the inheritance chain from **derived → base** (`currentType.BaseType` loop)
2. At each level, collect declared public instance properties
3. Sort **alphabetically** (`StringComparer.Ordinal`) within each level
4. Result: **base properties first, then derived, alphabetical within each level**
This order is stable across serializer/deserializer as long as the type hierarchy doesn't change.
### Cross-Type Deserialization (UseMetadata)
When `UseMetadata=true`, property name hashes (FNV-1a via `FnvHash.ComputeString`) are written per type, enabling schema evolution:
- **Serializer** writes property hashes in the metadata section (`ObjectWithMetadata(69)`)
- **Deserializer** builds an index mapping array (`GetIndexMapping()`) that maps source property indices to destination indices by matching FNV-1a hashes
- This allows deserialization even when source and destination types have different property sets or ordering
When `UseMetadata=false`, properties are matched by **positional index only** — source and destination must have identical property layouts.
**Edge cases:**
- **Hash collision** (`CheckDuplicatePropName=true`, default): throws `InvalidOperationException`. When `false`: collision silently ignored — risk of data corruption.
- **Source has unknown property** (not in destination): silently skipped via `SkipValue()`, no error.
- **Destination has extra property** (not in source): left at default value (new instance) or unchanged (populate mode).
## Configuration Options
Options defined in `AcBinarySerializerOptions` (inherits `AcSerializerOptions`). Each option controls which code paths execute and how the wire format changes.
### WireMode
| Value | Integers | Strings | Output size | Speed |
|-------|----------|---------|-------------|-------|
| `Compact` (default) | VarInt/VarUInt (1–5 bytes) | UTF-8 with speculative ASCII fast path | Smaller | Slightly slower |
| `Fast` | Fixed-width raw bytes (4/8 bytes) | UTF-16 memcpy (`charCount * 2` bytes) | Larger | Fastest encode/decode |
**Format difference for strings:**
- Compact: `[VarUInt byteLength] [UTF-8 bytes]` — speculative ASCII (1 pass if all ASCII, rewind+UTF-8 fallback otherwise)
- Fast: `[VarUInt charCount] [raw UTF-16 bytes]` — zero-encoding memcpy
**Code branch:** `context.FastWire` flag set at `context.Reset()`. Checked in `WriteStringUtf8()` and integer write methods. FixStr optimization is skipped in Fast mode (UTF-8 specific).
### ReferenceHandling
| Value | Tracked objects | Scan pass | Header flags | Wire markers |
|-------|----------------|-----------|--------------|-------------|
| `None` | Nothing | Skipped | `0x00` | Standard object markers only |
| `OnlyId` | `IId` objects only (by ID value) | Partial | `0x02` | `ObjectRefFirst(70)` + `ObjectRef(65)` |
| `All` (default) | All reference types | Full graph walk | `0x06` | `ObjectRefFirst(70)` + `ObjectRef(65)` |
**Format impact:** When enabled, multi-referenced objects are written once with `ObjectRefFirst(70) + VarUInt(refCacheIndex)` on first encounter, then replaced by `ObjectRef(65) + VarUInt(refCacheIndex)` on subsequent encounters. Header `HasCacheCount` flag is set and cache count written.
**Interaction with `ThrowOnCircularReference` (default: `true`):**
- `true` + ref handling enabled: all objects tracked for cycle detection, throws `InvalidOperationException` on circular reference
- `false` + ref handling enabled: only IId types tracked for deduplication, non-IId circular refs silently truncated at `MaxDepth`
### UseMetadata
| Value | Wire markers | Property matching | Overhead |
|-------|-------------|-------------------|----------|
| `false` (default) | `FixObj`/`Object` | Positional index only — types must match | None |
| `true` | `ObjectWithMetadata(69)` / `ObjectWithMetadataRefFirst(71)` | FNV-1a property name hashes | 4 bytes per property per type |
**Format impact:** When enabled, each type's first occurrence writes `[VarUInt hashCount] [FNV-1a hash × N]` before properties. Deserializer uses hashes to build source→destination index mapping, enabling cross-type deserialization (different property sets/ordering).
**Code branch:** `context.UseMetadata` controls whether `ObjectWithMetadata(69)` or plain `Object(64)` markers are used. When `false`, `IsDirectObjectWrite=true` allows source-generated writers to bypass `WriteObject` entirely and inline property writes.
**Related:** `CheckDuplicatePropName` (default: `true`) — throws if FNV-1a hash collision detected between property names of the same type. Disable in production for performance.
### UseStringInterning
| Value | Eligible strings | Scan overhead | Wire markers |
|-------|-----------------|---------------|-------------|
| `None` | Nothing | None | `String(91)` / `FixStr` only |
| `Attribute` (default) | Properties with `[AcStringIntern(true)]` | Scans marked properties | `StringInternFirst(94)` + `StringInterned(92)` |
| `All` | All strings within length limits | Scans all strings | `StringInternFirst(94)` + `StringInterned(92)` |
**Length limits:** `MinStringInternLength` (default: 4) and `MaxStringInternLength` (default: 64, 0=unlimited). Strings outside this range are always written inline.
**Format impact:** Interned strings on first occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [string data]`. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]` (1–2 bytes vs full string). Single-occurrence strings are never interned — no overhead for unique strings.
**Code branch:** `context.StringInternEligible` flag set per-property before `WriteString`. Scan pass builds a `WriteDuplicateEntry[]` plan; write pass consumes it via cursor.
### MaxDepth
| Value | Behavior |
|-------|----------|
| `255` (default) | Effectively unlimited nesting |
| `0` | Root level only — nested objects/collections written as `Null(76)` |
| `N` | Objects deeper than N levels written as `Null(76)` |
**Format impact:** Depth-exceeded values appear as `Null(76)` in the stream — indistinguishable from actual null values. No special marker.
**Code branch:** Checked at entry of every object/collection write: `if (depth > MaxDepth) { WriteByte(Null); return; }`.
### UseCompression
| Value | Method | Granularity | Memory |
|-------|--------|-------------|--------|
| `None` (default) | No compression | — | — |
| `Block` | LZ4 single block | Entire payload | Full buffer in memory |
| `BlockArray` | LZ4 chunked | 64KB chunks | Streaming-friendly, lower peak memory |
**Format impact:** Compression is applied **post-serialization** as a transparent wrapper — the inner wire format is unchanged. Both modes are pure managed C# (WASM-compatible, no native dependencies).
**Code branch:** Applied in `AcBinarySerializer.Serialize()` after the serialization context produces the raw buffer: `if (UseCompression != None) Lz4.Compress(buffer, mode)`. Decompression is automatic on deserialize.
### PropertyFilter
Optional delegate `BinaryPropertyFilter?` (default: `null`). When set, invoked for each property to decide inclusion.
```
delegate bool BinaryPropertyFilter(in BinaryPropertyFilterContext context);
```
**BinaryPropertyFilterContext fields:** `DeclaringType`, `PropertyName`, `PropertyType`, `Instance` (null during metadata phase), `IsMetadataPhase`, `GetValue()` (lazy).
**Format impact:** Excluded properties are completely absent from the stream — no marker, no placeholder. The deserializer must use `UseMetadata=true` or identical filter to correctly match property indices.
**Code branch:** `context.HasPropertyFilter` checked in `ShouldSerializeProperty()`. Called twice: once during metadata registration (`Instance=null`), once during write phase.
### PropertyMapper
Optional delegate `PropertyMapperDelegate?` (default: `null`) for cross-type deserialization property remapping.
```
delegate PropertyInfo? PropertyMapperDelegate(PropertyInfo sourceProperty, Type destinationType);
```
**Purpose:** Maps properties between different class hierarchies (renamed properties, external DTOs). Result is cached — zero overhead on same-type operations (`Deserialize<T>`).
### WASM Options
| Option | Default | Purpose |
|--------|---------|---------|
| `IsWasm` | `OperatingSystem.IsBrowser()` | Auto-detect WASM environment |
| `UseStringCaching` | follows `IsWasm` | Cache short strings during deserialization to reduce GC pressure |
| `MaxCachedStringLength` | 64 | Max string length to cache |
**Format impact:** None — these are deserialization-only optimizations. When `UseStringCaching=true`, the deserializer maintains an intern cache for strings ≤ `MaxCachedStringLength` chars. Disabled automatically when `StringInternFirst` marker is encountered (interning takes precedence).
### Other Options
| Option | Type | Default | Purpose |
|--------|------|---------|---------|
| `UseGeneratedCode` | bool | `true` | Use source-generated writers/readers when available |
| `InitialBufferCapacity` | int | 4096 | Starting buffer size (bytes) for serialization output |
| `RemoveOrphanedItems` | bool | `false` | During `PopulateMerge`: remove destination collection items with no matching source ID |
| `UseAsync` | bool | `false` | Async context pool return via ThreadPool. Auto-disabled in WASM and when `ReferenceHandling=None` |
| `MaxContextPoolSize` | int | 8 | Max serialization contexts kept in pool |
## Presets
| Preset | WireMode | Metadata | StringInterning | RefHandling | MaxDepth | Compression | Other |
|--------|----------|----------|-----------------|-------------|----------|-------------|-------|
| `Default` | Compact | false | Attribute | All | 255 | None | — |
| `FastMode` | Compact | false | None | None | 255 | None | No scan pass |
| `ShallowCopy` | Compact | false | None | None | **0** | None | Root level only |
| `WasmOptimized` | Compact | false | Attribute | All | 255 | None | +StringCaching |
| `WithoutReferenceHandling` | Compact | false | Attribute | **None** | 255 | None | No scan pass |
| `WithoutMetadata` | Compact | **false** | Attribute | All | 255 | None | — |
**Performance implication of presets:**
- `Default` / `WasmOptimized` — two-phase (scan + serialize) due to `ReferenceHandling=All`
- `FastMode` / `ShallowCopy` — single-phase (no scan pass) since both interning and refs are disabled
- The scan pass adds ~20-30% overhead; disable it when the object graph is a simple tree
## Option Interactions
Key interdependencies that affect which code branches execute:
| Combination | Effect |
|-------------|--------|
| `ReferenceHandling=None` + `UseStringInterning=None` | **No scan pass** — fastest path, single-phase serialization |
| `ReferenceHandling=All` + `UseMetadata=true` | Uses `ObjectWithMetadataRefFirst(71)` marker — combined ref + metadata |
| `UseMetadata=false` + `UseGeneratedCode=true` | `IsDirectObjectWrite=true` — generated code inlines property writes, bypasses `WriteObject` |
| `UseMetadata=true` + `PropertyFilter` set | Filter invoked twice (metadata phase + write phase); filter results must be stable |
| `WireMode=Fast` + `UseStringInterning!=None` | Interned strings still use the fast string path (UTF-16 for first occurrence, VarUInt index for subsequent) |
| `UseCompression!=None` + any other option | Compression is orthogonal — applied post-serialization, inner format unchanged |

4
docs/CONVENTIONS.md
View File

@ -24,11 +24,11 @@
## SignalR Conventions ## SignalR Conventions
See [`SIGNALR.md`](SIGNALR.md) for full architecture documentation. See [`AyCode.Services/docs/SIGNALR.md`](../AyCode.Services/docs/SIGNALR.md) for full architecture documentation.
- **Single dispatch method** — all communication goes through `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)`. Do not add new hub methods. - **Single dispatch method** — all communication goes through `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)`. Do not add new hub methods.
- **Tag-based routing** — associate methods with integer tags via `[SignalR(tag)]` (server) or `[SignalRSendToClient(tag)]` (client). Tags must be unique across the entire system. - **Tag-based routing** — associate methods with integer tags via `[SignalR(tag)]` (server) or `[SignalRSendToClient(tag)]` (client). Tags must be unique across the entire system.
- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). - **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See [`AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`](../AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md).
- **Binary protocol** — `AcBinaryHubProtocol` is the transport protocol. Responses use pure Binary serialization. - **Binary protocol** — `AcBinaryHubProtocol` is the transport protocol. Responses use pure Binary serialization.
### ⚠️ Temporary: JSON-in-Binary Request Parameters ### ⚠️ Temporary: JSON-in-Binary Request Parameters

25
docs/GLOSSARY.md
View File

@ -19,11 +19,11 @@ Core terminology for the AyCode framework. Read this before working on unfamilia
| **Toon** | Token-Oriented Object Notation. LLM-optimized format with @meta (schema) and @data (values) sections. Designed for maximum LLM comprehension accuracy. | | **Toon** | Token-Oriented Object Notation. LLM-optimized format with @meta (schema) and @data (values) sections. Designed for maximum LLM comprehension accuracy. |
| **AcJson** | Newtonsoft.Json wrapper with $id/$ref reference handling, IId-based resolution, and chain deserialization API. | | **AcJson** | Newtonsoft.Json wrapper with $id/$ref reference handling, IId-based resolution, and chain deserialization API. |
| **Chain API** | Fluent deserialization: `CreateDeserializeChain<T>().ThenDeserialize<U>()...Execute()`. Resolves cross-references across multiple types. | | **Chain API** | Fluent deserialization: `CreateDeserializeChain<T>().ThenDeserialize<U>()...Execute()`. Resolves cross-references across multiple types. |
| **String Interning** | Binary serializer deduplicates repeated strings. Controlled via `[AcStringIntern]` attribute or `StringInterningMode`. See [BINARY_FORMAT.md](BINARY_FORMAT.md#string-interning-protocol). | | **String Interning** | Binary serializer deduplicates repeated strings. Controlled via `[AcStringIntern]` attribute or `StringInterningMode`. See [`AyCode.Core/docs/BINARY_FEATURES.md`](../AyCode.Core/docs/BINARY_FEATURES.md#string-interning-protocol). |
## Binary Wire Format ## Binary Wire Format
For full specification see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). For full specification see [`AyCode.Core/docs/BINARY_FORMAT.md`](../AyCode.Core/docs/BINARY_FORMAT.md).
| Term | Definition | | Term | Definition |
|---|---| |---|---|
@ -66,14 +66,14 @@ For full specification see [`BINARY_FORMAT.md`](BINARY_FORMAT.md).
## SignalR Infrastructure ## SignalR Infrastructure
For full architecture see [`SIGNALR.md`](SIGNALR.md). For full architecture see [`AyCode.Services/docs/SIGNALR.md`](../AyCode.Services/docs/SIGNALR.md).
| Term | Definition | | Term | Definition |
|---|---| |---|---|
| **OnReceiveMessage** | The single SignalR method for all communication. Signature: `(int messageTag, byte[] messageBytes, int? requestId)`. | | **OnReceiveMessage** | The single SignalR method for all communication. Signature: `(int messageTag, byte[] messageBytes, int? requestId)`. |
| **Message Tag** | Integer identifier mapping to a method via `[SignalR(tag)]` or `[SignalRSendToClient(tag)]` attributes. | | **Message Tag** | Integer identifier mapping to a method via `[SignalR(tag)]` or `[SignalRSendToClient(tag)]` attributes. |
| **DynamicMethodRegistry** | Resolves message tags to `MethodInfo` at runtime. Static `ConcurrentDictionary` cache with lazy scan on miss. | | **DynamicMethodRegistry** | Resolves message tags to `MethodInfo` at runtime. Static `ConcurrentDictionary` cache with lazy scan on miss. |
| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). | | **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See [`AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`](../AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md). |
| **AcBinaryHubProtocol** | Custom `IHubProtocol` replacing SignalR's JSON+Base64 with `AcBinarySerializer`. Protocol name: `"acbinary"`. | | **AcBinaryHubProtocol** | Custom `IHubProtocol` replacing SignalR's JSON+Base64 with `AcBinarySerializer`. Protocol name: `"acbinary"`. |
| **SignalResponseDataMessage** | Response message supporting Binary or JSON+GZip. Responses use pure Binary (no JSON overhead). | | **SignalResponseDataMessage** | Response message supporting Binary or JSON+GZip. Responses use pure Binary (no JSON overhead). |
| **SignalPostJsonDataMessage** | ⚠️ TECH DEBT — request params serialized to JSON inside Binary envelope. Planned for pure Binary replacement. | | **SignalPostJsonDataMessage** | ⚠️ TECH DEBT — request params serialized to JSON inside Binary envelope. Planned for pure Binary replacement. |
@ -88,7 +88,20 @@ For full architecture see [`SIGNALR.md`](SIGNALR.md).
## Logging ## Logging
For full architecture see [`AyCode.Core/docs/LOGGING.md`](../AyCode.Core/docs/LOGGING.md).
| Term | Definition | | Term | Definition |
|---|---| |---|---|
| **AcLoggerBase** | Custom logger with Detail(0)→Debug(1)→Info(2)→Warning(3)→Suggest(4)→Error(5)→Disabled(255). Values are DB-synced — do not renumber. | | **LogLevel** | Byte enum: Detail(0), Trace(5), Debug(10), Info(15), Suggest(17), Warning(20), Error(25), Disabled(255). Values are DB-synced — do NOT renumber. |
| **GlobalLogger** | Server-side singleton (`AcGlobalLogger`) for cross-service logging. | | **AcLoggerBase** | Abstract logger with multi-writer fan-out. Two-level filtering: logger-level + per-writer level. Implements both `IAcLogWriterBase` and MS `ILogger`. |
| **AcLogWriterBase** | Abstract writer base. Own `LogLevel` loaded from appsettings by `AssemblyQualifiedName` match. Two branches: text (`AcTextLogWriterBase`) and structured (`AcLogItemWriterBase<T>`). |
| **AcTextLogWriterBase** | Text formatting base. Format: `[HH:mm:ss.fff] [AppType[0]] [Level] [Category->Method] [ThreadId] Text`. |
| **AcConsoleLogWriter** | Colored console writer. Gray=≤Trace, Cyan=Suggest, Yellow=Warning, Red=≥Error. Thread-safe via static lock. |
| **AcLogItemWriterBase\<T\>** | Abstract structured writer. ThreadPool dispatch + Mutex serialization. `Activator.CreateInstance<T>()` factory for log items. |
| **GlobalLogger** | Server-side singleton wrapping `AcGlobalLoggerBase`. Static methods for all log levels. Default category: `"GLOBAL_LOGGER"`. |
| **AcLoggerProvider\<T\>** | `ILoggerProvider` implementation with `ConcurrentDictionary<string, T>` per-category cache. Registered via `AddAcLogger<T>()` or `UseOnlyAcLogger<T>()`. |
| **IAcLogItemClient** | Structured log item DTO interface. Fields: TimeStampUtc, AppType, LogLevel, ThreadId, CategoryName, CallerName, Text, Exception, ErrorType. |
| **AcLogItemClient** | `[MessagePackObject]` implementation of `IAcLogItemClient`. Client-side transport entity. |
| **AcLogItem** | Server-side entity extending `AcLogItemClient`. `[Table("LogItem")]` with auto-generated `Id` and `LogHeaderId`. |
| **AcLoggerSignalRHub\<T\>** | Server hub receiving log items via `AddLogItem(AcLogItem)`. Simple `Hub` (not tag-based dispatch). |
| **IAcLogWriterClientBase** | Empty marker interface for client-side writers (`: IAcLogWriterBase`). |