diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 7e89811..bb0e046 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -15,7 +15,7 @@ ## SignalR 7. **Single-method transport** — all SignalR communication uses `OnReceiveMessage(tag, bytes, requestId)`. Tags are `int` constants resolved via `DynamicMethodRegistry`. Never add conventional hub methods. -8. **AcSignalRDataSource** — generic `IList` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `docs/SIGNALR_DATASOURCE.md`. Transport docs: `docs/SIGNALR.md`. +8. **AcSignalRDataSource** — generic `IList` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. Transport docs: `AyCode.Services/docs/SIGNALR.md`. 9. **JSON-in-Binary tech debt** — client→server request parameters are currently JSON inside a Binary envelope (`SignalPostJsonDataMessage`). Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects. ## Critical Warnings diff --git a/AyCode.Core.Server/AyCode.Core.Server.csproj b/AyCode.Core.Server/AyCode.Core.Server.csproj index 5364cf2..f5354fc 100644 --- a/AyCode.Core.Server/AyCode.Core.Server.csproj +++ b/AyCode.Core.Server/AyCode.Core.Server.csproj @@ -15,4 +15,8 @@ + + + + diff --git a/AyCode.Core.Server/Loggers/README.md b/AyCode.Core.Server/Loggers/README.md index be0ebd7..2c56143 100644 --- a/AyCode.Core.Server/Loggers/README.md +++ b/AyCode.Core.Server/Loggers/README.md @@ -1,7 +1,9 @@ # Loggers -Provides a singleton `GlobalLogger` for application-wide logging with multiple severity levels (Detail, Debug, Info, Warning, Suggest, Error) and support for pluggable log writers. +Server-side singleton logger for static access across the application. + +> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). For core logger and writer abstractions see [`AyCode.Core/Loggers/README.md`](../../AyCode.Core/Loggers/README.md). ## Key Files -- **`GlobalLogger.cs`** — Singleton static logger that delegates to `AcLoggerBase`. Supports category names, caller member tracking, and configurable `LogLevel` and `AppType`. +- **`GlobalLogger.cs`** — Singleton static wrapper around an internal `AcGlobalLoggerBase` (sealed `AcLoggerBase` subclass). Provides static methods for all log levels (`Detail`, `Debug`, `Info`, `Warning`, `Suggest`, `Error`, `Write`). Default category: `"GLOBAL_LOGGER"`. Reads config from `appsettings.json` like any `AcLoggerBase`. Exposes `GetWriters` and `Writer()` for accessing specific writer instances. diff --git a/AyCode.Core.Server/README.md b/AyCode.Core.Server/README.md index a5c6dc5..b952e41 100644 --- a/AyCode.Core.Server/README.md +++ b/AyCode.Core.Server/README.md @@ -2,6 +2,12 @@ Server-side extension of AyCode.Core. Provides server-specific implementations that build on the shared core library. +## Documentation + +| Document | Topic | +|---|---| +| [LOGGING_SERVER.md](docs/LOGGING_SERVER.md) | GlobalLogger singleton, server-side logging | + ## Folder Structure | Folder | Purpose | diff --git a/AyCode.Core.Server/docs/LOGGING_SERVER.md b/AyCode.Core.Server/docs/LOGGING_SERVER.md new file mode 100644 index 0000000..66af4cc --- /dev/null +++ b/AyCode.Core.Server/docs/LOGGING_SERVER.md @@ -0,0 +1,23 @@ +# Server Logging + +Server-side logging extensions. For core framework (base classes, configuration, LogLevel, ILogger bridge) see [`AyCode.Core/docs/LOGGING.md`](../../AyCode.Core/docs/LOGGING.md). For remote writers (HTTP, browser, SignalR) see [`AyCode.Services/docs/LOGGING_REMOTE.md`](../../AyCode.Services/docs/LOGGING_REMOTE.md). + +## GlobalLogger + +Server-side singleton for static access. Wraps an internal `AcGlobalLoggerBase` instance (sealed `AcLoggerBase` subclass): + +```csharp +GlobalLogger.Info("Server started"); +GlobalLogger.Error("Failed to process", ex, "MyCategory"); +GlobalLogger.Writer()?.Suggest("hint"); +``` + +Default category: `"GLOBAL_LOGGER"`. Reads config from `appsettings.json` like any other `AcLoggerBase` instance. + +All static methods mirror the `IAcLogWriterBase` contract: `Detail`, `Debug`, `Info`, `Warning`, `Suggest`, `Error`, `Write`. + +## Key Source Files + +| Component | Path | +|-----------|------| +| GlobalLogger | `Loggers/GlobalLogger.cs` | diff --git a/AyCode.Core.sln b/AyCode.Core.sln index 065674e..670e63f 100644 --- a/AyCode.Core.sln +++ b/AyCode.Core.sln @@ -60,10 +60,8 @@ EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{D4B2E9F1-A6C3-4F7E-8D5B-3E2A1C4F6B8D}" ProjectSection(SolutionItems) = preProject docs\ARCHITECTURE.md = docs\ARCHITECTURE.md - docs\BINARY_FORMAT.md = docs\BINARY_FORMAT.md docs\CONVENTIONS.md = docs\CONVENTIONS.md docs\GLOSSARY.md = docs\GLOSSARY.md - docs\SIGNALR_ARCHITECTURE.md = docs\SIGNALR_ARCHITECTURE.md EndProjectSection EndProject Global diff --git a/AyCode.Core/AyCode.Core.csproj b/AyCode.Core/AyCode.Core.csproj index 9f8c9c2..194082c 100644 --- a/AyCode.Core/AyCode.Core.csproj +++ b/AyCode.Core/AyCode.Core.csproj @@ -24,6 +24,10 @@ + + + + diff --git a/AyCode.Core/Loggers/README.md b/AyCode.Core/Loggers/README.md index b25176a..8e4beda 100644 --- a/AyCode.Core/Loggers/README.md +++ b/AyCode.Core/Loggers/README.md @@ -1,34 +1,39 @@ # Loggers -Custom logging framework with `Microsoft.Extensions.Logging` integration. Supports multiple output writers, colored console output, and structured log formatting. +Custom logging framework with multi-writer fan-out and `Microsoft.Extensions.Logging` integration. This directory contains the **core** logger and writer abstractions — the defining layer for the logging system. + +> For full architecture, configuration, all writers, and remote logging flow see [`docs/LOGGING.md`](../../docs/LOGGING.md). ## Architecture ``` IAcLoggerBase (: IAcLogWriterBase, ILogger) - └─ AcLoggerBase (abstract, delegates to writers) - └─ [concrete loggers per app] + └─ AcLoggerBase (abstract, multi-writer fan-out, ILogger bridge) + └─ [concrete loggers per consuming project] IAcLogWriterBase - └─ AcLogWriterBase (abstract, config from appsettings) - └─ AcTextLogWriterBase (abstract, text formatting) - └─ AcConsoleLogWriter (colored console output) + └─ AcLogWriterBase (abstract, per-writer config from appsettings) + ├─ AcTextLogWriterBase (abstract, text formatting) + │ └─ AcConsoleLogWriter (colored console output) + └─ [AcLogItemWriterBase in AyCode.Entities — structured writers] ``` +**Two-level filtering:** Logger has a global `LogLevel` gate; each writer has its own `LogLevel`. Both must pass for a log entry to be written. See [`docs/LOGGING.md` → Design Overview](../../docs/LOGGING.md#design-overview). + ## Key Files ### Logger Core - **`IAcLoggerBase.cs`** — Unified interface combining `IAcLogWriterBase` + `ILogger`. Exposes `GetWriters` and `Writer()`. -- **`AcLoggerBase.cs`** — Abstract logger implementing `ILogger`. Manages a list of `IAcLogWriterBase` writers. Maps MS `LogLevel` to `AcLogLevel`. Reads writer config from `appsettings.json` (`AyCode:Logger:LogWriters`). -- **`AcLoggerAdapter.cs`** — `AcLoggerProvider` implementing `ILoggerProvider` for DI integration. Extension methods: `AddAcLogger()`, `UseOnlyAcLogger()`. +- **`AcLoggerBase.cs`** — Abstract logger implementing `ILogger`. Manages `List`. Reads config from `appsettings.json` (`AyCode:Logger:{AppType, LogLevel, LogWriters[]}`). Maps MS `LogLevel` to AC `LogLevel`. Fan-out dispatch to all writers. `[Conditional("DEBUG")]` variants for all methods. +- **`AcLoggerAdapter.cs`** — `AcLoggerProvider` implementing `ILoggerProvider` with `ConcurrentDictionary` per-category cache. Extension methods: `AddAcLogger()`, `UseOnlyAcLogger()`. ### Writers -- **`IAcLogWriterBase.cs`** — Writer contract: `Detail()`, `Debug()`, `Info()`, `Warning()`, `Suggest()`, `Error()`, `Write()`. -- **`IAcLogWriterClientBase.cs`** — Marker interface for client-side writers. -- **`AcLogWriterBase.cs`** — Abstract base with config loading from appsettings. -- **`AcTextLogWriterBase.cs`** — Abstract text formatter. Output format: `[TIME] [APP] [LEVEL] [CATEGORY->METHOD] [THREADID] TEXT [ERROR]`. -- **`AcConsoleLogWriter.cs`** — Colored console writer (Gray=Trace, Cyan=Suggest, Yellow=Warning, Red=Error). Thread-safe via lock. +- **`IAcLogWriterBase.cs`** — Writer contract: `Detail()`, `Debug()`, `Info()`, `Warning()`, `Suggest()`, `Error()`, plus `Write()` overloads and `Write(IAcLogItemClient)`. +- **`IAcLogWriterClientBase.cs`** — Marker interface for client-side writers (no additional members). +- **`AcLogWriterBase.cs`** — Abstract base. Own `LogLevel` loaded from appsettings by matching `AssemblyQualifiedName`. Named methods delegate to terminal `Write(AppType, LogLevel, text, caller, category, errorType, exMessage)`. +- **`AcTextLogWriterBase.cs`** — Abstract text formatter. Format: `[HH:mm:ss.fff] [AppType[0]] [Level] [Category->Method] [ThreadId] Text [Error]`. Subclasses implement `WriteText(string, LogLevel)`. +- **`AcConsoleLogWriter.cs`** — Colored console writer. Thread-safe via `static lock`. Colors: Gray=≤Trace, White=Debug–Info, Cyan=Suggest, Yellow=Warning, Red=≥Error. ### Supporting -- **`IAcLogItemClient.cs`** — Structured log item DTO for remote transmission. -- **`LogLevel.cs`** — Byte enum: Detail(0), Trace(5), Debug(10), Info(15), Suggest(17), Warning(20), Error(25), Disabled(255). **Must match the database LogLevel table.** +- **`IAcLogItemClient.cs`** — Structured log item DTO interface for remote transmission (TimeStampUtc, AppType, LogLevel, ThreadId, CategoryName, CallerName, Text, Exception, ErrorType). +- **`LogLevel.cs`** — Byte enum: Detail(0), Trace(5), Debug(10), Info(15), Suggest(17), Warning(20), Error(25), Disabled(255). ⚠️ **Values synchronized with database `LogLevel` table — do NOT renumber.** diff --git a/AyCode.Core/README.md b/AyCode.Core/README.md index 36592d5..cb74611 100644 --- a/AyCode.Core/README.md +++ b/AyCode.Core/README.md @@ -2,6 +2,15 @@ Core library for the AyCode platform. Targets .NET 9 (set in `AyCode.Core.targets`). Provides serialization (Binary, JSON, Toon), compression, logging, validation, and shared utilities. +## Documentation + +| Document | Topic | +|---|---| +| [BINARY_FORMAT.md](docs/BINARY_FORMAT.md) | Binary wire format spec (stream layout, type markers) | +| [BINARY_FEATURES.md](docs/BINARY_FEATURES.md) | Binary features (interning, ref tracking, property ordering) | +| [BINARY_OPTIONS.md](docs/BINARY_OPTIONS.md) | Binary configuration options & presets | +| [LOGGING.md](docs/LOGGING.md) | Logging framework (hierarchy, config, ILogger bridge) | + ## Folder Structure | Folder | Purpose | README | diff --git a/AyCode.Core/docs/BINARY_FEATURES.md b/AyCode.Core/docs/BINARY_FEATURES.md new file mode 100644 index 0000000..a47c6cb --- /dev/null +++ b/AyCode.Core/docs/BINARY_FEATURES.md @@ -0,0 +1,93 @@ +# AcBinary Features + +Advanced serialization features built on top of the wire format. For core type markers and encoding see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). For configuration options and presets see [`BINARY_OPTIONS.md`](BINARY_OPTIONS.md). + +## Compact Encoding Selection + +The serializer applies compact encodings automatically: + +| Data | Condition | Encoding | Savings | +|------|-----------|----------|---------| +| Integer | −16 ≤ v ≤ 47 | TinyInt (1 byte) | 2–5 bytes | +| String | ≤31 bytes, ASCII | FixStr (1+N bytes) | 1 byte (no length prefix) | +| Object | type index < 64 | FixObj (1 byte) | 1–5 bytes (no VarUInt index) | +| String | empty | StringEmpty (1 byte) | 1+ bytes | +| Bool | — | True/False (1 byte) | no payload | + +## String Interning Protocol + +Controls deduplication of repeated string values. + +**Modes** (`StringInterningMode`): +- `None` — all strings inline, no overhead +- `Attribute` — only `[AcStringIntern]` properties interned (default) +- `All` — all strings within length limits interned + +**Length limits:** `MinStringInternLength=4`, `MaxStringInternLength=64` (configurable). + +**Wire protocol:** +1. Serializer pre-scans all eligible strings to build a plan (which strings repeat) +2. First occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]` +3. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]` +4. Single-occurrence strings: written as normal `String`/`FixStr` (no interning overhead) + +## Reference Tracking + +Prevents infinite loops and preserves object identity for repeated references. + +**Modes** (`ReferenceHandlingMode`): +- `None` — no tracking (fastest, use when graph is a tree) +- `OnlyId` — track only `IId` objects (matched by ID value) +- `All` — track all reference types (two-phase scan required) + +**Two-phase process:** +1. **Scan pass** (`ScanPass.cs`) — walks the object graph, detects multi-referenced objects and repeated strings. Builds a `WriteDuplicateEntry[]` array (the "write plan") containing `VisitIndex`, `CacheMapIndex`, `IsFirst`, and `Value` for each duplicate. +2. **Sort** — write plan entries are sorted by `VisitIndex` to match the write pass traversal order. +3. **Serialize pass** — consumes the sorted write plan via `TryConsumeWritePlanEntry()`. A cursor (`_nextWritePlanVisitIndex`) advances through the plan in O(1) — no dictionary lookups during serialization. + +**Wire protocol:** +- First occurrence: `[ObjectRefFirst(70)] [VarUInt refCacheIndex] [object body...]` +- Subsequent: `[ObjectRef(65)] [VarUInt refCacheIndex]` + +**Example — same object referenced twice:** + +``` +Input: { Users: [userA, userA] } (same instance) + +Scan pass → WritePlan: + [{VisitIndex:2, CacheMapIndex:0, IsFirst:true}, + {VisitIndex:3, CacheMapIndex:0, IsFirst:false}] + +Wire output (Compact mode, ReferenceHandling=All): + [version=1] [flags=0x96] [VarUInt cacheCount=1] ← header + [FixObj(0)] ← root object + [Array(66)] [VarUInt(2)] ← Users array, 2 elements + [ObjectRefFirst(70)] [VarUInt(0)] [props...] ← userA, 1st occurrence + [ObjectRef(65)] [VarUInt(0)] ← userA, 2nd (2 bytes only) +``` + +## Property Ordering + +Properties are serialized in a deterministic order defined by `TypeMetadataBase.GetUnfilteredProperties()`: + +1. Walk the inheritance chain from **derived → base** (`currentType.BaseType` loop) +2. At each level, collect declared public instance properties +3. Sort **alphabetically** (`StringComparer.Ordinal`) within each level +4. Result: **base properties first, then derived, alphabetical within each level** + +This order is stable across serializer/deserializer as long as the type hierarchy doesn't change. + +### Cross-Type Deserialization (UseMetadata) + +When `UseMetadata=true`, property name hashes (FNV-1a via `FnvHash.ComputeString`) are written per type, enabling schema evolution: + +- **Serializer** writes property hashes in the metadata section (`ObjectWithMetadata(69)`) +- **Deserializer** builds an index mapping array (`GetIndexMapping()`) that maps source property indices to destination indices by matching FNV-1a hashes +- This allows deserialization even when source and destination types have different property sets or ordering + +When `UseMetadata=false`, properties are matched by **positional index only** — source and destination must have identical property layouts. + +**Edge cases:** +- **Hash collision** (`CheckDuplicatePropName=true`, default): throws `InvalidOperationException`. When `false`: collision silently ignored — risk of data corruption. +- **Source has unknown property** (not in destination): silently skipped via `SkipValue()`, no error. +- **Destination has extra property** (not in source): left at default value (new instance) or unchanged (populate mode). diff --git a/AyCode.Core/docs/BINARY_FORMAT.md b/AyCode.Core/docs/BINARY_FORMAT.md new file mode 100644 index 0000000..5b6e934 --- /dev/null +++ b/AyCode.Core/docs/BINARY_FORMAT.md @@ -0,0 +1,167 @@ +# AcBinary Wire Format + +Complete wire format specification for the AcBinary serializer. Source of truth: [`Serializers/Binaries/BinaryTypeCode.cs`](../Serializers/Binaries/BinaryTypeCode.cs). + +> For advanced features (compact encoding, string interning, reference tracking, property ordering) see [`BINARY_FEATURES.md`](BINARY_FEATURES.md). +> For configuration options, presets, and option interactions see [`BINARY_OPTIONS.md`](BINARY_OPTIONS.md). + +## Stream Layout + +``` +[version : 1 byte] [flags : 1 byte] [cacheCount : VarUInt?] [payload...] +``` + +- **version** — `FormatVersion = 1` (current). +- **flags** — See [Header Flags](#header-flags). +- **cacheCount** — Present only when `HeaderFlag_HasCacheCount` is set. Number of type wrapper slots used by serializer. + +## Header Flags + +The flags byte uses `0x90` (144) as base with bit flags in the lower nibble: + +| Bit | Mask | Flag | Meaning | +|-----|------|------|---------| +| 0 | `0x01` | Metadata | Property hash metadata included (cross-type deserialization) | +| 1 | `0x02` | RefHandling_OnlyId | Reference tracking for `IId` objects only | +| 2 | `0x04` | RefHandling_All | Reference tracking for all objects (always combined with bit 1) | +| 3 | `0x08` | HasCacheCount | VarUInt cache count follows the flags byte | + +**Reference handling modes:** None = `0x00`, OnlyId = `0x02`, All = `0x06` (bits 1+2). + +## Variable-Length Encoding + +### VarUInt (unsigned) + +LEB128: 7 data bits per byte, MSB = continuation flag. + +``` +value < 128 → 1 byte [0xxxxxxx] +value < 16384 → 2 bytes [1xxxxxxx] [0xxxxxxx] +value < 2097152 → 3 bytes ... +(max 5 bytes for uint32) +``` + +### VarInt (signed) + +ZigZag encoding maps signed to unsigned, then LEB128: + +``` +encode: (value << 1) ^ (value >> 31) +decode: (raw >> 1) ^ -(raw & 1) +``` + +Maps: `0 → 0`, `-1 → 1`, `1 → 2`, `-2 → 3`, etc. + +### VarULong (unsigned 64-bit) + +Same LEB128 encoding, max 10 bytes for uint64. + +## Type Markers + +All markers defined in `BinaryTypeCode.cs`. `SlotCount = 64`. + +### FixObj (0–63) + +Single-byte object type. The marker byte **is** the type slot index — no additional type identifier needed. + +``` +[FixObj(N)] [properties...] +``` + +**Slot allocation:** Slots 0–63 are reserved for runtime polymorphic types, assigned dynamically on first encounter during serialization. Source-generated (SGen) types receive slots starting at 64+ via `AllocateWrapperSlot()` (sequential, `Interlocked.Increment`). SGen slots are compile-time stable; runtime slots depend on serialization order. + +### Complex Types (64–71) + +| Code | Name | Wire format | +|------|------|-------------| +| 64 | Object | `[64] [VarUInt typeIndex] [properties...]` | +| 65 | ObjectRef | `[65] [VarUInt refCacheIndex]` | +| 66 | Array | `[66] [VarUInt count] [elements...]` | +| 67 | Dictionary | `[67] [VarUInt count] [key, value pairs...]` | +| 68 | ByteArray | `[68] [VarUInt length] [raw bytes]` | +| 69 | ObjectWithMetadata | `[69] [VarUInt typeIndex] [VarUInt hashCount] [hashes...] [properties...]` | +| 70 | ObjectRefFirst | `[70] [VarUInt refCacheIndex] [object body...]` | +| 71 | ObjectWithMetadataRefFirst | `[71] [VarUInt refCacheIndex] [metadata + properties...]` | + +### Polymorphic Types (72–75) + +Used when runtime type differs from declared property type and `UseMetadata=false`. + +| Code | Name | Wire format | +|------|------|-------------| +| 72 | ObjectWithTypeName | `[72] [UTF8 typeName] [inner marker] [body...]` — prefix, inner Object/Array/Dict follows | +| 73 | ObjectWithTypeNameRefFirst | `[73] [UTF8 typeName] [VarUInt refCacheIndex] [properties...]` — combined, no inner marker | +| 74 | ObjectWithTypeIndex | `[74] [VarUInt typeIndex] [inner marker] [body...]` — prefix | +| 75 | ObjectWithTypeIndexRefFirst | `[75] [VarUInt typeIndex] [VarUInt refCacheIndex] [properties...]` — combined | + +Second occurrence of a referenced polymorphic object uses plain `ObjectRef(65)` — no polymorphic prefix needed. + +### Primitives (76–90) + +| Code | Name | Wire format | +|------|------|-------------| +| 76 | Null | `[76]` — no payload | +| 77 | True | `[77]` — no payload | +| 78 | False | `[78]` — no payload | +| 79 | Int8 | `[79] [1 byte]` | +| 80 | UInt8 | `[80] [1 byte]` | +| 81 | Int16 | `[81] [VarInt]` | +| 82 | UInt16 | `[82] [VarUInt]` | +| 83 | Int32 | `[83] [VarInt]` | +| 84 | UInt32 | `[84] [VarUInt]` | +| 85 | Int64 | `[85] [VarLong]` | +| 86 | UInt64 | `[86] [VarULong]` | +| 87 | Float32 | `[87] [4 bytes IEEE 754]` | +| 88 | Float64 | `[88] [8 bytes IEEE 754]` | +| 89 | Decimal | `[89] [16 bytes]` | +| 90 | Char | `[90] [VarUInt]` | + +### Strings (91–94) + +| Code | Name | Wire format | +|------|------|-------------| +| 91 | String | `[91] [VarUInt byteLength] [UTF-8 bytes]` | +| 92 | StringInterned | `[92] [VarUInt cacheIndex]` — 2nd+ occurrence | +| 93 | StringEmpty | `[93]` — no payload | +| 94 | StringInternFirst | `[94] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]` — 1st occurrence | + +### Date/Time (95–98) + +| Code | Name | Wire format | +|------|------|-------------| +| 95 | DateTime | `[95] [8 bytes ticks]` | +| 96 | DateTimeOffset | `[96] [8 bytes ticks] [VarInt offsetMinutes]` | +| 97 | TimeSpan | `[97] [VarLong ticks]` | +| 98 | Guid | `[98] [16 bytes]` | + +### Other Markers + +| Code | Name | Wire format | +|------|------|-------------| +| 99 | Enum | `[99] [VarInt underlyingValue]` | +| 100 | MetadataHeader | Legacy: implies `RefHandling=true` + metadata present | +| 101 | NoMetadataHeader | Legacy: implies `RefHandling=true`, no metadata | +| 102 | PropertySkip | `[102]` — marks skipped property (default/null value) | + +### FixStr (103–134) + +Short ASCII strings encoded in a single marker byte + raw bytes (no length prefix): + +``` +[FixStrBase + byteLength] [ASCII bytes] +``` + +- Length range: 0–31 bytes (`FixStrBase=103`, `FixStrMax=134`) +- Saves 1 byte vs `String` marker + VarUInt length +- Falls back to `String(91)` if content is non-ASCII + +### TinyInt (192–255) + +Single-byte integer encoding for small values: + +``` +value = marker - 192 - 16 (range: -16 to 47) +marker = value + 16 + 192 (64 values total) +``` + +Saves 2+ bytes vs `Int32(83)` + VarInt for frequently occurring small integers. diff --git a/AyCode.Core/docs/BINARY_OPTIONS.md b/AyCode.Core/docs/BINARY_OPTIONS.md new file mode 100644 index 0000000..426504c --- /dev/null +++ b/AyCode.Core/docs/BINARY_OPTIONS.md @@ -0,0 +1,154 @@ +# AcBinary Configuration + +Configuration options, presets, and option interactions for `AcBinarySerializerOptions`. For wire format see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). For features (interning, ref tracking, property ordering) see [`BINARY_FEATURES.md`](BINARY_FEATURES.md). + +## WireMode + +| Value | Integers | Strings | Output size | Speed | +|-------|----------|---------|-------------|-------| +| `Compact` (default) | VarInt/VarUInt (1–5 bytes) | UTF-8 with speculative ASCII fast path | Smaller | Slightly slower | +| `Fast` | Fixed-width raw bytes (4/8 bytes) | UTF-16 memcpy (`charCount * 2` bytes) | Larger | Fastest encode/decode | + +**Format difference for strings:** +- Compact: `[VarUInt byteLength] [UTF-8 bytes]` — speculative ASCII (1 pass if all ASCII, rewind+UTF-8 fallback otherwise) +- Fast: `[VarUInt charCount] [raw UTF-16 bytes]` — zero-encoding memcpy + +**Code branch:** `context.FastWire` flag set at `context.Reset()`. Checked in `WriteStringUtf8()` and integer write methods. FixStr optimization is skipped in Fast mode (UTF-8 specific). + +## ReferenceHandling + +| Value | Tracked objects | Scan pass | Header flags | Wire markers | +|-------|----------------|-----------|--------------|-------------| +| `None` | Nothing | Skipped | `0x00` | Standard object markers only | +| `OnlyId` | `IId` objects only (by ID value) | Partial | `0x02` | `ObjectRefFirst(70)` + `ObjectRef(65)` | +| `All` (default) | All reference types | Full graph walk | `0x06` | `ObjectRefFirst(70)` + `ObjectRef(65)` | + +**Format impact:** When enabled, multi-referenced objects are written once with `ObjectRefFirst(70) + VarUInt(refCacheIndex)` on first encounter, then replaced by `ObjectRef(65) + VarUInt(refCacheIndex)` on subsequent encounters. Header `HasCacheCount` flag is set and cache count written. + +**Interaction with `ThrowOnCircularReference` (default: `true`):** +- `true` + ref handling enabled: all objects tracked for cycle detection, throws `InvalidOperationException` on circular reference +- `false` + ref handling enabled: only IId types tracked for deduplication, non-IId circular refs silently truncated at `MaxDepth` + +## UseMetadata + +| Value | Wire markers | Property matching | Overhead | +|-------|-------------|-------------------|----------| +| `false` (default) | `FixObj`/`Object` | Positional index only — types must match | None | +| `true` | `ObjectWithMetadata(69)` / `ObjectWithMetadataRefFirst(71)` | FNV-1a property name hashes | 4 bytes per property per type | + +**Format impact:** When enabled, each type's first occurrence writes `[VarUInt hashCount] [FNV-1a hash × N]` before properties. Deserializer uses hashes to build source→destination index mapping, enabling cross-type deserialization (different property sets/ordering). + +**Code branch:** `context.UseMetadata` controls whether `ObjectWithMetadata(69)` or plain `Object(64)` markers are used. When `false`, `IsDirectObjectWrite=true` allows source-generated writers to bypass `WriteObject` entirely and inline property writes. + +**Related:** `CheckDuplicatePropName` (default: `true`) — throws if FNV-1a hash collision detected between property names of the same type. Disable in production for performance. + +## UseStringInterning + +| Value | Eligible strings | Scan overhead | Wire markers | +|-------|-----------------|---------------|-------------| +| `None` | Nothing | None | `String(91)` / `FixStr` only | +| `Attribute` (default) | Properties with `[AcStringIntern(true)]` | Scans marked properties | `StringInternFirst(94)` + `StringInterned(92)` | +| `All` | All strings within length limits | Scans all strings | `StringInternFirst(94)` + `StringInterned(92)` | + +**Length limits:** `MinStringInternLength` (default: 4) and `MaxStringInternLength` (default: 64, 0=unlimited). Strings outside this range are always written inline. + +**Format impact:** Interned strings on first occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [string data]`. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]` (1–2 bytes vs full string). Single-occurrence strings are never interned — no overhead for unique strings. + +**Code branch:** `context.StringInternEligible` flag set per-property before `WriteString`. Scan pass builds a `WriteDuplicateEntry[]` plan; write pass consumes it via cursor. + +## MaxDepth + +| Value | Behavior | +|-------|----------| +| `255` (default) | Effectively unlimited nesting | +| `0` | Root level only — nested objects/collections written as `Null(76)` | +| `N` | Objects deeper than N levels written as `Null(76)` | + +**Format impact:** Depth-exceeded values appear as `Null(76)` in the stream — indistinguishable from actual null values. No special marker. + +**Code branch:** Checked at entry of every object/collection write: `if (depth > MaxDepth) { WriteByte(Null); return; }`. + +## UseCompression + +| Value | Method | Granularity | Memory | +|-------|--------|-------------|--------| +| `None` (default) | No compression | — | — | +| `Block` | LZ4 single block | Entire payload | Full buffer in memory | +| `BlockArray` | LZ4 chunked | 64KB chunks | Streaming-friendly, lower peak memory | + +**Format impact:** Compression is applied **post-serialization** as a transparent wrapper — the inner wire format is unchanged. Both modes are pure managed C# (WASM-compatible, no native dependencies). + +**Code branch:** Applied in `AcBinarySerializer.Serialize()` after the serialization context produces the raw buffer: `if (UseCompression != None) Lz4.Compress(buffer, mode)`. Decompression is automatic on deserialize. + +## PropertyFilter + +Optional delegate `BinaryPropertyFilter?` (default: `null`). When set, invoked for each property to decide inclusion. + +``` +delegate bool BinaryPropertyFilter(in BinaryPropertyFilterContext context); +``` + +**BinaryPropertyFilterContext fields:** `DeclaringType`, `PropertyName`, `PropertyType`, `Instance` (null during metadata phase), `IsMetadataPhase`, `GetValue()` (lazy). + +**Format impact:** Excluded properties are completely absent from the stream — no marker, no placeholder. The deserializer must use `UseMetadata=true` or identical filter to correctly match property indices. + +**Code branch:** `context.HasPropertyFilter` checked in `ShouldSerializeProperty()`. Called twice: once during metadata registration (`Instance=null`), once during write phase. + +## PropertyMapper + +Optional delegate `PropertyMapperDelegate?` (default: `null`) for cross-type deserialization property remapping. + +``` +delegate PropertyInfo? PropertyMapperDelegate(PropertyInfo sourceProperty, Type destinationType); +``` + +**Purpose:** Maps properties between different class hierarchies (renamed properties, external DTOs). Result is cached — zero overhead on same-type operations (`Deserialize`). + +## WASM Options + +| Option | Default | Purpose | +|--------|---------|---------| +| `IsWasm` | `OperatingSystem.IsBrowser()` | Auto-detect WASM environment | +| `UseStringCaching` | follows `IsWasm` | Cache short strings during deserialization to reduce GC pressure | +| `MaxCachedStringLength` | 64 | Max string length to cache | + +**Format impact:** None — these are deserialization-only optimizations. When `UseStringCaching=true`, the deserializer maintains an intern cache for strings ≤ `MaxCachedStringLength` chars. Disabled automatically when `StringInternFirst` marker is encountered (interning takes precedence). + +## Other Options + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `UseGeneratedCode` | bool | `true` | Use source-generated writers/readers when available | +| `InitialBufferCapacity` | int | 4096 | Starting buffer size (bytes) for serialization output | +| `RemoveOrphanedItems` | bool | `false` | During `PopulateMerge`: remove destination collection items with no matching source ID | +| `UseAsync` | bool | `false` | Async context pool return via ThreadPool. Auto-disabled in WASM and when `ReferenceHandling=None` | +| `MaxContextPoolSize` | int | 8 | Max serialization contexts kept in pool | + +## Presets + +| Preset | WireMode | Metadata | StringInterning | RefHandling | MaxDepth | Compression | Other | +|--------|----------|----------|-----------------|-------------|----------|-------------|-------| +| `Default` | Compact | false | Attribute | All | 255 | None | — | +| `FastMode` | Compact | false | None | None | 255 | None | No scan pass | +| `ShallowCopy` | Compact | false | None | None | **0** | None | Root level only | +| `WasmOptimized` | Compact | false | Attribute | All | 255 | None | +StringCaching | +| `WithoutReferenceHandling` | Compact | false | Attribute | **None** | 255 | None | No scan pass | +| `WithoutMetadata` | Compact | **false** | Attribute | All | 255 | None | — | + +**Performance implication of presets:** +- `Default` / `WasmOptimized` — two-phase (scan + serialize) due to `ReferenceHandling=All` +- `FastMode` / `ShallowCopy` — single-phase (no scan pass) since both interning and refs are disabled +- The scan pass adds ~20-30% overhead; disable it when the object graph is a simple tree + +## Option Interactions + +Key interdependencies that affect which code branches execute: + +| Combination | Effect | +|-------------|--------| +| `ReferenceHandling=None` + `UseStringInterning=None` | **No scan pass** — fastest path, single-phase serialization | +| `ReferenceHandling=All` + `UseMetadata=true` | Uses `ObjectWithMetadataRefFirst(71)` marker — combined ref + metadata | +| `UseMetadata=false` + `UseGeneratedCode=true` | `IsDirectObjectWrite=true` — generated code inlines property writes, bypasses `WriteObject` | +| `UseMetadata=true` + `PropertyFilter` set | Filter invoked twice (metadata phase + write phase); filter results must be stable | +| `WireMode=Fast` + `UseStringInterning!=None` | Interned strings still use the fast string path (UTF-16 for first occurrence, VarUInt index for subsequent) | +| `UseCompression!=None` + any other option | Compression is orthogonal — applied post-serialization, inner format unchanged | diff --git a/AyCode.Core/docs/LOGGING.md b/AyCode.Core/docs/LOGGING.md new file mode 100644 index 0000000..c2d4e9b --- /dev/null +++ b/AyCode.Core/docs/LOGGING.md @@ -0,0 +1,243 @@ +# Logging + +Custom logging framework with multi-writer fan-out and `Microsoft.Extensions.Logging` integration. Source: `Loggers/` in this project. + +> For server-side GlobalLogger see [`AyCode.Core.Server/docs/LOGGING_SERVER.md`](../../AyCode.Core.Server/docs/LOGGING_SERVER.md). +> For remote writers (HTTP, browser console, SignalR) see [`AyCode.Services/docs/LOGGING_REMOTE.md`](../../AyCode.Services/docs/LOGGING_REMOTE.md). + +## Design Overview + +A logger holds a list of writers. Every log call fans out to all writers that pass the level filter. Two independent level gates control what gets written: + +``` +logger.Info("msg") + │ + ├─ Gate 1: Logger.LogLevel <= Info? ← global minimum + │ NO → discard + │ YES ↓ + ├─ Writer[0].Write(...) + │ └─ Gate 2: Writer.LogLevel <= Info? ← per-writer minimum + │ NO → discard + │ YES → write to output + ├─ Writer[1].Write(...) + │ └─ Gate 2: ... + └─ Writer[N].Write(...) + └─ Gate 2: ... +``` + +## Class Hierarchy + +``` +IAcLogWriterBase (writer contract) + └─ AcLogWriterBase (abstract, config from appsettings) + ├─ AcTextLogWriterBase (abstract, text formatting) + │ ├─ AcConsoleLogWriter (colored console output) + │ └─ AcBrowserConsoleLogWriter (AyCode.Services — Blazor JSInterop console) + └─ AcLogItemWriterBase (AyCode.Entities — abstract, structured → ThreadPool + Mutex) + ├─ AcDbLogItemWriter (AyCode.Database — EF Core database writer) + ├─ AcHttpClientLogItemWriter (AyCode.Services — HTTP POST as JSON) + └─ AcSignaRClientLogItemWriter (AyCode.Services — SignalR hub transport) + +IAcLoggerBase (: IAcLogWriterBase, ILogger) + └─ AcLoggerBase (abstract, multi-writer fan-out) + └─ AcGlobalLoggerBase (sealed, used by GlobalLogger singleton) + └─ [concrete loggers in consuming projects] + +IAcLogWriterClientBase (: IAcLogWriterBase) (marker for client-side writers) +``` + +**Two writer branches:** + +| Branch | Base class | Output | Concurrency | +|--------|-----------|--------|-------------| +| **Text** | `AcTextLogWriterBase` | Formatted string → `WriteText(string, LogLevel)` | Depends on subclass (Console uses `lock`) | +| **Structured** | `AcLogItemWriterBase` | `TLogItem` object → `WriteLogItemCallback(TLogItem)` | `TaskHelper.RunOnThreadPool` + `Mutex` | + +## LogLevel + +```csharp +public enum LogLevel : byte +{ + Detail = 0, + Trace = 5, + Debug = 10, + Info = 15, + Suggest = 17, + Warning = 20, + Error = 25, + Disabled = 255, +} +``` + +⚠️ **Values are synchronized with the database `LogLevel` table.** Do NOT renumber. + +Comparison is `<=`: a logger/writer with `LogLevel = Info` will process `Info`, `Suggest`, `Warning`, `Error` (anything ≥ 15). + +## Configuration + +All config lives under `AyCode:Logger` in `appsettings.json`: + +```json +{ + "AyCode": { + "Logger": { + "AppType": "Server", + "LogLevel": "Debug", + "LogWriters": [ + { + "LogWriterType": "MyApp.Loggers.MyConsoleLogWriter, MyApp", + "LogLevel": "Info" + }, + { + "LogWriterType": "MyApp.Loggers.MyDbLogWriter, MyApp", + "LogLevel": "Warning" + } + ] + } + } +} +``` + +| Key | Type | Purpose | +|-----|------|---------| +| `AppType` | `AppType` enum | Identifies the application (Server, Web, etc.). Stamped on every log entry. | +| `LogLevel` | `LogLevel` enum | Global minimum — the logger's own gate. | +| `LogWriters[]` | array | One entry per writer. Each has `LogWriterType` (AssemblyQualifiedName) and `LogLevel` (per-writer gate). | + +### Writer Instantiation + +`AcLoggerBase` constructor iterates `LogWriters[]` and calls: + +```csharp +Activator.CreateInstance(logWriterType, AppType, logWriterLogLevel, CategoryName) +``` + +Each writer's constructor signature must accept `(AppType, LogLevel, string?)`. + +### Writer Config Self-Lookup + +`AcLogWriterBase` also reads its own `LogLevel` from config by matching its `AssemblyQualifiedName` against the `LogWriterType` entries. This means a writer instantiated manually (not via config) can still pick up config values if a matching entry exists. + +## Core Components + +### AcLoggerBase + +Abstract logger implementing both `IAcLogWriterBase` and `ILogger`. Central responsibilities: + +1. **Writer management** — `List LogWriters`, `GetWriters`, `Writer()` +2. **Fan-out dispatch** — Named methods (`Detail`, `Debug`, `Info`, `Warning`, `Suggest`, `Error`) check `LogLevel` then `ForEach` writers +3. **Write overloads** — Terminal: `Write(AppType, LogLevel, text, caller, category, errorType, exMessage)` +4. **Write(IAcLogItemClient)** — Accepts structured log items (from remote clients) +5. **ILogger bridge** — `Log`, `IsEnabled`, `BeginScope` (no-op `NullScope`) +6. **[Conditional("DEBUG")]** — Every named method has a `*Conditional` variant stripped from Release builds + +**Constructors:** + +| Signature | Behavior | +|-----------|----------| +| `(string? categoryName)` | Reads config from `appsettings.json`, creates writers via `Activator.CreateInstance` | +| `(string? categoryName, params IAcLogWriterBase[])` | Reads `AppType` + `LogLevel` from config, uses provided writers | +| `(AppType, LogLevel, string?, params IAcLogWriterBase[])` | Fully manual, no config reading | + +### AcLogWriterBase + +Abstract writer base. Each writer has its own `LogLevel` and `AppType`. Named methods (`Detail`, `Debug`, etc.) delegate to the terminal `Write(AppType, LogLevel, text, caller, category, errorType, exMessage)` which subclasses override. + +The base `Write` throws `NotImplementedException` — subclasses **must** override either the 7-parameter `Write` (for text writers) or `Write(IAcLogItemClient)` (for structured writers), or both. + +### AcTextLogWriterBase + +Text formatting base for human-readable output. Overrides `Write` to format via `GetDiagnosticText` then calls abstract `WriteText(string, LogLevel)`. + +**Output format:** +``` +[HH:mm:ss.fff] [S] [Level] [Category->Method] [ThreadId] Text +[ERROR_TYPE]: exception details... +``` + +- Timestamp: `DateTime.UtcNow.ToLocalTime()` (local display, UTC storage) +- `[S]` — First character of `AppType` (e.g., `S`=Server, `W`=Web) +- Level: Left-padded to 9 chars +- Category→Method: Left-padded to 54 chars +- Error text on new line only if `errorType` is non-empty + +### AcConsoleLogWriter + +Colored console output. Thread-safe via `static readonly object ForWriterLock`. + +| LogLevel | Color | +|----------|-------| +| ≤ Trace | Gray | +| Debug – Info | White (default) | +| Suggest | Cyan | +| Warning | Yellow | +| ≥ Error | Red (with extra newlines) | + +## Microsoft.Extensions.Logging Bridge + +### ILogger Bridge + +`AcLoggerBase` implements `ILogger` directly. The `Log` method maps MS log levels to AC methods: + +| Microsoft.Extensions.Logging.LogLevel | AyCode LogLevel | Method called | +|---------------------------------------|-----------------|---------------| +| Trace | Detail | `Detail()` | +| Debug | Debug | `Debug()` | +| Information | Info | `Info()` | +| Warning | Warning | `Warning()` | +| Error | Error | `Error()` | +| Critical | Error | `Error("[CRITICAL] ...")` | +| None | Disabled | — (ignored) | + +`BeginScope` returns a no-op `NullScope` (scopes not supported). + +**ShortenCategoryNames** (default: `true`): When MS logging provides fully-qualified type names as categories (e.g., `Microsoft.AspNetCore.SignalR.HubConnectionHandler`), shortens to just the class name (`HubConnectionHandler`). + +### ILoggerProvider + +`AcLoggerProvider` implements `ILoggerProvider` with a `ConcurrentDictionary` per-category cache. Factory function provided at registration. + +**Extension methods:** + +```csharp +// Add alongside default providers +builder.Logging.AddAcLogger(categoryName => new MyLogger(categoryName)); + +// Replace all providers with only AcLogger +builder.Logging.UseOnlyAcLogger(categoryName => new MyLogger(categoryName)); +``` + +## Log Item Entity Hierarchy + +Log items flow across projects as a chain of types: + +``` +IAcLogItemClient (AyCode.Core — DTO interface) + └─ AcLogItemClient (AyCode.Entities — [MessagePackObject]) + └─ AcLogItem (AyCode.Entities.Server — [Table("LogItem")]) +``` + +**IAcLogItemClient Fields:** TimeStampUtc, AppType, LogLevel, ThreadId, CategoryName, CallerName, Text, ErrorType, Exception. + +## Patterns + +### [Conditional("DEBUG")] Pattern + +Every named log method has a `*Conditional` counterpart (e.g., `InfoConditional`, `ErrorConditional`) decorated with `[Conditional("DEBUG")]`. These are completely stripped from Release builds by the compiler. Both `AcLoggerBase` and `AcLogWriterBase` provide these. Use them for development-only diagnostics that should have zero cost in production. + +### CallerMemberName Auto-Capture + +All named log methods use `[CallerMemberName] string? memberName = null`. The compiler auto-fills the calling method name. For `ILogger.Log` calls (from MS logging), the `EventId.Name` is used as member name if available, otherwise defaults to `"Log"`. + +## Key Source Files + +| Component | Path | +|-----------|------| +| Logger base | `Loggers/AcLoggerBase.cs` | +| Writer base | `Loggers/AcLogWriterBase.cs` | +| Text writer base | `Loggers/AcTextLogWriterBase.cs` | +| Console writer | `Loggers/AcConsoleLogWriter.cs` | +| ILoggerProvider | `Loggers/AcLoggerAdapter.cs` | +| LogLevel enum | `Loggers/LogLevel.cs` | +| Log item DTO interface | `Loggers/IAcLogItemClient.cs` | +| Client marker | `Loggers/IAcLogWriterClientBase.cs` | diff --git a/AyCode.Database/DbContexts/Loggers/README.md b/AyCode.Database/DbContexts/Loggers/README.md index 325ca68..f7a35be 100644 --- a/AyCode.Database/DbContexts/Loggers/README.md +++ b/AyCode.Database/DbContexts/Loggers/README.md @@ -1,8 +1,10 @@ # DbContexts / Loggers -Logger-specific EF Core DbContext with NoTracking query behavior for read performance. +Logger-specific EF Core DbContext with `NoTracking` query behavior for read performance. Used by `AcDbLogItemWriter` to persist log items. + +> For full logging architecture see [`docs/LOGGING.md`](../../../../docs/LOGGING.md). ## Key Files -- **`IAcLoggerDbContextBase.cs`** — Interface for log item DbSet. -- **`AcLoggerDbContextBase.cs`** — Implementation with NoTracking default, optimized for log reads. +- **`IAcLoggerDbContextBase.cs`** — Interface extending `IAcLogItemDbSetBase` for log item DbSet. +- **`AcLoggerDbContextBase.cs`** — Generic implementation (`AcDbContextBase` subclass). Configures `QueryTrackingBehavior.NoTracking` for read performance. Exposes `DbSet LogItems`. diff --git a/AyCode.Database/DbSets/Loggers/README.md b/AyCode.Database/DbSets/Loggers/README.md index a815a47..e6d16e8 100644 --- a/AyCode.Database/DbSets/Loggers/README.md +++ b/AyCode.Database/DbSets/Loggers/README.md @@ -1,7 +1,9 @@ # DbSets / Loggers -Log item DbSet interface. +Log item DbSet interface for EF Core log storage. + +> For full logging architecture see [`docs/LOGGING.md`](../../../../docs/LOGGING.md). ## Key Files -- **`IAcLogItemDbSetBase.cs`** — Interface for LogItem DbSet. +- **`IAcLogItemDbSetBase.cs`** — Interface exposing `DbSet` for log items. Implemented by `AcLoggerDbContextBase`. diff --git a/AyCode.Entities.Server/LogItems/README.md b/AyCode.Entities.Server/LogItems/README.md index d269ca0..3d82159 100644 --- a/AyCode.Entities.Server/LogItems/README.md +++ b/AyCode.Entities.Server/LogItems/README.md @@ -1,8 +1,10 @@ # LogItems -Server-side log item entity and interface, extending the client-side `AcLogItemClient` with a database-mapped identity and header reference. +Server-side log item entity and interface, extending the client-side `AcLogItemClient` with database-mapped identity and header reference. + +> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). For client-side entity see [`AyCode.Entities/LogItems/README.md`](../../AyCode.Entities/LogItems/README.md). ## Key Files -- **`IAcLogItem.cs`** — Interface extending `IAcLogItemClient` and `IEntityInt`, adding a `LogHeaderId` property. -- **`AcLogItem.cs`** — Entity class mapped to the `LogItem` database table with MessagePack serialization. Provides `Id` (auto-generated) and `LogHeaderId`. +- **`IAcLogItem.cs`** — Interface extending `IAcLogItemClient` + `IEntityInt`, adding `LogHeaderId` property. +- **`AcLogItem.cs`** — `[Table("LogItem")]` entity extending `AcLogItemClient`. MessagePack keys: `Id` (Key 55, auto-generated PK), `LogHeaderId` (Key 60). Used by `AcDbLogItemWriter` for EF Core persistence. diff --git a/AyCode.Entities/LogItems/README.md b/AyCode.Entities/LogItems/README.md index 4422c63..edc60a0 100644 --- a/AyCode.Entities/LogItems/README.md +++ b/AyCode.Entities/LogItems/README.md @@ -1,7 +1,9 @@ # LogItems -Client-side log item entity used for structured logging. Serialized with MessagePack for efficient transport. +Client-side log item entity used for structured logging. Serialized with MessagePack for efficient transport over SignalR. + +> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). ## Key Files -- **`AcLogItemClient.cs`** — Implements `IAcLogItemClient` with fields for `AppType`, `LogLevel`, `ThreadId`, `CategoryName`, `CallerName`, `Text`, and exception details. Uses `[MessagePackObject]` serialization with explicit key indices. +- **`AcLogItemClient.cs`** — Implements `IAcLogItemClient`. `[MessagePackObject]` with explicit key indices (5, 15, 20, 25, 30, 35, 40, 45, 50). Fields: `TimeStampUtc`, `AppType`, `LogLevel`, `ThreadId`, `CategoryName`, `CallerName`, `Text`, `ErrorType`, `Exception`. Extended by `AcLogItem` on the server side. diff --git a/AyCode.Services.Server/AyCode.Services.Server.csproj b/AyCode.Services.Server/AyCode.Services.Server.csproj index f837aff..4f96509 100644 --- a/AyCode.Services.Server/AyCode.Services.Server.csproj +++ b/AyCode.Services.Server/AyCode.Services.Server.csproj @@ -33,4 +33,8 @@ + + + + diff --git a/AyCode.Services.Server/README.md b/AyCode.Services.Server/README.md index 3147e75..82bc5a2 100644 --- a/AyCode.Services.Server/README.md +++ b/AyCode.Services.Server/README.md @@ -2,6 +2,13 @@ Server-side service implementations: JWT authentication, SendGrid email delivery, SignalR hub infrastructure with binary protocol, session management, and change-tracked data sources. +## Documentation + +| Document | Topic | +|---|---| +| [SIGNALR_SERVER.md](docs/SIGNALR_SERVER.md) | Server-side SignalR hub (dispatch, session, broadcast) | +| [SIGNALR_DATASOURCE.md](docs/SIGNALR_DATASOURCE.md) | Real-time DataSource with CRUD & change tracking | + ## Folder Structure | Folder | Purpose | diff --git a/docs/SIGNALR_DATASOURCE.md b/AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md similarity index 76% rename from docs/SIGNALR_DATASOURCE.md rename to AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md index 0f36b5d..560be6e 100644 --- a/docs/SIGNALR_DATASOURCE.md +++ b/AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md @@ -1,8 +1,9 @@ # SignalR DataSource -Change-tracked real-time collection built on top of the SignalR transport layer. Source: `AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs`. +Change-tracked real-time collection built on top of the SignalR transport layer. Source: `SignalRs/AcSignalRDataSource.cs` in this project. -> For the underlying transport (tag system, wire protocol, dispatch) see [`SIGNALR.md`](SIGNALR.md). +> For the underlying transport (tag system, wire protocol, client base) see [`AyCode.Services/docs/SIGNALR.md`](../../AyCode.Services/docs/SIGNALR.md). +> For server hub infrastructure see [`SIGNALR_SERVER.md`](SIGNALR_SERVER.md). ## Overview @@ -43,7 +44,6 @@ public sealed class SignalRCrudTags( **Usage (consuming project):** ```csharp -// Tags are defined as independent constants in the project's tag class public abstract class MyProjectTags : AcSignalRTags { public const int OrderGetAll = 300; @@ -51,10 +51,8 @@ public abstract class MyProjectTags : AcSignalRTags public const int OrderAdd = 302; public const int OrderUpdate = 303; public const int OrderRemove = 304; - // Tags don't have to be sequential — they just happen to be here } -// Construct with 5 explicit tags var crudTags = new SignalRCrudTags( MyProjectTags.OrderGetAll, MyProjectTags.OrderGetItem, @@ -69,21 +67,12 @@ var crudTags = new SignalRCrudTags( ## Data Loading ```csharp -// Awaits full response via sync-wait transport pattern (polls up to 60s) -await dataSource.LoadDataSource(); - -// Async callback — requests SignalResponseDataMessage directly to avoid double deserialization -await dataSource.LoadDataSourceAsync(); - -// Load from raw response bytes (public — usable when response is already available) +await dataSource.LoadDataSource(); // sync-wait transport (polls up to 60s) +await dataSource.LoadDataSourceAsync(); // async callback path await dataSource.LoadDataSourceFromResponseData(responseData, serializerType); - -// Single item by ID — updates or adds to InnerList -await dataSource.LoadItem(id); +await dataSource.LoadItem(id); // single item by ID ``` -All load methods are `async Task`. `LoadDataSource` uses the sync-wait transport pattern (`GetAllAsync` → polls for response), while `LoadDataSourceAsync` uses the async callback path (avoids deserializing `ResponseData` into a typed object — works directly with `byte[]`). - **Binary deserialization paths:** - `AcObservableCollection`: `BeginUpdate()` → `BinaryToMerge()` → `EndUpdate()` — single batched UI notification. - `List`: `BinaryTo(InnerList)` — direct populate. @@ -120,14 +109,12 @@ Each operation has an optional `autoSave` parameter — if true, immediately cal | Event | Signature | Fires when | |-------|-----------|------------| -| `OnDataSourceItemChanged` | `Func, Task>?` | After each item is saved or loaded (carries item + TrackingState) | +| `OnDataSourceItemChanged` | `Func, Task>?` | After each item is saved or loaded | | `OnDataSourceLoaded` | `Func?` | After `LoadDataSource` / `LoadDataSourceAsync` completes | | `OnSyncingStateChanged` | `Action?` | On 0→1 (true) and 1→0 (false) sync transitions | ## SaveChanges -Two variants with different transport patterns: - | Method | Returns | Transport pattern | Use case | |--------|---------|-------------------|----------| | `SaveChanges()` | `List` (remaining failures) | Sync-wait (`ContinueWith`) | When caller needs to know what failed | @@ -144,8 +131,6 @@ Both follow the same flow: EndSync() ``` -**SaveItem(item, trackingState):** saves a single item (same flow). **SaveItem(id):** looks up tracking item by ID, then saves. - **Rollback:** `TryRollbackItem(id)` restores `OriginalValue` to `InnerList`. For `TrackingState.Add`: removes item entirely. For `Remove`: re-adds `OriginalValue`. Manual `Rollback()` reverts all tracked changes at once. ## Sync State @@ -162,17 +147,14 @@ UI binds to `IsSyncing` to show loading indicators. The counter supports nested ## Working Reference List -Allows an external list to become the DataSource's inner storage: - ```csharp dataSource.SetWorkingReferenceList(externalList); // Now dataSource operates directly on externalList — same reference, no copy -// Read back the current inner list reference TIList innerList = dataSource.GetReferenceInnerList(); ``` -This is useful when the UI already has a bound collection and you want the DataSource to manage it in-place. `HasWorkingReferenceList` indicates whether an external list has been set. +Useful when the UI already has a bound collection and you want the DataSource to manage it in-place. ## Locking Strategy @@ -185,7 +167,7 @@ This is useful when the UI already has a bound collection and you want the DataS ## Relationship to Transport -The DataSource is a **consumer** of the SignalR transport, not part of it. The flow: +The DataSource is a **consumer** of the SignalR transport, not part of it: ``` DataSource.SaveChanges() @@ -195,14 +177,13 @@ DataSource.SaveChanges() → Server method with [SignalR(tag)] ← tag dispatch ``` -Projects can also call the transport directly without DataSource — see [`SIGNALR.md` § How Projects Use Tags](SIGNALR.md#how-projects-use-tags). +Projects can also call the transport directly without DataSource — see [`AyCode.Services/docs/SIGNALR.md`](../../AyCode.Services/docs/SIGNALR.md). ## Key Source Files | Component | Path | |-----------|------| -| DataSource | `AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs` | +| DataSource | `SignalRs/AcSignalRDataSource.cs` | +| Tracking helpers | `SignalRs/TrackingItemHelpers.cs` | | CRUD tags | `AyCode.Services/SignalRs/SignalRCrudTags.cs` | -| Tracking helpers | `AyCode.Services.Server/SignalRs/TrackingItemHelpers.cs` | | Transport client | `AyCode.Services/SignalRs/AcSignalRClientBase.cs` | -| Transport doc | [`docs/SIGNALR.md`](SIGNALR.md) | diff --git a/AyCode.Services.Server/docs/SIGNALR_SERVER.md b/AyCode.Services.Server/docs/SIGNALR_SERVER.md new file mode 100644 index 0000000..952c75a --- /dev/null +++ b/AyCode.Services.Server/docs/SIGNALR_SERVER.md @@ -0,0 +1,104 @@ +# SignalR Server + +Server-side SignalR hub infrastructure: method dispatch, session management, broadcast, and diagnostics. Source: `SignalRs/` in this project. + +> For client-side transport (tags, wire protocol, client base) see [`AyCode.Services/docs/SIGNALR.md`](../../AyCode.Services/docs/SIGNALR.md). +> For the DataSource collection see [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). + +## Server Processing + +``` +6. OnReceiveMessage(tag, bytes, requestId) +7. DynamicMethodRegistry.GetMethodByMessageTag(tag) ← ConcurrentDictionary lookup +8. DeserializeParameters(bytes): + ├─ DeserializeFromBinary() ← unwrap Binary envelope + ├─ IdMessage format? → parse each Ids[i] as JSON per parameter type + └─ Complex object? → json.JsonTo(paramType) ⚠️ tech debt: JSON parse +9. MethodInfo.InvokeMethod(instance, params) ← unwraps Task/ValueTask +10. CreateResponseMessage(tag, Success, result) ← pure Binary serialization +11. ResponseToCaller(tag, message, requestId) +12. If SendToOtherClientType != None: + └─ SendMessageToOthers(sendToOtherClientTag, result) ← uses sendToOtherClientTag, not messageTag +``` + +## Dynamic Method Dispatch + +See also: [`AyCode.Models.Server/DynamicMethods/README.md`](../../AyCode.Models.Server/DynamicMethods/README.md) + +### Server-Side Lookup + +``` +1. OnReceiveMessage(tag=100, bytes, requestId) + +2. DynamicMethodRegistry.GetMethodByMessageTag(100) + ├─ Check static ConcurrentDictionary cache + ├─ Hit? → find instance of cached Type from registered instances + ├─ Miss? → scan all registered instances' methods for [SignalR(100)] + │ cache the result (including negative = null) + └─ Return (instance, methodInfoModel) or null + +3. AcMethodInfoModel contains: + ├─ MethodInfo (the method to invoke) + ├─ SignalRAttribute (tag, sendToOtherClientTag, sendToOtherClientType) + └─ ParamInfos[] (ParameterInfo for deserialization) +``` + +The `DynamicMethodRegistry` uses a static `ConcurrentDictionary` for the global tag→method cache. + +### Registration + +The hub registers service instances during initialization: + +```csharp +DynamicMethodRegistry.Register(myService); // scans [SignalR(tag)] methods lazily +DynamicMethodRegistry.Register(anotherService); +``` + +Reflection runs lazily per tag on first request, then results are cached statically. + +## Session Management + +`AcSessionService` tracks connected clients: + +```csharp +ConcurrentDictionary Sessions +``` + +`IAcSessionItem` requires `SessionId` property. Used for targeting messages to specific users/connections. + +## Broadcast Service + +`AcSignalRSendToClientService` provides server-push methods: + +| Method | Target | +|--------|--------| +| `SendMessageToAllClients` | All connected | +| `SendMessageToConnection(connectionId)` | Single connection | +| `SendMessageToUser(userId)` | User (all connections) | +| `SendMessageToUsers(userIds)` | Multiple users | + +All messages wrapped in `SignalResponseDataMessage` → binary serialized → `OnReceiveMessage`. + +## Hub Events + +- `OnConnectedAsync()` — log connection +- `OnDisconnectedAsync(exception)` — log disconnection, cleanup session + +## Diagnostics + +Enable with `AcWebSignalRHubBase.EnableBinaryDiagnostics = true`. + +Logs: hex dump (500 byte sample), header parsing (version, marker), property count + names via VarUInt reading. + +`SignalResponseDataMessage.DiagnosticLogger` — per-response logging: target type info, property list, inheritance chain, hex dump. + +## Key Source Files + +| Component | Path | +|-----------|------| +| Hub base | `SignalRs/AcWebSignalRHubBase.cs` | +| Session service | `SignalRs/AcSessionService.cs` | +| Broadcast service | `SignalRs/AcSignalRSendToClientService.cs` | +| Logger hub | `SignalRs/AcLoggerSignalRHub.cs` | +| Tracking helpers | `SignalRs/TrackingItemHelpers.cs` | +| Dynamic dispatch | `AyCode.Models.Server/DynamicMethods/AcDynamicMethodRegistry.cs` | diff --git a/AyCode.Services/AyCode.Services.csproj b/AyCode.Services/AyCode.Services.csproj index 6105957..d49c7e4 100644 --- a/AyCode.Services/AyCode.Services.csproj +++ b/AyCode.Services/AyCode.Services.csproj @@ -18,4 +18,8 @@ + + + + diff --git a/AyCode.Services/Loggers/README.md b/AyCode.Services/Loggers/README.md index 22bb0f0..acbd623 100644 --- a/AyCode.Services/Loggers/README.md +++ b/AyCode.Services/Loggers/README.md @@ -1,11 +1,11 @@ # Loggers -Remote log writers for sending log items over HTTP, SignalR, or to the browser console. +Remote log writers for sending log items over HTTP, SignalR, or to the browser console. All three implement `IAcLogWriterClientBase`. + +> For full logging architecture see [`docs/LOGGING.md`](../../docs/LOGGING.md). For core logger and writer abstractions see [`AyCode.Core/Loggers/README.md`](../../AyCode.Core/Loggers/README.md). ## Key Files -- **`AcHttpClientLogItemWriter.cs`** — Abstract generic writer extending `AcLogItemWriterBase`. Sends log items via HTTP POST as JSON. Manages its own `HttpClient` lifecycle. -- **`AcBrowserConsoleLogWriter.cs`** — Browser console writer via JS interop (`IJSRuntime`). Maps `LogLevel` to console methods (log, warn, error). Used in Blazor apps. -- **`AcSignaRClientLogItemWriter.cs`** — Sends `AcLogItemClient` items to a SignalR hub. Manages `HubConnection` lifecycle with `StartConnection()`/`StopConnection()`. - -All three implement `IAcLogWriterClientBase`. +- **`AcHttpClientLogItemWriter.cs`** — Abstract structured writer extending `AcLogItemWriterBase`. Sends log items via `HttpClient.PostAsJsonAsync()` (fire-and-forget). Manages its own `HttpClient` + `HttpClientHandler`. HTTP/2 default. +- **`AcBrowserConsoleLogWriter.cs`** — Blazor browser console writer extending `AcTextLogWriterBase` (text branch). Uses `IJSRuntime.InvokeVoidAsync()` to call `console.info` / `console.warn` / `console.error` based on `LogLevel`. +- **`AcSignaRClientLogItemWriter.cs`** — SignalR log transport writer extending `AcLogItemWriterBase`. Sends structured log items to `AcLoggerSignalRHub` via `HubConnection.SendAsync("AddLogItem", logItem)`. Manages connection lifecycle (`StartConnection`/`StopConnection`). Converts `TimeStampUtc` to UTC before sending (SignalR doesn't transmit `DateTime.Kind`). diff --git a/AyCode.Services/README.md b/AyCode.Services/README.md index afd12d3..b6c2d88 100644 --- a/AyCode.Services/README.md +++ b/AyCode.Services/README.md @@ -2,6 +2,13 @@ Shared service implementations: SignalR communication (custom binary protocol), login services, and remote log writers. +## Documentation + +| Document | Topic | +|---|---| +| [SIGNALR.md](docs/SIGNALR.md) | Client-side SignalR transport (tags, wire protocol, req/resp flow) | +| [LOGGING_REMOTE.md](docs/LOGGING_REMOTE.md) | Remote log writers (HTTP, browser console, SignalR) | + ## Folder Structure | Folder | Purpose | diff --git a/AyCode.Services/docs/LOGGING_REMOTE.md b/AyCode.Services/docs/LOGGING_REMOTE.md new file mode 100644 index 0000000..c9a7087 --- /dev/null +++ b/AyCode.Services/docs/LOGGING_REMOTE.md @@ -0,0 +1,69 @@ +# Remote Log Writers + +Client-side log writers that send log data to remote endpoints. Source: `Loggers/` in this project. For core logging framework see [`AyCode.Core/docs/LOGGING.md`](../../AyCode.Core/docs/LOGGING.md). For server-side GlobalLogger see [`AyCode.Core.Server/docs/LOGGING_SERVER.md`](../../AyCode.Core.Server/docs/LOGGING_SERVER.md). + +## AcBrowserConsoleLogWriter + +Blazor browser console writer. Extends `AcTextLogWriterBase` (text branch, not structured). Uses `IJSRuntime` to invoke browser console methods: + +| LogLevel | JS method | +|----------|-----------| +| Detail – Suggest | `console.info` | +| Warning | `console.warn` | +| Error | `console.error` | + +Implements `IAcLogWriterClientBase` (client-side marker). + +## AcHttpClientLogItemWriter\ + +HTTP POST writer. Extends `AcLogItemWriterBase` (structured branch). Manages its own `HttpClient` + `HttpClientHandler`. Sends log items as JSON: + +```csharp +_httpClient.PostAsJsonAsync(_url, logItem).Forget(); +``` + +Note: `Forget()` — fire-and-forget, no await on the HTTP response. HTTP/2 by default. + +## AcSignaRClientLogItemWriter + +SignalR log transport writer. Extends `AcLogItemWriterBase`. Sends items to a dedicated logger hub. Manages its own `HubConnection`: + +```csharp +_hubConnection.SendAsync("AddLogItem", logItem).Forget(); +``` + +**UTC conversion note:** SignalR doesn't transmit `DateTime.Kind`. The writer calls `logItem.TimeStampUtc.ToUniversalTime()` before sending to ensure the server receives actual UTC. + +Connection lifecycle: `StartConnection()` waits up to 10s for `Connected` state. `StopConnection()` stops and disposes the hub. + +## Remote Logging Flow + +Client applications can send log items to a server via SignalR: + +``` +Client App Server +────────── ────── +AcSignaRClientLogItemWriter AcLoggerSignalRHub + │ │ + │ logItem.ToUniversalTime() │ + │ StartConnection() │ + ├──SendAsync("AddLogItem", item)──► │ + │ ├─ logItem.TimeStampUtc = UtcNow + │ ├─ _logger.Write(logItem) + │ │ ├─ Console writer + │ │ ├─ DB writer + │ │ └─ ... (all server writers) +``` + +Note: `AcLoggerSignalRHub` overrides the client's `TimeStampUtc` with `DateTime.UtcNow` on the server side for authoritative timestamps. The hub is a simple `Hub` (not `AcWebSignalRHubBase`) — it does NOT use tag-based dispatch. + +## Key Source Files + +| Component | Path | +|-----------|------| +| Browser writer | `Loggers/AcBrowserConsoleLogWriter.cs` | +| HTTP writer | `Loggers/AcHttpClientLogItemWriter.cs` | +| SignalR writer | `Loggers/AcSignaRClientLogItemWriter.cs` | +| Structured writer base | `AyCode.Entities/AcLogItemWriterBase.cs` | +| DB writer | `AyCode.Database/AcDbLogItemWriter.cs` | +| Logger hub (server) | `AyCode.Services.Server/SignalRs/AcLoggerSignalRHub.cs` | diff --git a/docs/SIGNALR.md b/AyCode.Services/docs/SIGNALR.md similarity index 56% rename from docs/SIGNALR.md rename to AyCode.Services/docs/SIGNALR.md index 72bdc4d..a334681 100644 --- a/docs/SIGNALR.md +++ b/AyCode.Services/docs/SIGNALR.md @@ -1,8 +1,9 @@ -# SignalR Communication +# SignalR Client -The transport layer for AyCode real-time communication. Source: `AyCode.Services/SignalRs/` (client), `AyCode.Services.Server/SignalRs/` (server). +Client-side SignalR transport with custom binary protocol and tag-based dispatch. Source: `SignalRs/` in this project. -> For the change-tracked collection built on top of this transport, see [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). +> For server-side hub, session, broadcast see [`AyCode.Services.Server/docs/SIGNALR_SERVER.md`](../../AyCode.Services.Server/docs/SIGNALR_SERVER.md). +> For the DataSource collection see [`AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`](../../AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md). ## Design Overview @@ -13,7 +14,7 @@ Client ──OnReceiveMessage(tag, bytes, requestId)──► Server Client ◄──OnReceiveMessage(tag, bytes, requestId)── Server ``` -`OnReceiveMessage` is the only transport channel — all calls go through it. The `tag` (int) determines **which server method to invoke**. The `DynamicMethodRegistry` maps the tag to a concrete `[SignalR(tag)]`-annotated method at runtime. Projects call any endpoint by specifying the tag; the server dispatches accordingly. +`OnReceiveMessage` is the only transport channel — all calls go through it. The `tag` (int) determines **which server method to invoke**. Projects call any endpoint by specifying the tag; the server dispatches accordingly. ``` Client side: Server side: @@ -48,7 +49,6 @@ public abstract class MyProjectTags : AcSignalRTags public const int GetOrders = 100; public const int GetOrderById = 101; public const int SaveOrder = 102; - // ... project-specific tags } ``` @@ -64,9 +64,6 @@ Three attribute levels: // Server method: tag + optional broadcast behavior [SignalR(messageTag: 100, sendToOtherClientTag: 100, sendToOtherClientType: SendToClientType.Others)] -// messageTag: incoming tag this method handles -// sendToOtherClientTag: tag used when broadcasting result to other clients (can differ from messageTag) -// sendToOtherClientType: who receives the broadcast // Client receive: marks method for server→client dispatch [SignalRSendToClient(100)] @@ -79,7 +76,6 @@ Three attribute levels: Projects call the SignalR transport directly — not only through DataSource: ```csharp -// Direct call: project code calls any endpoint by tag var orders = await signalRClient.PostAsync>(MyTags.GetOrders, companyId); var order = await signalRClient.GetByIdAsync(MyTags.GetOrderById, orderId); await signalRClient.PostDataAsync(MyTags.SaveOrder, order); @@ -91,42 +87,7 @@ await signalRClient.PostDataAsync(MyTags.SaveOrder, order, async response => { . await signalRClient.PostDataAsync(MyTags.SaveOrder, order, response => { ... }); ``` -The `AcSignalRClientBase` CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are all generic transport methods that work with any tag — they are not tied to DataSource. - -## Dynamic Method Dispatch - -See also: [`AyCode.Models.Server/DynamicMethods/README.md`](../AyCode.Models.Server/DynamicMethods/README.md) - -### Server-Side Lookup - -``` -1. OnReceiveMessage(tag=100, bytes, requestId) - -2. DynamicMethodRegistry.GetMethodByMessageTag(100) - ├─ Check static ConcurrentDictionary cache - ├─ Hit? → find instance of cached Type from registered instances - ├─ Miss? → scan all registered instances' methods for [SignalR(100)] - │ cache the result (including negative = null) - └─ Return (instance, methodInfoModel) or null - -3. AcMethodInfoModel contains: - ├─ MethodInfo (the method to invoke) - ├─ SignalRAttribute (tag, sendToOtherClientTag, sendToOtherClientType) - └─ ParamInfos[] (ParameterInfo for deserialization) -``` - -The `DynamicMethodRegistry` uses a static `ConcurrentDictionary` for the global tag→method cache. Note: `AcDynamicMethodCallModel` uses a `FrozenDictionary` per-type cache internally, but the registry's own lookup path does direct method scanning with `ConcurrentDictionary` caching. - -### Registration - -The hub registers service instances during initialization: - -```csharp -DynamicMethodRegistry.Register(myService); // scans [SignalR(tag)] methods lazily -DynamicMethodRegistry.Register(anotherService); -``` - -Reflection runs lazily per tag on first request, then results are cached statically. +The CRUD helpers (`PostAsync`, `GetByIdAsync`, `GetAllAsync`, `PostDataAsync`) are all generic transport methods that work with any tag — they are not tied to DataSource. ## Wire Protocol @@ -170,22 +131,6 @@ Arguments serialized individually with VarUInt length prefix. Direct write to `I 5. AcBinaryHubProtocol frames it on the wire ``` -### Server Processing - -``` -6. OnReceiveMessage(tag, bytes, requestId) -7. DynamicMethodRegistry.GetMethodByMessageTag(tag) ← ConcurrentDictionary lookup -8. DeserializeParameters(bytes): - ├─ DeserializeFromBinary() ← unwrap Binary envelope - ├─ IdMessage format? → parse each Ids[i] as JSON per parameter type - └─ Complex object? → json.JsonTo(paramType) ⚠️ tech debt: JSON parse -9. MethodInfo.InvokeMethod(instance, params) ← unwraps Task/ValueTask -10. CreateResponseMessage(tag, Success, result) ← pure Binary serialization -11. ResponseToCaller(tag, message, requestId) -12. If SendToOtherClientType != None: - └─ SendMessageToOthers(sendToOtherClientTag, result) ← uses sendToOtherClientTag, not messageTag -``` - ### Server → Client ``` @@ -206,11 +151,11 @@ Arguments serialized individually with VarUInt length prefix. Direct write to `I └─ Consuming project overrides to handle server push ``` -**Broadcast receive:** When the server broadcasts to `Others`/`All`, receiving clients have no matching `requestId`. The message falls through to `MessageReceived(int messageTag, byte[] messageBytes)` — an abstract method the consuming project overrides to handle unsolicited server pushes (e.g., refresh UI, update local state). +**Broadcast receive:** When the server broadcasts to `Others`/`All`, receiving clients have no matching `requestId`. The message falls through to `MessageReceived(int messageTag, byte[] messageBytes)` — an abstract method the consuming project overrides. **Request model pooling:** `SignalRRequestModel` instances are managed via `SignalRRequestModelPool` (ObjectPool + IResettable) to avoid allocations per request. -### Response Patterns +## Response Patterns | Pattern | Method | Blocking | |---------|--------|----------| @@ -218,29 +163,6 @@ Arguments serialized individually with VarUInt length prefix. Direct write to `I | Async callback | `PostDataAsync(tag, data, async msg => {...})` | No, `Func` | | Fire-and-forget | `PostDataAsync(tag, data, msg => {...})` | No, `Action` | -## Session Management - -`AcSessionService` tracks connected clients: - -```csharp -ConcurrentDictionary Sessions -``` - -`IAcSessionItem` requires `SessionId` property. Used for targeting messages to specific users/connections. - -## Broadcast Service - -`AcSignalRSendToClientService` provides server-push methods: - -| Method | Target | -|--------|--------| -| `SendMessageToAllClients` | All connected | -| `SendMessageToConnection(connectionId)` | Single connection | -| `SendMessageToUser(userId)` | User (all connections) | -| `SendMessageToUsers(userIds)` | Multiple users | - -All messages wrapped in `SignalResponseDataMessage` → binary serialized → `OnReceiveMessage`. - ## Connection Lifecycle **Client configuration:** @@ -249,18 +171,12 @@ All messages wrapped in `SignalResponseDataMessage` → binary serialized → `O - Reconnection: `WithAutomaticReconnect()` + `WithStatefulReconnect()` - Keepalive: 60s interval, 180s server timeout -**Hub events:** -- `OnConnectedAsync()` — log connection -- `OnDisconnectedAsync(exception)` — log disconnection, cleanup session - ## Diagnostics -Enable with `AcSignalRClientBase.EnableBinaryDiagnostics = true` or `AcWebSignalRHubBase.EnableBinaryDiagnostics = true`. +Enable with `AcSignalRClientBase.EnableBinaryDiagnostics = true`. Logs: hex dump (500 byte sample), header parsing (version, marker), property count + names via VarUInt reading. -`SignalResponseDataMessage.DiagnosticLogger` — per-response logging: target type info, property list, inheritance chain, hex dump. - ## Known Technical Debt **JSON-in-Binary request parameters:** Client→server requests currently wrap parameters in JSON inside the binary envelope (`SignalPostJsonDataMessage`). This adds an unnecessary serialization round-trip. Responses are already pure binary. Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects. @@ -269,13 +185,11 @@ Logs: hex dump (500 byte sample), header parsing (version, marker), property cou | Component | Path | |-----------|------| -| Hub base | `AyCode.Services.Server/SignalRs/AcWebSignalRHubBase.cs` | -| Client base | `AyCode.Services/SignalRs/AcSignalRClientBase.cs` | -| Binary protocol | `AyCode.Services/SignalRs/AcBinaryHubProtocol.cs` | -| Message types | `AyCode.Services/SignalRs/IAcSignalRHubClient.cs` | -| Serialization | `AyCode.Services/SignalRs/SignalRSerializationHelper.cs` | -| Tag attributes | `AyCode.Services/SignalRs/SignalMessageTagAttribute.cs` | -| Base tags | `AyCode.Services/SignalRs/AcSignalRTags.cs` | -| Session service | `AyCode.Services.Server/SignalRs/AcSessionService.cs` | -| Broadcast service | `AyCode.Services.Server/SignalRs/AcSignalRSendToClientService.cs` | -| Dynamic dispatch | `AyCode.Models.Server/DynamicMethods/AcDynamicMethodRegistry.cs` | +| Client base | `SignalRs/AcSignalRClientBase.cs` | +| Binary protocol | `SignalRs/AcBinaryHubProtocol.cs` | +| Tag attributes | `SignalRs/SignalMessageTagAttribute.cs` | +| Base tags | `SignalRs/AcSignalRTags.cs` | +| CRUD tags | `SignalRs/SignalRCrudTags.cs` | +| SendToClientType | `SignalRs/SendToClientType.cs` | +| Message types | `SignalRs/IAcSignalRHubClient.cs` | +| Serialization | `SignalRs/SignalRSerializationHelper.cs` | diff --git a/README.md b/README.md index 61c012a..fb5cbaa 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,22 @@ Domain rules and key pitfalls live in a single file: [`.github/copilot-instructi | Claude Code | ✅ `CLAUDE.md` → references above | None | | Cursor / Windsurf | ✅ `README.md` | Read `copilot-instructions.md` via @file | -Detailed docs: [`docs/`](docs/) — GLOSSARY.md, ARCHITECTURE.md, CONVENTIONS.md +Solution-level docs in [`docs/`](docs/): + +| Document | Topic | +|---|---| +| [GLOSSARY.md](docs/GLOSSARY.md) | Core terminology reference | +| [ARCHITECTURE.md](docs/ARCHITECTURE.md) | Solution layers, dependency rules | +| [CONVENTIONS.md](docs/CONVENTIONS.md) | Coding conventions | + +Project-level docs — each project's `docs/` folder documents the code it defines: + +| Project | Documents | +|---|---| +| [`AyCode.Core/docs/`](AyCode.Core/docs/) | [BINARY_FORMAT](AyCode.Core/docs/BINARY_FORMAT.md), [BINARY_FEATURES](AyCode.Core/docs/BINARY_FEATURES.md), [BINARY_OPTIONS](AyCode.Core/docs/BINARY_OPTIONS.md), [LOGGING](AyCode.Core/docs/LOGGING.md) | +| [`AyCode.Core.Server/docs/`](AyCode.Core.Server/docs/) | [LOGGING_SERVER](AyCode.Core.Server/docs/LOGGING_SERVER.md) | +| [`AyCode.Services/docs/`](AyCode.Services/docs/) | [SIGNALR](AyCode.Services/docs/SIGNALR.md), [LOGGING_REMOTE](AyCode.Services/docs/LOGGING_REMOTE.md) | +| [`AyCode.Services.Server/docs/`](AyCode.Services.Server/docs/) | [SIGNALR_SERVER](AyCode.Services.Server/docs/SIGNALR_SERVER.md), [SIGNALR_DATASOURCE](AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md) | ## Solution Structure diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 3895a29..c50506e 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -36,7 +36,7 @@ AyCode.Services ← AyCode.Services.Server - **AyCode.Services.Server** — Server-side: SignalR hub with custom binary protocol, email (SendGrid), JWT auth. - **AyCode.Models.Server/DynamicMethods** — Reflection-based tag→method dispatch used by the SignalR hub. -> **SignalR Dispatch:** Both directions use a single method `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)` with integer tag-based routing instead of standard Hub methods. See [`SIGNALR.md`](SIGNALR.md) for full details. +> **SignalR Dispatch:** Both directions use a single method `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)` with integer tag-based routing instead of standard Hub methods. See [`AyCode.Services/docs/SIGNALR.md`](../AyCode.Services/docs/SIGNALR.md) for full details. ### Server Extensions - **AyCode.Core.Server**, **AyCode.Interfaces.Server**, **AyCode.Entities.Server**, **AyCode.Models.Server** — Server-only additions that don't belong in shared code. diff --git a/docs/BINARY_FORMAT.md b/docs/BINARY_FORMAT.md deleted file mode 100644 index 8ad8438..0000000 --- a/docs/BINARY_FORMAT.md +++ /dev/null @@ -1,409 +0,0 @@ -# AcBinary Wire Format - -Complete wire format specification for the AcBinary serializer. Source of truth: [`AyCode.Core/Serializers/Binaries/BinaryTypeCode.cs`](../AyCode.Core/Serializers/Binaries/BinaryTypeCode.cs). - -## Stream Layout - -``` -[version : 1 byte] [flags : 1 byte] [cacheCount : VarUInt?] [payload...] -``` - -- **version** — `FormatVersion = 1` (current). -- **flags** — See [Header Flags](#header-flags). -- **cacheCount** — Present only when `HeaderFlag_HasCacheCount` is set. Number of type wrapper slots used by serializer. - -## Header Flags - -The flags byte uses `0x90` (144) as base with bit flags in the lower nibble: - -| Bit | Mask | Flag | Meaning | -|-----|------|------|---------| -| 0 | `0x01` | Metadata | Property hash metadata included (cross-type deserialization) | -| 1 | `0x02` | RefHandling_OnlyId | Reference tracking for `IId` objects only | -| 2 | `0x04` | RefHandling_All | Reference tracking for all objects (always combined with bit 1) | -| 3 | `0x08` | HasCacheCount | VarUInt cache count follows the flags byte | - -**Reference handling modes:** None = `0x00`, OnlyId = `0x02`, All = `0x06` (bits 1+2). - -## Variable-Length Encoding - -### VarUInt (unsigned) - -LEB128: 7 data bits per byte, MSB = continuation flag. - -``` -value < 128 → 1 byte [0xxxxxxx] -value < 16384 → 2 bytes [1xxxxxxx] [0xxxxxxx] -value < 2097152 → 3 bytes ... -(max 5 bytes for uint32) -``` - -### VarInt (signed) - -ZigZag encoding maps signed to unsigned, then LEB128: - -``` -encode: (value << 1) ^ (value >> 31) -decode: (raw >> 1) ^ -(raw & 1) -``` - -Maps: `0 → 0`, `-1 → 1`, `1 → 2`, `-2 → 3`, etc. - -### VarULong (unsigned 64-bit) - -Same LEB128 encoding, max 10 bytes for uint64. - -## Type Markers - -All markers defined in `BinaryTypeCode.cs`. `SlotCount = 64`. - -### FixObj (0–63) - -Single-byte object type. The marker byte **is** the type slot index — no additional type identifier needed. - -``` -[FixObj(N)] [properties...] -``` - -**Slot allocation:** Slots 0–63 are reserved for runtime polymorphic types, assigned dynamically on first encounter during serialization. Source-generated (SGen) types receive slots starting at 64+ via `AllocateWrapperSlot()` (sequential, `Interlocked.Increment`). SGen slots are compile-time stable; runtime slots depend on serialization order. - -### Complex Types (64–71) - -| Code | Name | Wire format | -|------|------|-------------| -| 64 | Object | `[64] [VarUInt typeIndex] [properties...]` | -| 65 | ObjectRef | `[65] [VarUInt refCacheIndex]` | -| 66 | Array | `[66] [VarUInt count] [elements...]` | -| 67 | Dictionary | `[67] [VarUInt count] [key, value pairs...]` | -| 68 | ByteArray | `[68] [VarUInt length] [raw bytes]` | -| 69 | ObjectWithMetadata | `[69] [VarUInt typeIndex] [VarUInt hashCount] [hashes...] [properties...]` | -| 70 | ObjectRefFirst | `[70] [VarUInt refCacheIndex] [object body...]` | -| 71 | ObjectWithMetadataRefFirst | `[71] [VarUInt refCacheIndex] [metadata + properties...]` | - -### Polymorphic Types (72–75) - -Used when runtime type differs from declared property type and `UseMetadata=false`. - -| Code | Name | Wire format | -|------|------|-------------| -| 72 | ObjectWithTypeName | `[72] [UTF8 typeName] [inner marker] [body...]` — prefix, inner Object/Array/Dict follows | -| 73 | ObjectWithTypeNameRefFirst | `[73] [UTF8 typeName] [VarUInt refCacheIndex] [properties...]` — combined, no inner marker | -| 74 | ObjectWithTypeIndex | `[74] [VarUInt typeIndex] [inner marker] [body...]` — prefix | -| 75 | ObjectWithTypeIndexRefFirst | `[75] [VarUInt typeIndex] [VarUInt refCacheIndex] [properties...]` — combined | - -Second occurrence of a referenced polymorphic object uses plain `ObjectRef(65)` — no polymorphic prefix needed. - -### Primitives (76–90) - -| Code | Name | Wire format | -|------|------|-------------| -| 76 | Null | `[76]` — no payload | -| 77 | True | `[77]` — no payload | -| 78 | False | `[78]` — no payload | -| 79 | Int8 | `[79] [1 byte]` | -| 80 | UInt8 | `[80] [1 byte]` | -| 81 | Int16 | `[81] [VarInt]` | -| 82 | UInt16 | `[82] [VarUInt]` | -| 83 | Int32 | `[83] [VarInt]` | -| 84 | UInt32 | `[84] [VarUInt]` | -| 85 | Int64 | `[85] [VarLong]` | -| 86 | UInt64 | `[86] [VarULong]` | -| 87 | Float32 | `[87] [4 bytes IEEE 754]` | -| 88 | Float64 | `[88] [8 bytes IEEE 754]` | -| 89 | Decimal | `[89] [16 bytes]` | -| 90 | Char | `[90] [VarUInt]` | - -### Strings (91–94) - -| Code | Name | Wire format | -|------|------|-------------| -| 91 | String | `[91] [VarUInt byteLength] [UTF-8 bytes]` | -| 92 | StringInterned | `[92] [VarUInt cacheIndex]` — 2nd+ occurrence | -| 93 | StringEmpty | `[93]` — no payload | -| 94 | StringInternFirst | `[94] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]` — 1st occurrence | - -### Date/Time (95–98) - -| Code | Name | Wire format | -|------|------|-------------| -| 95 | DateTime | `[95] [8 bytes ticks]` | -| 96 | DateTimeOffset | `[96] [8 bytes ticks] [VarInt offsetMinutes]` | -| 97 | TimeSpan | `[97] [VarLong ticks]` | -| 98 | Guid | `[98] [16 bytes]` | - -### Other Markers - -| Code | Name | Wire format | -|------|------|-------------| -| 99 | Enum | `[99] [VarInt underlyingValue]` | -| 100 | MetadataHeader | Legacy: implies `RefHandling=true` + metadata present | -| 101 | NoMetadataHeader | Legacy: implies `RefHandling=true`, no metadata | -| 102 | PropertySkip | `[102]` — marks skipped property (default/null value) | - -### FixStr (103–134) - -Short ASCII strings encoded in a single marker byte + raw bytes (no length prefix): - -``` -[FixStrBase + byteLength] [ASCII bytes] -``` - -- Length range: 0–31 bytes (`FixStrBase=103`, `FixStrMax=134`) -- Saves 1 byte vs `String` marker + VarUInt length -- Falls back to `String(91)` if content is non-ASCII - -### TinyInt (192–255) - -Single-byte integer encoding for small values: - -``` -value = marker - 192 - 16 (range: -16 to 47) -marker = value + 16 + 192 (64 values total) -``` - -Saves 2+ bytes vs `Int32(83)` + VarInt for frequently occurring small integers. - -## Compact Encoding Selection - -The serializer applies compact encodings automatically: - -| Data | Condition | Encoding | Savings | -|------|-----------|----------|---------| -| Integer | −16 ≤ v ≤ 47 | TinyInt (1 byte) | 2–5 bytes | -| String | ≤31 bytes, ASCII | FixStr (1+N bytes) | 1 byte (no length prefix) | -| Object | type index < 64 | FixObj (1 byte) | 1–5 bytes (no VarUInt index) | -| String | empty | StringEmpty (1 byte) | 1+ bytes | -| Bool | — | True/False (1 byte) | no payload | - -## String Interning Protocol - -Controls deduplication of repeated string values. - -**Modes** (`StringInterningMode`): -- `None` — all strings inline, no overhead -- `Attribute` — only `[AcStringIntern]` properties interned (default) -- `All` — all strings within length limits interned - -**Length limits:** `MinStringInternLength=4`, `MaxStringInternLength=64` (configurable). - -**Wire protocol:** -1. Serializer pre-scans all eligible strings to build a plan (which strings repeat) -2. First occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [VarUInt byteLength] [UTF-8 bytes]` -3. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]` -4. Single-occurrence strings: written as normal `String`/`FixStr` (no interning overhead) - -## Reference Tracking - -Prevents infinite loops and preserves object identity for repeated references. - -**Modes** (`ReferenceHandlingMode`): -- `None` — no tracking (fastest, use when graph is a tree) -- `OnlyId` — track only `IId` objects (matched by ID value) -- `All` — track all reference types (two-phase scan required) - -**Two-phase process:** -1. **Scan pass** (`ScanPass.cs`) — walks the object graph, detects multi-referenced objects and repeated strings. Builds a `WriteDuplicateEntry[]` array (the "write plan") containing `VisitIndex`, `CacheMapIndex`, `IsFirst`, and `Value` for each duplicate. -2. **Sort** — write plan entries are sorted by `VisitIndex` to match the write pass traversal order. -3. **Serialize pass** — consumes the sorted write plan via `TryConsumeWritePlanEntry()`. A cursor (`_nextWritePlanVisitIndex`) advances through the plan in O(1) — no dictionary lookups during serialization. - -**Wire protocol:** -- First occurrence: `[ObjectRefFirst(70)] [VarUInt refCacheIndex] [object body...]` -- Subsequent: `[ObjectRef(65)] [VarUInt refCacheIndex]` - -**Example — same object referenced twice:** - -``` -Input: { Users: [userA, userA] } (same instance) - -Scan pass → WritePlan: - [{VisitIndex:2, CacheMapIndex:0, IsFirst:true}, - {VisitIndex:3, CacheMapIndex:0, IsFirst:false}] - -Wire output (Compact mode, ReferenceHandling=All): - [version=1] [flags=0x96] [VarUInt cacheCount=1] ← header - [FixObj(0)] ← root object - [Array(66)] [VarUInt(2)] ← Users array, 2 elements - [ObjectRefFirst(70)] [VarUInt(0)] [props...] ← userA, 1st occurrence - [ObjectRef(65)] [VarUInt(0)] ← userA, 2nd (2 bytes only) -``` - -## Property Ordering - -Properties are serialized in a deterministic order defined by `TypeMetadataBase.GetUnfilteredProperties()`: - -1. Walk the inheritance chain from **derived → base** (`currentType.BaseType` loop) -2. At each level, collect declared public instance properties -3. Sort **alphabetically** (`StringComparer.Ordinal`) within each level -4. Result: **base properties first, then derived, alphabetical within each level** - -This order is stable across serializer/deserializer as long as the type hierarchy doesn't change. - -### Cross-Type Deserialization (UseMetadata) - -When `UseMetadata=true`, property name hashes (FNV-1a via `FnvHash.ComputeString`) are written per type, enabling schema evolution: - -- **Serializer** writes property hashes in the metadata section (`ObjectWithMetadata(69)`) -- **Deserializer** builds an index mapping array (`GetIndexMapping()`) that maps source property indices to destination indices by matching FNV-1a hashes -- This allows deserialization even when source and destination types have different property sets or ordering - -When `UseMetadata=false`, properties are matched by **positional index only** — source and destination must have identical property layouts. - -**Edge cases:** -- **Hash collision** (`CheckDuplicatePropName=true`, default): throws `InvalidOperationException`. When `false`: collision silently ignored — risk of data corruption. -- **Source has unknown property** (not in destination): silently skipped via `SkipValue()`, no error. -- **Destination has extra property** (not in source): left at default value (new instance) or unchanged (populate mode). - -## Configuration Options - -Options defined in `AcBinarySerializerOptions` (inherits `AcSerializerOptions`). Each option controls which code paths execute and how the wire format changes. - -### WireMode - -| Value | Integers | Strings | Output size | Speed | -|-------|----------|---------|-------------|-------| -| `Compact` (default) | VarInt/VarUInt (1–5 bytes) | UTF-8 with speculative ASCII fast path | Smaller | Slightly slower | -| `Fast` | Fixed-width raw bytes (4/8 bytes) | UTF-16 memcpy (`charCount * 2` bytes) | Larger | Fastest encode/decode | - -**Format difference for strings:** -- Compact: `[VarUInt byteLength] [UTF-8 bytes]` — speculative ASCII (1 pass if all ASCII, rewind+UTF-8 fallback otherwise) -- Fast: `[VarUInt charCount] [raw UTF-16 bytes]` — zero-encoding memcpy - -**Code branch:** `context.FastWire` flag set at `context.Reset()`. Checked in `WriteStringUtf8()` and integer write methods. FixStr optimization is skipped in Fast mode (UTF-8 specific). - -### ReferenceHandling - -| Value | Tracked objects | Scan pass | Header flags | Wire markers | -|-------|----------------|-----------|--------------|-------------| -| `None` | Nothing | Skipped | `0x00` | Standard object markers only | -| `OnlyId` | `IId` objects only (by ID value) | Partial | `0x02` | `ObjectRefFirst(70)` + `ObjectRef(65)` | -| `All` (default) | All reference types | Full graph walk | `0x06` | `ObjectRefFirst(70)` + `ObjectRef(65)` | - -**Format impact:** When enabled, multi-referenced objects are written once with `ObjectRefFirst(70) + VarUInt(refCacheIndex)` on first encounter, then replaced by `ObjectRef(65) + VarUInt(refCacheIndex)` on subsequent encounters. Header `HasCacheCount` flag is set and cache count written. - -**Interaction with `ThrowOnCircularReference` (default: `true`):** -- `true` + ref handling enabled: all objects tracked for cycle detection, throws `InvalidOperationException` on circular reference -- `false` + ref handling enabled: only IId types tracked for deduplication, non-IId circular refs silently truncated at `MaxDepth` - -### UseMetadata - -| Value | Wire markers | Property matching | Overhead | -|-------|-------------|-------------------|----------| -| `false` (default) | `FixObj`/`Object` | Positional index only — types must match | None | -| `true` | `ObjectWithMetadata(69)` / `ObjectWithMetadataRefFirst(71)` | FNV-1a property name hashes | 4 bytes per property per type | - -**Format impact:** When enabled, each type's first occurrence writes `[VarUInt hashCount] [FNV-1a hash × N]` before properties. Deserializer uses hashes to build source→destination index mapping, enabling cross-type deserialization (different property sets/ordering). - -**Code branch:** `context.UseMetadata` controls whether `ObjectWithMetadata(69)` or plain `Object(64)` markers are used. When `false`, `IsDirectObjectWrite=true` allows source-generated writers to bypass `WriteObject` entirely and inline property writes. - -**Related:** `CheckDuplicatePropName` (default: `true`) — throws if FNV-1a hash collision detected between property names of the same type. Disable in production for performance. - -### UseStringInterning - -| Value | Eligible strings | Scan overhead | Wire markers | -|-------|-----------------|---------------|-------------| -| `None` | Nothing | None | `String(91)` / `FixStr` only | -| `Attribute` (default) | Properties with `[AcStringIntern(true)]` | Scans marked properties | `StringInternFirst(94)` + `StringInterned(92)` | -| `All` | All strings within length limits | Scans all strings | `StringInternFirst(94)` + `StringInterned(92)` | - -**Length limits:** `MinStringInternLength` (default: 4) and `MaxStringInternLength` (default: 64, 0=unlimited). Strings outside this range are always written inline. - -**Format impact:** Interned strings on first occurrence: `[StringInternFirst(94)] [VarUInt cacheIndex] [string data]`. Subsequent: `[StringInterned(92)] [VarUInt cacheIndex]` (1–2 bytes vs full string). Single-occurrence strings are never interned — no overhead for unique strings. - -**Code branch:** `context.StringInternEligible` flag set per-property before `WriteString`. Scan pass builds a `WriteDuplicateEntry[]` plan; write pass consumes it via cursor. - -### MaxDepth - -| Value | Behavior | -|-------|----------| -| `255` (default) | Effectively unlimited nesting | -| `0` | Root level only — nested objects/collections written as `Null(76)` | -| `N` | Objects deeper than N levels written as `Null(76)` | - -**Format impact:** Depth-exceeded values appear as `Null(76)` in the stream — indistinguishable from actual null values. No special marker. - -**Code branch:** Checked at entry of every object/collection write: `if (depth > MaxDepth) { WriteByte(Null); return; }`. - -### UseCompression - -| Value | Method | Granularity | Memory | -|-------|--------|-------------|--------| -| `None` (default) | No compression | — | — | -| `Block` | LZ4 single block | Entire payload | Full buffer in memory | -| `BlockArray` | LZ4 chunked | 64KB chunks | Streaming-friendly, lower peak memory | - -**Format impact:** Compression is applied **post-serialization** as a transparent wrapper — the inner wire format is unchanged. Both modes are pure managed C# (WASM-compatible, no native dependencies). - -**Code branch:** Applied in `AcBinarySerializer.Serialize()` after the serialization context produces the raw buffer: `if (UseCompression != None) Lz4.Compress(buffer, mode)`. Decompression is automatic on deserialize. - -### PropertyFilter - -Optional delegate `BinaryPropertyFilter?` (default: `null`). When set, invoked for each property to decide inclusion. - -``` -delegate bool BinaryPropertyFilter(in BinaryPropertyFilterContext context); -``` - -**BinaryPropertyFilterContext fields:** `DeclaringType`, `PropertyName`, `PropertyType`, `Instance` (null during metadata phase), `IsMetadataPhase`, `GetValue()` (lazy). - -**Format impact:** Excluded properties are completely absent from the stream — no marker, no placeholder. The deserializer must use `UseMetadata=true` or identical filter to correctly match property indices. - -**Code branch:** `context.HasPropertyFilter` checked in `ShouldSerializeProperty()`. Called twice: once during metadata registration (`Instance=null`), once during write phase. - -### PropertyMapper - -Optional delegate `PropertyMapperDelegate?` (default: `null`) for cross-type deserialization property remapping. - -``` -delegate PropertyInfo? PropertyMapperDelegate(PropertyInfo sourceProperty, Type destinationType); -``` - -**Purpose:** Maps properties between different class hierarchies (renamed properties, external DTOs). Result is cached — zero overhead on same-type operations (`Deserialize`). - -### WASM Options - -| Option | Default | Purpose | -|--------|---------|---------| -| `IsWasm` | `OperatingSystem.IsBrowser()` | Auto-detect WASM environment | -| `UseStringCaching` | follows `IsWasm` | Cache short strings during deserialization to reduce GC pressure | -| `MaxCachedStringLength` | 64 | Max string length to cache | - -**Format impact:** None — these are deserialization-only optimizations. When `UseStringCaching=true`, the deserializer maintains an intern cache for strings ≤ `MaxCachedStringLength` chars. Disabled automatically when `StringInternFirst` marker is encountered (interning takes precedence). - -### Other Options - -| Option | Type | Default | Purpose | -|--------|------|---------|---------| -| `UseGeneratedCode` | bool | `true` | Use source-generated writers/readers when available | -| `InitialBufferCapacity` | int | 4096 | Starting buffer size (bytes) for serialization output | -| `RemoveOrphanedItems` | bool | `false` | During `PopulateMerge`: remove destination collection items with no matching source ID | -| `UseAsync` | bool | `false` | Async context pool return via ThreadPool. Auto-disabled in WASM and when `ReferenceHandling=None` | -| `MaxContextPoolSize` | int | 8 | Max serialization contexts kept in pool | - -## Presets - -| Preset | WireMode | Metadata | StringInterning | RefHandling | MaxDepth | Compression | Other | -|--------|----------|----------|-----------------|-------------|----------|-------------|-------| -| `Default` | Compact | false | Attribute | All | 255 | None | — | -| `FastMode` | Compact | false | None | None | 255 | None | No scan pass | -| `ShallowCopy` | Compact | false | None | None | **0** | None | Root level only | -| `WasmOptimized` | Compact | false | Attribute | All | 255 | None | +StringCaching | -| `WithoutReferenceHandling` | Compact | false | Attribute | **None** | 255 | None | No scan pass | -| `WithoutMetadata` | Compact | **false** | Attribute | All | 255 | None | — | - -**Performance implication of presets:** -- `Default` / `WasmOptimized` — two-phase (scan + serialize) due to `ReferenceHandling=All` -- `FastMode` / `ShallowCopy` — single-phase (no scan pass) since both interning and refs are disabled -- The scan pass adds ~20-30% overhead; disable it when the object graph is a simple tree - -## Option Interactions - -Key interdependencies that affect which code branches execute: - -| Combination | Effect | -|-------------|--------| -| `ReferenceHandling=None` + `UseStringInterning=None` | **No scan pass** — fastest path, single-phase serialization | -| `ReferenceHandling=All` + `UseMetadata=true` | Uses `ObjectWithMetadataRefFirst(71)` marker — combined ref + metadata | -| `UseMetadata=false` + `UseGeneratedCode=true` | `IsDirectObjectWrite=true` — generated code inlines property writes, bypasses `WriteObject` | -| `UseMetadata=true` + `PropertyFilter` set | Filter invoked twice (metadata phase + write phase); filter results must be stable | -| `WireMode=Fast` + `UseStringInterning!=None` | Interned strings still use the fast string path (UTF-16 for first occurrence, VarUInt index for subsequent) | -| `UseCompression!=None` + any other option | Compression is orthogonal — applied post-serialization, inner format unchanged | diff --git a/docs/CONVENTIONS.md b/docs/CONVENTIONS.md index 48b9472..3b0bb4c 100644 --- a/docs/CONVENTIONS.md +++ b/docs/CONVENTIONS.md @@ -24,11 +24,11 @@ ## SignalR Conventions -See [`SIGNALR.md`](SIGNALR.md) for full architecture documentation. +See [`AyCode.Services/docs/SIGNALR.md`](../AyCode.Services/docs/SIGNALR.md) for full architecture documentation. - **Single dispatch method** — all communication goes through `OnReceiveMessage(int messageTag, byte[] messageBytes, int? requestId)`. Do not add new hub methods. - **Tag-based routing** — associate methods with integer tags via `[SignalR(tag)]` (server) or `[SignalRSendToClient(tag)]` (client). Tags must be unique across the entire system. -- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). +- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See [`AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`](../AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md). - **Binary protocol** — `AcBinaryHubProtocol` is the transport protocol. Responses use pure Binary serialization. ### ⚠️ Temporary: JSON-in-Binary Request Parameters diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index b7f4241..c38a0ab 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -19,11 +19,11 @@ Core terminology for the AyCode framework. Read this before working on unfamilia | **Toon** | Token-Oriented Object Notation. LLM-optimized format with @meta (schema) and @data (values) sections. Designed for maximum LLM comprehension accuracy. | | **AcJson** | Newtonsoft.Json wrapper with $id/$ref reference handling, IId-based resolution, and chain deserialization API. | | **Chain API** | Fluent deserialization: `CreateDeserializeChain().ThenDeserialize()...Execute()`. Resolves cross-references across multiple types. | -| **String Interning** | Binary serializer deduplicates repeated strings. Controlled via `[AcStringIntern]` attribute or `StringInterningMode`. See [BINARY_FORMAT.md](BINARY_FORMAT.md#string-interning-protocol). | +| **String Interning** | Binary serializer deduplicates repeated strings. Controlled via `[AcStringIntern]` attribute or `StringInterningMode`. See [`AyCode.Core/docs/BINARY_FEATURES.md`](../AyCode.Core/docs/BINARY_FEATURES.md#string-interning-protocol). | ## Binary Wire Format -For full specification see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). +For full specification see [`AyCode.Core/docs/BINARY_FORMAT.md`](../AyCode.Core/docs/BINARY_FORMAT.md). | Term | Definition | |---|---| @@ -66,14 +66,14 @@ For full specification see [`BINARY_FORMAT.md`](BINARY_FORMAT.md). ## SignalR Infrastructure -For full architecture see [`SIGNALR.md`](SIGNALR.md). +For full architecture see [`AyCode.Services/docs/SIGNALR.md`](../AyCode.Services/docs/SIGNALR.md). | Term | Definition | |---|---| | **OnReceiveMessage** | The single SignalR method for all communication. Signature: `(int messageTag, byte[] messageBytes, int? requestId)`. | | **Message Tag** | Integer identifier mapping to a method via `[SignalR(tag)]` or `[SignalRSendToClient(tag)]` attributes. | | **DynamicMethodRegistry** | Resolves message tags to `MethodInfo` at runtime. Static `ConcurrentDictionary` cache with lazy scan on miss. | -| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See [`SIGNALR_DATASOURCE.md`](SIGNALR_DATASOURCE.md). | +| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See [`AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`](../AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md). | | **AcBinaryHubProtocol** | Custom `IHubProtocol` replacing SignalR's JSON+Base64 with `AcBinarySerializer`. Protocol name: `"acbinary"`. | | **SignalResponseDataMessage** | Response message supporting Binary or JSON+GZip. Responses use pure Binary (no JSON overhead). | | **SignalPostJsonDataMessage** | ⚠️ TECH DEBT — request params serialized to JSON inside Binary envelope. Planned for pure Binary replacement. | @@ -88,7 +88,20 @@ For full architecture see [`SIGNALR.md`](SIGNALR.md). ## Logging +For full architecture see [`AyCode.Core/docs/LOGGING.md`](../AyCode.Core/docs/LOGGING.md). + | Term | Definition | |---|---| -| **AcLoggerBase** | Custom logger with Detail(0)→Debug(1)→Info(2)→Warning(3)→Suggest(4)→Error(5)→Disabled(255). Values are DB-synced — do not renumber. | -| **GlobalLogger** | Server-side singleton (`AcGlobalLogger`) for cross-service logging. | +| **LogLevel** | Byte enum: Detail(0), Trace(5), Debug(10), Info(15), Suggest(17), Warning(20), Error(25), Disabled(255). Values are DB-synced — do NOT renumber. | +| **AcLoggerBase** | Abstract logger with multi-writer fan-out. Two-level filtering: logger-level + per-writer level. Implements both `IAcLogWriterBase` and MS `ILogger`. | +| **AcLogWriterBase** | Abstract writer base. Own `LogLevel` loaded from appsettings by `AssemblyQualifiedName` match. Two branches: text (`AcTextLogWriterBase`) and structured (`AcLogItemWriterBase`). | +| **AcTextLogWriterBase** | Text formatting base. Format: `[HH:mm:ss.fff] [AppType[0]] [Level] [Category->Method] [ThreadId] Text`. | +| **AcConsoleLogWriter** | Colored console writer. Gray=≤Trace, Cyan=Suggest, Yellow=Warning, Red=≥Error. Thread-safe via static lock. | +| **AcLogItemWriterBase\** | Abstract structured writer. ThreadPool dispatch + Mutex serialization. `Activator.CreateInstance()` factory for log items. | +| **GlobalLogger** | Server-side singleton wrapping `AcGlobalLoggerBase`. Static methods for all log levels. Default category: `"GLOBAL_LOGGER"`. | +| **AcLoggerProvider\** | `ILoggerProvider` implementation with `ConcurrentDictionary` per-category cache. Registered via `AddAcLogger()` or `UseOnlyAcLogger()`. | +| **IAcLogItemClient** | Structured log item DTO interface. Fields: TimeStampUtc, AppType, LogLevel, ThreadId, CategoryName, CallerName, Text, Exception, ErrorType. | +| **AcLogItemClient** | `[MessagePackObject]` implementation of `IAcLogItemClient`. Client-side transport entity. | +| **AcLogItem** | Server-side entity extending `AcLogItemClient`. `[Table("LogItem")]` with auto-generated `Id` and `LogHeaderId`. | +| **AcLoggerSignalRHub\** | Server hub receiving log items via `AddLogItem(AcLogItem)`. Simple `Hub` (not tag-based dispatch). | +| **IAcLogWriterClientBase** | Empty marker interface for client-side writers (`: IAcLogWriterBase`). |