diff --git a/.claude/settings.local.json b/.claude/settings.local.json index 86d0e8a..558045a 100644 --- a/.claude/settings.local.json +++ b/.claude/settings.local.json @@ -139,7 +139,15 @@ "Bash(sed -n '50108,58000p' jitasm_tier1.txt)", "Bash(awk '/^; Assembly listing for method AyCode.Core.Tests.TestModels.TestOrder_All_False_GeneratedWriter:WriteProperties/{found=1; next} found && /^; Assembly listing for method/{exit} found && /mov[[:space:]]+byte[[:space:]]+ptr/{print}' jitasm_tier1.txt)", "Bash(git -C \"H:/Applications/Aycode/Source/AyCode.Core\" log --oneline -15 -- \"AyCode.Core/Serializers/Toons/AcToonSerializer.ToonSerializationContext.cs\")", - "Bash(dotnet workload *)" + "Bash(dotnet workload *)", + "Bash(git -C \"H:/Applications/Aycode/Source/AyCode.Core\" mv \"AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs\" \"AyCode.Services/SignalRs/AcSignalRDataSource.cs\")", + "Bash(git -C \"H:/Applications/Aycode/Source/AyCode.Core\" mv \"AyCode.Services.Server/SignalRs/TrackingItemHelpers.cs\" \"AyCode.Services/SignalRs/TrackingItemHelpers.cs\")", + "Bash(echo \"=== EXIT $? ===\")", + "Bash(rm -rf \"H:/Applications/Mango/Source/FruitBankHybridApp/FruitBankHybrid.Web.Client/obj\" \"H:/Applications/Aycode/Source/AyCode.Blazor/AyCode.Blazor.Components/obj\" && dotnet publish \"H:/Applications/Mango/Source/FruitBankHybridApp/FruitBankHybrid.Web.Client/FruitBankHybrid.Web.Client.csproj\" -c Release 2>&1)", + "Bash(git -C \"H:/Applications/Aycode/Source/AyCode.Core\" mv \"AyCode.Services.Server/docs/SIGNALR_DATASOURCE\" \"AyCode.Services/docs/SIGNALR_DATASOURCE\")", + "Bash(awk 'NR==135' \"H:/Applications/Aycode/Source/AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md\")", + "Bash(git -C \"H:/Applications/Aycode/Source/AyCode.Core\" status --short)", + "Bash(git -C \"H:/Applications/Aycode/Source/AyCode.Core\" reset)" ] } } diff --git a/.github/LLM_PROTOCOL_DECISIONS.md b/.github/LLM_PROTOCOL_DECISIONS.md index 7585b03..4052bfa 100644 --- a/.github/LLM_PROTOCOL_DECISIONS.md +++ b/.github/LLM_PROTOCOL_DECISIONS.md @@ -133,6 +133,7 @@ The "primary" vs "inherit" distinction is defined in `protocol-audit/references/ | LLMP-DEC-65 | 2026-04-27 | Topic-code length cap raised 2-5 → 2-6 in `TOPIC_CODES.md` | Adding the new `SIGDS` topic (5 chars, fits the prior cap) prompted a forward-looking review of the topic-code length spec. The 2-5 cap originally derived from C# class-name visual-distinction concerns (≥3 to avoid `Ac*`/`Mg*` collision; ≤5 to keep IDs visually compact). User judgment: future composite topics (transport-prefixed DataSource variants, multi-domain layered topics) may benefit from one extra char of namespace headroom. Existing 9 ACCORE topics max at 4 chars (XCUT, AUTH, META, LLMP, TOON) — non-breaking change. The 2-6 range still stays well below the `@repo.prefix` 4-12 char range — visually distinct. **Two edits**: `TOPIC_CODES.md` Step 1 ("(2-5 uppercase chars)" → "(2-6 uppercase chars)") and Collision-avoidance section ("are 2-5 chars" → "are 2-6 chars"). Class-name collision rule preserved (no `Ac*`/`Mg*` start). | `docs-check/references/TOPIC_CODES.md` (2 sentence-level edits — Registry maintenance Step 1 + Collision avoidance section) | | LLMP-DEC-66 | 2026-04-27 | Created `SIGDS` topic (`SIGNALR_DATASOURCE`) — registered in `TOPIC_CODES.md`, scaffolded `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/{README, SIGNALR_DATASOURCE_ISSUES, SIGNALR_DATASOURCE_TODO}.md`, relocated DataSource doc from sibling `SIGNALR/` folder, updated 15 cross-refs across AyCode.Core (9) + AyCode.Blazor (6); README rewrite incorporates 7 doc-vs-code drift fixes + Sample-placeholder Framework-First cleanup; 3 new entries (B-T4K9 / I-W2H6 / T-N5R8) | `AcSignalRDataSource` is conceptually a **client-server DataSource** (change tracking, rollback, sync state, lifecycle events, `IList` wrapper) — the SignalR transport is one current implementation detail, not the topic's defining identity. Previous placement under the `SIG` topic conflated transport concerns with state-management concerns. **Topic chosen**: `SIGDS` (5 chars, within the 2-6 cap raised in LLMP-DEC-65) — `SIG` prefix preserves "SignalR-coupled DataSource" identity (distinguishes from any future `RESTDS`, `GRPCDS` etc.); `DS` suffix is the universal DataSource shorthand. **Topic name**: `SIGNALR_DATASOURCE` (matches existing `SIGNALR_BINARY_PROTOCOL` uppercase+underscore convention). **Folder migration**: `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md` → `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md` (folder + paired `SIGNALR_DATASOURCE_ISSUES.md` + `SIGNALR_DATASOURCE_TODO.md`). **README content rewrite** incorporates 7 doc-vs-code drift fixes identified in pre-migration code review: (a) abstract class declaration explicit + subclassing example; (b) `SerializerType` property + JSON/GZip path documented; (c) full `LoadDataSource*` overload set including in-memory `LoadDataSource(TIList fromSource, ...)`; (d) fallback re-serialize path in `LoadDataSourceFromResponseData`; (e) missing public methods enumerated — `AddRange`, `TryRemove`, `TryGetValue`, `TryGetIndex`, `AsReadOnly`, `BinarySearch`; (f) `TrackingState.Get` added to event/state values; (g) `TryRollbackItem(id, out originalValue)` signature shown. **Framework-First cleanup**: replaced consumer-leaking `Order*` placeholder (`OrderGetAll = 300`, etc.) with domain-agnostic `Sample*` (`SampleGetAll = 300`, etc.) — Layer 0 doc no longer references consumer domain types. **Cross-ref updates** (15 references): `AyCode.Core/.github/copilot-instructions.md` Layer #151; `AyCode.Core/README.md` doc-folder table cell; `docs/GLOSSARY.md`; `docs/CONVENTIONS.md`; `AyCode.Services/SignalRs/README.md`; `AyCode.Services.Server/SignalRs/README.md`; `AyCode.Services.Server/README.md`; `AyCode.Services.Server/docs/SIGNALR/README.md` (relative cross-ref `SIGNALR_DATASOURCE.md` → `../SIGNALR_DATASOURCE/README.md`); `AyCode.Services/docs/SIGNALR/README.md`; AyCode.Blazor: `docs/GLOSSARY.md` (×2 same path), `MGGRID/README.md` (×2 same path), `MGGRID/MGGRID_DATASOURCE.md`, `MGGRID/MGGRID_PARAMETERS.md`. **3 new entries**: `ACCORE-SIGDS-B-T4K9` (SaveChangesAsync inner Task drop), `ACCORE-SIGDS-I-W2H6` (BinarySearch NotImpl), `ACCORE-SIGDS-T-N5R8` (ChangeTracking Dictionary perf refactor — pre-existing `//TODO: Dictionary... - J.` comment). **Existing `ACCORE-SIG-I-T7S2`** (GetAll raw byte[] for populate/merge) stays in `SIGNALR_ISSUES.md` per append-only ID rule — its body has cross-cutting transport+DataSource concerns, naturally lives under `SIG`; the new `SIGNALR_DATASOURCE_ISSUES.md` opens with a cross-ref pointer to it. | `docs-check/references/TOPIC_CODES.md` (new `SIGDS` row inserted between `SBP` and `BIN`); 3 new files: `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md`, `SIGNALR_DATASOURCE_ISSUES.md`, `SIGNALR_DATASOURCE_TODO.md`; deleted: `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md`; modified: 15 cross-ref locations across `AyCode.Core` (9 files) + `AyCode.Blazor` (4 files, 6 occurrences) | | LLMP-DEC-67 | 2026-04-27 | Decision Log retention policy revised: **last-15-active rolling window + year-month bucket archives** (supersedes LLMP-DEC-25's "annual at ~50"); first archive operation executed (LLMP-DEC-1..52 → `LLM_PROTOCOL_DECISIONS_2026_04.md`); `docs-archive/SKILL.md` Rule A + Rule B refactor | **User-driven policy revision triggered by acute pain**: this `LLM_PROTOCOL_DECISIONS.md` file had grown past 25K tokens (multiple LLM sessions flagged the size as routine pre-load cost). The original LLMP-DEC-25 archival plan ("annual split at ~50 entries") was set when the protocol stack was nascent and entry velocity low. The April 2026 ID-format migration (LLMP-DEC-50..63 batch) generated 14 entries in 2 days; year-bucket archives would still grow into 100+-entry monsters during similar future high-velocity periods. **Two coordinated changes** based on user input *("inkább csak az utolsó 10 [→ 15] active entry-t tartsuk meg, többit archiválni; year+month bucket year helyett")*: **(A) Retention rule shift**: from "annual at ~50" to **last-N-active rolling window + year-month bucket archives**, where N = 15 (compromise between user's initial "10" suggestion and the migration-batch size of 14 entries — 15 keeps a complete recent initiative + buffer). **(B) Bucket granularity shift**: from `*_.md` to `*__.md`. Sparse files OK — only month-buckets containing archived entries get a file. Year-month scales for high-velocity periods (a heavy month produces a manageable file, a quiet month produces no file). **First archive operation**: LLMP-DEC-1..52 (2026-04-20 through 2026-04-26) moved to `LLM_PROTOCOL_DECISIONS_2026_04.md`. Active log now contains LLMP-DEC-53..67 (15 entries). **Second-pass archive note**: parallel sessions added LLMP-DEC-65 (topic-code length cap raise) and LLMP-DEC-66 (SIGDS topic creation) AFTER my drift-fix LLMP-DEC-64 but BEFORE this entry; the policy entry was originally planned as LLMP-DEC-65, became 67. The archive operation extended from 49 entries (LLMP-DEC-1..49) to 52 (LLMP-DEC-1..52) to maintain strict 15-cap at this entry's commit (without LLMP-DEC-50/51/52 archived too, active count would have been 18). **Cross-link integrity**: archive file is on-disk, on-demand readable; references from active entries to archived IDs (e.g., LLMP-DEC-58 references LLMP-DEC-43, LLMP-DEC-50) work — readers open the archive when following a link. IDs are append-only and stable across active ↔ archive boundaries. **Foundational decisions preserved**: the `## Current protocol state (quick reference)` summary at the top of the active file (introduced by LLMP-DEC-25, refreshed by LLMP-DEC-60) stays unchanged — survives all archiving. Foundational decisions (Rules #1-6, ID format spec, governance, etc.) reachable via the summary even when their dated entries are archived. **`docs-archive/SKILL.md` updates** (6 edits): frontmatter description rewritten; intro rewritten; `## Archive criterion (single rule)` restructured into `## Archive criteria (two rules)` with Rule A (Status-based for ISSUES/TODO) + Rule B (Last-15-active for DEC log) + "Why two rules" rationale; Step 4 updated to year-month bucket + sparse-files clause; pointer block example updated; Step 7 fully rewritten. **Governance section in summary** updated: LLMP-DEC-25-era "Archive annually at ~50" bullet replaced with "Last-15-active + year-month bucket" wording, cross-linking LLMP-DEC-67. **Active entry count = 15** — next archive triggers at LLMP-DEC-68 (LLMP-DEC-53 would rotate). | new file: `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS_2026_04.md` (52 entries); modified: `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md` (entries 1..52 removed via two sed passes; pointer block added + updated; governance bullet rewritten); `AyCode.Core/.github/skills/docs-archive/SKILL.md` (6 edits) | +| LLMP-DEC-68 | 2026-06-07 | SIGDS phase 2 (code side): relocated `AcSignalRDataSource` + `TrackingItemHelpers` (`AyCode.Services.Server` → `AyCode.Services`) + removed orphan `SignalR.Core 1.2.0` from `AyCode.Blazor.Components` → unblocked WASM AOT for Blazor WebAssembly consumers; closes `ACCORE-SIGDS-T-D9F2` | Completes the code side of LLMP-DEC-66 (which moved only the SIGDS docs). **Symptom**: `FruitBankHybrid.Web.Client` (InteractiveWebAssembly) could not AOT-publish — Mono AOT cross-compiler crashed (`exit -1073741819`) precompiling `Microsoft.AspNetCore.SignalR.Core.dll` (`HubCallerContext.get_Features` → missing `Http.Features 2.x`). **Two root sources**: (1) `AcSignalRDataSource` lived in `AyCode.Services.Server` — a server assembly whose graph pulls `SignalR.Core` + `AyCode.Database`/`KeyDerivation`/`SendGrid` (none WASM-compatible) — forcing every WASM consumer to reference it; (2) an **orphan `Microsoft.AspNetCore.SignalR.Core 1.2.0`** (netstandard2.0 server-hub) `PackageReference` in `AyCode.Blazor.Components.csproj`, unused by code. JIT-WASM tolerated both (lazy load); AOT compiles all assemblies eagerly → crash. **Fix**: (a) `git mv` both files to `AyCode.Services/SignalRs/`, namespace `…Server.SignalRs` → `AyCode.Services.SignalRs` (zero server-only deps; dropped stray `Castle.Core.Logging` using); (b) removed the `SignalR.Core 1.2.0` package; (c) cleaned `.Server` DLL refs from Web.Client/Shared/Components csproj; (d) updated consumer usings; (e) `RunAOTCompilation=true` on Web.Client. **Verified**: AOT-input 175→165 assemblies, `SignalR.Core` gone, publish green (`[164/164]` AOT + emcc link). **Server stays JIT** (IIS in-process compatible — server Native AOT was never the goal; only the browser-WASM client AOTs, the user-observed ~2-3× deserialize speedup). **TourIAm consumer intentionally NOT updated** (per user — its build breaks until separately fixed). | moved: `AyCode.Services/SignalRs/{AcSignalRDataSource,TrackingItemHelpers}.cs` + `AyCode.Services/docs/SIGNALR_DATASOURCE/` folder; modified: `AyCode.Blazor.Components.csproj` (SignalR.Core 1.2.0 + Services.Server ref removed), `MgGridBase.cs`/`MgGridSignalRDataSource.cs` usings; `FruitBankHybrid.Web.Client.csproj` (5 `.Server` refs removed, AOT on), `FruitBankHybrid.Shared.csproj` (Services.Server ref removed) + 3 Shared usings; `TOPIC_CODES.md` SIGDS path; `SIGNALR_DATASOURCE_TODO.md` (closed). **Doc-link cross-refs (~18) pending** — mechanical path rewrite, follow-up doc-sync. **Rolling-window**: per LLMP-DEC-67 this entry rotates LLMP-DEC-53 → archive on next `docs-archive` run. | ## Known follow-ups diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 032f158..fd6177d 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -148,7 +148,7 @@ Full doctrine: `../docs/ARCHITECTURE.md#framework-vs-consumer-boundary` ## SignalR 7. **Tag-based transport (no conventional hub methods)** — SignalR communication should generally use the generic methods provided by `AcWebSignalRHubBase` (server) and `AcSignalRClientBase` (client). Request types are conventionally identified by `int` tags. Try to avoid adding custom, business-specific, or conventional string-based Hub methods (e.g., `GetUsers()`). -8. **AcSignalRDataSource** — generic `IList` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md`. Transport docs: `AyCode.Services/docs/SIGNALR/README.md`. +8. **AcSignalRDataSource** — generic `IList` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `AyCode.Services/docs/SIGNALR_DATASOURCE/README.md`. Transport docs: `AyCode.Services/docs/SIGNALR/README.md`. 9. ~~**JSON-in-Binary tech debt**~~ — **REMOVED** 2026-04-26 (resolved: `AyCode.Core/docs/XCUT/XCUT_ISSUES.md#accore-xcut-i-x8q1`). Number kept reserved to prevent renumbering of subsequent items. ## Critical Warnings diff --git a/.github/skills/docs-check/references/TOPIC_CODES.md b/.github/skills/docs-check/references/TOPIC_CODES.md index 5a5d7d6..d4ddf18 100644 --- a/.github/skills/docs-check/references/TOPIC_CODES.md +++ b/.github/skills/docs-check/references/TOPIC_CODES.md @@ -16,7 +16,7 @@ To make IDs like `ACCORE-LOG-I-K7M2`, `ACCORE-SIG-B-3R9P`, `ACCORE-XCUT-I-A4B7` | `AUTH` | AUTH | User authentication: bearer tokens, JWT, login flow, hub authorization | `AyCode.Core/docs/AUTH/` | | `SIG` | SIGNALR | SignalR transport: tags, client base, dispatch, session | `AyCode.Core/AyCode.Services/docs/SIGNALR/` (+ variant in `AyCode.Services.Server`) | | `SBP` | SIGNALR_BINARY_PROTOCOL | Binary wire protocol over SignalR: framing, chunking, argument read | `AyCode.Core/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/` | -| `SIGDS` | SIGNALR_DATASOURCE | Client-server DataSource on SignalR transport: change tracking, rollback, sync state, load lifecycle, IList wrapper | `AyCode.Core/AyCode.Services.Server/docs/SIGNALR_DATASOURCE/` | +| `SIGDS` | SIGNALR_DATASOURCE | Client-server DataSource on SignalR transport: change tracking, rollback, sync state, load lifecycle, IList wrapper | `AyCode.Core/AyCode.Services/docs/SIGNALR_DATASOURCE/` | | `BIN` | BINARY | AcBinary serializer: features, format, writers, source generator | `AyCode.Core/AyCode.Core/docs/BINARY/` | | `TOON` | TOON | Toon serializer: LLM-optimized format with @meta/@types/@data sections | `AyCode.Core/AyCode.Core/docs/TOON/` | | `XCUT` | cross-cutting | Issues / TODOs spanning ≥2 ACCORE topics — one canonical home, referenced from each affected topic | `AyCode.Core/AyCode.Core/docs/XCUT/` | diff --git a/AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_ISSUES.md b/AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_ISSUES.md index bc7a9ad..cfdf15f 100644 --- a/AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_ISSUES.md +++ b/AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_ISSUES.md @@ -209,7 +209,7 @@ The framework explicitly does NOT provide type-dispatch infrastructure (no wire- This is a permanent architectural decision, not a "TODO not yet done." Do not file new TODOs proposing wire-format type-id, registry, or session/handshake infrastructure inside `AyCode.Core`. **Workarounds (canonical patterns the consumer implements):** -- **Tag-based dispatch above AcBinary** (recommended — matches CLAUDE.md Rule #7): prefix each message with an `int` (or enum) tag the consumer reads first to choose `Deserialize`. The consumer encapsulates the tag-read + type-dispatch in its own deserialization wrapper. Reference example: `AyCode.Services.Server/SignalRs/AcSignalRDataSource` uses `SignalRCrudTags`. +- **Tag-based dispatch above AcBinary** (recommended — matches CLAUDE.md Rule #7): prefix each message with an `int` (or enum) tag the consumer reads first to choose `Deserialize`. The consumer encapsulates the tag-read + type-dispatch in its own deserialization wrapper. Reference example: `AyCode.Services/SignalRs/AcSignalRDataSource` uses `SignalRCrudTags`. - **Polymorphic envelope type**: define a single `T = Envelope` containing a discriminator field + raw payload bytes; the consumer deserializes the envelope, switches on the discriminator, and re-deserializes the payload as the concrete type from the inner `byte[]`. Adds a small layer of indirection but works on top of fix-T `Deserialize`. - **One input per type-stream**: separate streams per message-class. Practical when the type-set is small and the transport can afford multiple connections. diff --git a/AyCode.Services.Server.Tests/SignalRs/SignalRDatasources/SignalRDataSourceTestBase.LoadItem.cs b/AyCode.Services.Server.Tests/SignalRs/SignalRDatasources/SignalRDataSourceTestBase.LoadItem.cs index 25e649e..92dcd29 100644 --- a/AyCode.Services.Server.Tests/SignalRs/SignalRDatasources/SignalRDataSourceTestBase.LoadItem.cs +++ b/AyCode.Services.Server.Tests/SignalRs/SignalRDatasources/SignalRDataSourceTestBase.LoadItem.cs @@ -1,6 +1,7 @@ using AyCode.Core.Enums; using AyCode.Core.Tests.TestModels; using AyCode.Services.Server.SignalRs; +using AyCode.Services.SignalRs; namespace AyCode.Services.Server.Tests.SignalRs.SignalRDatasources; diff --git a/AyCode.Services.Server/README.md b/AyCode.Services.Server/README.md index 49f64fc..403e34b 100644 --- a/AyCode.Services.Server/README.md +++ b/AyCode.Services.Server/README.md @@ -11,7 +11,6 @@ Server-side service implementations: JWT authentication, SendGrid email delivery | Document | Topic | |---|---| | `SIGNALR/README.md` | Server-side SignalR hub (dispatch, session, broadcast) | -| `SIGNALR_DATASOURCE/README.md` | Real-time DataSource with CRUD & change tracking | ## Folder Structure @@ -19,7 +18,7 @@ Server-side service implementations: JWT authentication, SendGrid email delivery |---|---| | [`Emails/`](Emails/README.md) | SendGrid email service (registration, password reset) | | [`Logins/`](Logins/README.md) | Server-side login with JWT token generation | -| [`SignalRs/`](SignalRs/README.md) | SignalR hub base, session service, data source with change tracking, client broadcast | +| [`SignalRs/`](SignalRs/README.md) | SignalR hub base, session service, client broadcast | ## Dependencies diff --git a/AyCode.Services.Server/SignalRs/README.md b/AyCode.Services.Server/SignalRs/README.md index 8a29b99..9fc2607 100644 --- a/AyCode.Services.Server/SignalRs/README.md +++ b/AyCode.Services.Server/SignalRs/README.md @@ -1,6 +1,6 @@ # SignalRs -Server-side SignalR hub infrastructure: hub base class, session management, data source with change tracking, and client broadcast service. +Server-side SignalR hub infrastructure: hub base class, session management, and client broadcast service. > **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR/README.md`. @@ -18,17 +18,9 @@ Server-side SignalR hub infrastructure: hub base class, session management, data ### Data Source -> **Full specification:** `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md` - -- **`AcSignalRDataSource.cs`** — Generic real-time collection (`AcSignalRDataSource`) implementing `IList` with full CRUD and change tracking. - - **Change tracking:** `TrackingItem` wraps each modified item with `TrackingState` + `OriginalValue` for rollback. `ChangeTracking` manages the tracking list. - - **Loading:** `LoadDataSource()` (sync), `LoadDataSourceAsync()` (async callback), `LoadItem(id)` (single). Binary path uses `BinaryToMerge()` for `AcObservableCollection` (batch UI update via `BeginUpdate/EndUpdate`). - - **Saving:** `SaveChanges()` iterates tracked items, posts each via CRUD tag, rollbacks on failure. `SaveItem()` for individual saves. - - **Sync state:** `IsSyncing` (Interlocked counter) + `OnSyncingStateChanged` event for UI loading indicators. - - **Locking:** `object _syncRoot` (sync ops) + `SemaphoreSlim _asyncLock` (async ops). `GetEnumerator()` returns safe copy. - - **Working reference list:** `SetWorkingReferenceList()` allows external list to become inner storage (zero-copy). - - **Context:** `ContextIds` (object[]) + `FilterText` (string) sent with GetAll requests for server-side filtering. +> **Relocated (LLMP-DEC-68):** `AcSignalRDataSource` — the client-side real-time collection — moved to `AyCode.Services/SignalRs/` and is **no longer part of this server-side assembly**. Full spec: `AyCode.Services/docs/SIGNALR_DATASOURCE/README.md`. ### Utilities - **`ExtensionMethods.cs`** — `InvokeMethod()` — invokes methods and unwraps `Task`/`Task`/`ValueTask` results. -- **`TrackingItemHelpers.cs`** — Deep clone helpers: `JsonClone()`, `ReflectionClone()`. + +> `TrackingItemHelpers.cs` (deep-clone helpers used by the data source) relocated to `AyCode.Services/SignalRs/` alongside `AcSignalRDataSource` — see LLMP-DEC-68. diff --git a/AyCode.Services.Server/docs/README.md b/AyCode.Services.Server/docs/README.md index ac93635..34615e7 100644 --- a/AyCode.Services.Server/docs/README.md +++ b/AyCode.Services.Server/docs/README.md @@ -5,7 +5,8 @@ Topic docs for the `AyCode.Services.Server` project (Layer 0, server-side servic ## Topics - [`SIGNALR/`](SIGNALR/README.md) — Server-side SignalR hub (dispatch, session, broadcast) -- [`SIGNALR_DATASOURCE/`](SIGNALR_DATASOURCE/README.md) — Client-server DataSource on SignalR transport (change tracking, rollback, sync state, IList wrapper) + +> **Note:** `SIGNALR_DATASOURCE/` (the `AcSignalRDataSource` client-side collection) relocated to `AyCode.Services/docs/SIGNALR_DATASOURCE/` — see `LLMP-DEC-68`. ## Navigation diff --git a/AyCode.Services.Server/docs/SIGNALR/README.md b/AyCode.Services.Server/docs/SIGNALR/README.md index 1f2c70b..b482ad0 100644 --- a/AyCode.Services.Server/docs/SIGNALR/README.md +++ b/AyCode.Services.Server/docs/SIGNALR/README.md @@ -3,7 +3,7 @@ Server-side SignalR hub infrastructure: method dispatch, session management, broadcast, and diagnostics. Source: `SignalRs/` in this project. > For client-side transport (tags, wire protocol, client base) see `AyCode.Services/docs/SIGNALR/README.md`. -> For the DataSource collection see `../SIGNALR_DATASOURCE/README.md`. +> For the DataSource collection see `AyCode.Services/docs/SIGNALR_DATASOURCE/README.md` (relocated to the client-side project). ## Server Processing diff --git a/AyCode.Services/SignalRs/AcBinaryDeserExecutor.cs b/AyCode.Services/SignalRs/AcBinaryDeserExecutor.cs new file mode 100644 index 0000000..b313058 --- /dev/null +++ b/AyCode.Services/SignalRs/AcBinaryDeserExecutor.cs @@ -0,0 +1,104 @@ +using System.Collections.Concurrent; + +namespace AyCode.Services.SignalRs; + +/// +/// Dedicated off-pool executor for the AsyncSegment streaming deserialization (see +/// SIGNALR_BINARY_PROTOCOL_ISSUES.md#accore-sbp-i-t3w9). +/// +/// The streaming deser blocks in AsyncPipeReaderInput.TryAdvanceSegment → +/// ManualResetEventSlim.Wait() between network-delivered chunks. Running that blocking +/// wait on a .NET thread-pool thread (plain Task.Run) starves the pool: the signal that +/// unblocks it (the producer Feed()+Set()) runs on the SignalR receive-continuation, +/// which ALSO needs a pool thread → circular dependency → deadlock under concurrent load + real +/// network latency. +/// +/// Design — never-queue, grow-on-demand, reuse-forever: every +/// immediately hands the work to an idle worker (reuse) or spawns a new dedicated background +/// thread (grow). It never queues. Consequences: +/// +/// The deser always starts immediately on its own off-pool thread → the [202] +/// GetResult() always waits on a running deser (no queue-coupling). +/// The blocking Wait() never consumes a .NET thread-pool thread → the producer's +/// receive-continuations always get pool threads → no circular dependency, no starvation. +/// Threads are reused (no per-message churn). Thread count is bounded in practice by peak +/// concurrent chunked receives — one in-flight deser per connection (the [202] +/// GetResult drains it before the next message on that connection). +/// +/// +/// This is TaskCreationOptions.LongRunning's unconditional safety (never-queue, +/// off-pool) plus thread reuse. Workers live forever — no idle-timeout, hence no idle-vs-assign +/// race (the simplest provably-correct variant). An idle-timeout reclaim can be added later if the +/// held-thread memory at peak concurrency becomes a concern; it must use a CAS idle→dead transition. +/// +/// Rejected: a bounded-queue executor reintroduces the GetResult-on-queued-deser +/// coupling and drops requests on overload. +/// +internal static class AcBinaryDeserExecutor +{ + private static readonly ConcurrentStack _idle = new(); + + // Monotonic id for distinct worker-thread names — eases dump / profiler / debug correlation. + // Incremented only on grow (new thread), never per message. + private static int _threadId; + + /// + /// Runs on a dedicated off-pool thread (reused idle worker, or a new + /// one) — never queues. Returns a task completing with the work's result. + /// + public static Task Run(Func work) + { + var tcs = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + + if (_idle.TryPop(out var worker)) + worker.Assign(work, tcs); // reuse an idle worker + else + new Worker().Start(work, tcs); // grow: spawn a new dedicated thread + + return tcs.Task; + } + + private sealed class Worker + { + // Single-permit park signal. Released exactly once per park (Assign), so the count + // never exceeds 1 → no SemaphoreFullException. Release/Wait provide the memory barrier + // that publishes the _work/_tcs writes to the worker thread. + private readonly SemaphoreSlim _signal = new(0, 1); + private Func? _work; + private TaskCompletionSource? _tcs; + + public void Start(Func work, TaskCompletionSource tcs) + { + _work = work; + _tcs = tcs; + new Thread(Loop) { IsBackground = true, Name = $"AcBinaryDeser-{Interlocked.Increment(ref _threadId)}" }.Start(); + } + + // Called by Run only when this worker is parked in _signal.Wait() (it was popped from + // _idle, which it entered only AFTER finishing the previous work). So _work/_tcs are + // written here strictly while the worker is idle — no race with the worker's active read. + public void Assign(Func work, TaskCompletionSource tcs) + { + _work = work; + _tcs = tcs; + _signal.Release(); + } + + private void Loop() + { + while (true) + { + var work = _work ?? throw new InvalidOperationException("AcBinaryDeserExecutor worker woke without work — invariant violated"); + var tcs = _tcs ?? throw new InvalidOperationException("AcBinaryDeserExecutor worker woke without tcs — invariant violated"); + _work = null; + _tcs = null; + + try { tcs.SetResult(work()); } + catch (Exception ex) { tcs.SetException(ex); } + + _idle.Push(this); // become reusable + _signal.Wait(); // park until the next Assign + } + } + } +} diff --git a/AyCode.Services/SignalRs/AcBinaryHubProtocol.cs b/AyCode.Services/SignalRs/AcBinaryHubProtocol.cs index 9eec93c..f50ed03 100644 --- a/AyCode.Services/SignalRs/AcBinaryHubProtocol.cs +++ b/AyCode.Services/SignalRs/AcBinaryHubProtocol.cs @@ -929,10 +929,20 @@ public class AcBinaryHubProtocol : IHubProtocol // Lazy start: begin background deserialization after first chunk is written. // The deser task reads via AsyncPipeReaderInputAdapter (struct over class) which // calls TryAdvanceSegment on the input — blocks on ManualResetEventSlim.Wait when - // out of data. Browser fallback: skip Task.Run — the MRES.Wait throws + // out of data. Browser fallback: skip the background task — the MRES.Wait throws // PlatformNotSupportedException on WASM. Instead, buffer all chunks and run the // deserializer synchronously on CHUNK_END, where state.Input.Complete() has // already been called → TryAdvanceSegment never enters the Wait path. + // + // CRITICAL — off-pool via AcBinaryDeserExecutor, NOT the .NET thread pool. The deser + // BLOCKS in ManualResetEventSlim.Wait() between network-delivered chunks. On the + // thread pool (plain Task.Run) this pins a pool thread for the message's whole + // network-transfer; the signal that unblocks it (the producer Feed()+Set()) runs on + // the SignalR receive-continuation, which ALSO needs a pool thread → circular + // dependency → pool starvation → "stuck loading → 60 s timeout" hang under concurrent + // load. AcBinaryDeserExecutor runs the blocking wait on a dedicated, reused, never- + // queued thread (see SIGNALR_BINARY_PROTOCOL_ISSUES.md#accore-sbp-i-t3w9) → pool stays + // free for producers, no coupling, streaming overlap preserved, no churn. if (state.DeserTask == null && !IsBrowser) { _logger?.LogDebug("TryParseChunkData starting background deserialization targetType={TargetType}", state.StreamedArgType.Name); @@ -941,7 +951,7 @@ public class AcBinaryHubProtocol : IHubProtocol var type = state.StreamedArgType; var opts = _options; - state.DeserTask = Task.Run(() => AcBinaryDeserializer.Deserialize(input2, type, opts)); + state.DeserTask = AcBinaryDeserExecutor.Run(() => AcBinaryDeserializer.Deserialize(input2, type, opts)); } input = input.Slice(totalNeeded); diff --git a/AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs b/AyCode.Services/SignalRs/AcSignalRDataSource.cs similarity index 99% rename from AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs rename to AyCode.Services/SignalRs/AcSignalRDataSource.cs index c7ffdc3..e497f0b 100644 --- a/AyCode.Services.Server/SignalRs/AcSignalRDataSource.cs +++ b/AyCode.Services/SignalRs/AcSignalRDataSource.cs @@ -7,13 +7,12 @@ using AyCode.Core.Loggers; using AyCode.Core.Serializers; using AyCode.Core.Serializers.Binaries; using AyCode.Services.SignalRs; -using Castle.Core.Logging; using System.Collections; using System.Collections.ObjectModel; using System.Diagnostics; using System.Diagnostics.CodeAnalysis; -namespace AyCode.Services.Server.SignalRs +namespace AyCode.Services.SignalRs { //public class TrackingItemGuid(TrackingState trackingState, TDataItem currentValue, TDataItem? originalValue = null) : TrackingItem(trackingState, currentValue, originalValue) // where TDataItem : class, IId { } diff --git a/AyCode.Services/SignalRs/README.md b/AyCode.Services/SignalRs/README.md index fbb61a7..2a36729 100644 --- a/AyCode.Services/SignalRs/README.md +++ b/AyCode.Services/SignalRs/README.md @@ -21,7 +21,7 @@ Custom binary SignalR protocol, client infrastructure, message tagging, and seri ### Message Tagging - **`SignalMessageTagAttribute.cs`** — Three attributes: `TagAttribute` (base, int messageTag), `SignalRAttribute` (server method routing + client notification), `SignalRSendToClientAttribute` (client-side receive). - **`AcSignalRTags.cs`** — Static constants: `None`, `PingTag`, `EchoTag`. -- **`SignalRCrudTags.cs`** — Sealed class bundling 5 independent CRUD tag integers. `GetMessageTagByTrackingState()` maps `TrackingState` -> tag. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md`. +- **`SignalRCrudTags.cs`** — Sealed class bundling 5 independent CRUD tag integers. `GetMessageTagByTrackingState()` maps `TrackingState` -> tag. See `AyCode.Services/docs/SIGNALR_DATASOURCE/README.md`. - **`SendToClientType.cs`** — Enum: None, Others, Caller, All. ### Serialization & Pooling diff --git a/AyCode.Services.Server/SignalRs/TrackingItemHelpers.cs b/AyCode.Services/SignalRs/TrackingItemHelpers.cs similarity index 96% rename from AyCode.Services.Server/SignalRs/TrackingItemHelpers.cs rename to AyCode.Services/SignalRs/TrackingItemHelpers.cs index 92da9c4..1e3af11 100644 --- a/AyCode.Services.Server/SignalRs/TrackingItemHelpers.cs +++ b/AyCode.Services/SignalRs/TrackingItemHelpers.cs @@ -1,7 +1,7 @@ using System.Reflection; using AyCode.Core.Extensions; -namespace AyCode.Services.Server.SignalRs; +namespace AyCode.Services.SignalRs; public static class TrackingItemHelpers { diff --git a/AyCode.Services/docs/README.md b/AyCode.Services/docs/README.md index d86786f..5a8fd8f 100644 --- a/AyCode.Services/docs/README.md +++ b/AyCode.Services/docs/README.md @@ -7,6 +7,7 @@ Topic docs for the `AyCode.Services` project (Layer 0, service abstractions). - [`LOGGING/`](LOGGING/README.md) — Remote logger (variant — sends log entries over the wire) - [`SIGNALR/`](SIGNALR/README.md) — SignalR transport (tag-based protocol, generic hub methods) - [`SIGNALR_BINARY_PROTOCOL/`](SIGNALR_BINARY_PROTOCOL/README.md) — Binary-over-SignalR wire format, chunked framing +- [`SIGNALR_DATASOURCE/`](SIGNALR_DATASOURCE/README.md) — Client-server DataSource on SignalR transport (change tracking, rollback, sync state, IList wrapper) - [`MVC/`](MVC/README.md) — ASP.NET Core MVC formatters for the AcBinary wire format (InputFormatter / OutputFormatter, `application/vnd.acbinary`) ## Navigation @@ -18,5 +19,4 @@ Per the folder-navigation rule, start here when browsing `docs/`. Each topic fol - **Base logger** (framework): `../../AyCode.Core/AyCode.Core/docs/LOGGING/README.md` - **Server-side logger** (variant): `../../AyCode.Core.Server/docs/LOGGING/README.md` - **Server-side SignalR**: `../../AyCode.Services.Server/docs/SIGNALR/README.md` -- **Server-side SignalR DataSource**: `../../AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md` - **Binary serializer** (used by SIGNALR_BINARY_PROTOCOL): `../../AyCode.Core/AyCode.Core/docs/BINARY/README.md` diff --git a/AyCode.Services/docs/SIGNALR/README.md b/AyCode.Services/docs/SIGNALR/README.md index 1217277..743857d 100644 --- a/AyCode.Services/docs/SIGNALR/README.md +++ b/AyCode.Services/docs/SIGNALR/README.md @@ -3,7 +3,7 @@ Client-side SignalR transport: custom binary protocol, tag-based dispatch. Source: `SignalRs/` > Server-side hub, session, broadcast: `AyCode.Services.Server/docs/SIGNALR/README.md` -> DataSource collection: `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md` +> DataSource collection: `../SIGNALR_DATASOURCE/README.md` ## Design diff --git a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md index 9e9097c..34593c6 100644 --- a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md +++ b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md @@ -190,9 +190,9 @@ Inner `AcBinarySerializerOptions` defaults relevant for SignalR: `UseGeneratedCo |-------|-----------|----------------------|-----------| | `Bytes` (default) | `ArrayBinaryOutput` → `byte[]` → write to pipe as raw blob | `ArrayBinaryInput` (single contiguous buffer via `MemoryMarshal.TryGetArray` zero-copy / pool-rent). | **Pro**: simplest, fastest per-call, WASM-safe on both sides. **Con**: no zero-copy write, no pipeline overlap. | | `Segment` | `BufferWriterBinaryOutput` → directly to `PipeWriter`, chunk-by-chunk, single `Flush` at end | Same as Bytes (unified `ArrayBinaryInput` receive path — `_protocolMode` affects send only). | **Pro**: zero-copy write, WASM-safe. **Con**: no pipeline overlap — receiver must wait for full payload before deser starts. | -| `AsyncSegment` | `AsyncPipeWriterOutput` → self-describing chunked framing `[201][UINT16 size][data]` per chunk + `[202]` end marker (or `[203]` CHUNK_ABORT on mid-stream serialize failure — fault isolation, see below), per-chunk `FlushAsync` with timeout-bounded sync-await | `AsyncPipeReaderInput` (growing contiguous byte[] with sliding-window cycle); background `Task.Run` deserializes while chunks arrive. WASM: synchronous deser on `CHUNK_END`. | **Pro**: zero-copy write + pipeline parallelism (ser / network / deser overlap); per-message fault isolation via CHUNK_ABORT. **Con**: send-side not WASM-compatible (see below); slow consumer propagates as server-thread blocking (bounded by `FlushTimeout`). Max chunk: 65535 bytes. | +| `AsyncSegment` | `AsyncPipeWriterOutput` → self-describing chunked framing `[201][UINT16 size][data]` per chunk + `[202]` end marker (or `[203]` CHUNK_ABORT on mid-stream serialize failure — fault isolation, see below), per-chunk `FlushAsync` with timeout-bounded sync-await | `AsyncPipeReaderInput` (growing contiguous byte[] with sliding-window cycle); the streaming deser runs on a dedicated off-pool thread via `AcBinaryDeserExecutor` while chunks arrive. WASM: synchronous deser on `CHUNK_END`. | **Pro**: zero-copy write + pipeline parallelism (ser / network / deser overlap); per-message fault isolation via CHUNK_ABORT. **Con**: send-side not WASM-compatible (see below); send-side per-chunk sync-flush blocks under client backpressure (`ACCORE-SBP-I-T3W9`, send-side fix TBD). Max chunk: 65535 bytes. | -In `AsyncSegment` mode, `WriteMessage` dispatches to `WriteMessageChunked` which sends: (1) CHUNK_START — standard SignalR framing `[INT32 len][200][original message with INT32 -1 for streamed arg]`, (2) N x CHUNK_DATA + final CHUNK_END — `[201][UINT16 size][data]` per chunk + `[202]` end marker, **all emitted by `AsyncPipeWriterOutput`** in framed mode (zero-copy via `PipeWriter.Advance` with 3-byte header reservation; protocol layer no longer writes its own `[202]` or extra `FlushAsync`). The receiver's `TryParseChunkData` accumulates into an `AsyncPipeReaderInput` (multiMessage:false — protocol parses `[201]/[202]` framing externally and feeds raw data via `Feed`); on non-WASM platforms a background `Task.Run` deserializes in parallel via `AsyncPipeReaderInputAdapter`, on WASM the deserializer runs synchronously on `CHUNK_END` over the already-buffered data. +In `AsyncSegment` mode, `WriteMessage` dispatches to `WriteMessageChunked` which sends: (1) CHUNK_START — standard SignalR framing `[INT32 len][200][original message with INT32 -1 for streamed arg]`, (2) N x CHUNK_DATA + final CHUNK_END — `[201][UINT16 size][data]` per chunk + `[202]` end marker, **all emitted by `AsyncPipeWriterOutput`** in framed mode (zero-copy via `PipeWriter.Advance` with 3-byte header reservation; protocol layer no longer writes its own `[202]` or extra `FlushAsync`). The receiver's `TryParseChunkData` accumulates into an `AsyncPipeReaderInput` (multiMessage:false — protocol parses `[201]/[202]` framing externally and feeds raw data via `Feed`); on non-WASM platforms the streaming deser runs in parallel on a dedicated off-pool thread via `AcBinaryDeserExecutor` (NOT the .NET thread pool — the deser blocks in `ManualResetEventSlim.Wait()` between chunks; running it on the pool causes starvation deadlock, see `ACCORE-SBP-I-T3W9`), on WASM the deserializer runs synchronously on `CHUNK_END` over the already-buffered data. **Fault isolation (CHUNK_ABORT, `[203]`):** if the streamed-arg serialize throws after CHUNK_START has been sent (e.g. a property-getter NRE on the data graph), `WriteMessageChunked`'s `catch` emits a single `[203]` byte on the host `PipeWriter` instead of letting the exception propagate. The receiver's `TryParseChunkData` recognises `[203]` as a first-class abort: tears down the chunk-state safely (mirrors the `[202]` Complete → DeserTask observe → Dispose ordering), then surfaces the failure to the awaiting caller as a synthesised `CompletionMessage.WithError(invocationId, ...)` — the caller's task faults with an actionable error instead of waiting for a transport-level timeout. Blast radius = one message; the SignalR transport connection stays alive and other in-flight invocations on the same WebSocket are unaffected. If the abort marker itself can't be emitted (transport-level fault), behaviour falls back to the baseline (exception propagates, SignalR aborts the connection). Any other unexpected byte in chunk mode (not `[201]/[202]/[203]`) is a protocol-invariant violation: `TryParseChunkData` tears down safely and throws `System.IO.InvalidDataException` rather than guessing. Edge case: a mid-header serialize failure (post-`[201][UINT16=N]` commit, pre-data) is currently caught by the protocol-violation path; the robust graceful-abort fix is tracked in [`../../../AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_TODO.md#accore-bin-t-z6n2`](../../../AyCode.Core/docs/BINARY/BINARY_ASYNCPIPE_TODO.md#accore-bin-t-z6n2). @@ -204,7 +204,7 @@ Send and receive paths handle WASM (`OperatingSystem.IsBrowser()`) asymmetricall - **Send path**: `AsyncSegment` is **not supported on WebAssembly**. `AcBinaryHubProtocolOptions.Validate()` throws `PlatformNotSupportedException` if `IsBrowser && ProtocolMode == AsyncSegment` (the `AsyncPipeWriterOutput.SyncAwaitFlush` sync-over-async pattern would block the single UI thread). WASM clients must use `Bytes` or `Segment`. - **Receive path**: works on WASM with **any** server-side mode (including `AsyncSegment` → chunked wire). `TryParseChunkData` detects the platform at runtime: - - **Non-browser**: first `CHUNK_DATA` spawns a background `Task.Run` over an `AsyncPipeReaderInput` (pipeline parallelism — serialize / network / deserialize overlap). `CHUNK_END` awaits the task's result. + - **Non-browser**: first `CHUNK_DATA` starts the streaming deser over an `AsyncPipeReaderInput` on a dedicated off-pool thread via `AcBinaryDeserExecutor` (pipeline parallelism — serialize / network / deserialize overlap; off-pool to avoid the starvation deadlock in `ACCORE-SBP-I-T3W9`). `CHUNK_END` awaits the task's result. - **Browser**: the background task is skipped. Chunks accumulate in `AsyncPipeReaderInput`; on `CHUNK_END` the buffer is `Complete()`d and the deserializer runs synchronously on the current thread via the streaming overload `AcBinaryDeserializer.Deserialize(AsyncPipeReaderInput, Type, opts)`. After `Complete()`, `TryAdvanceSegment` sees `_completed=true` and never calls `ManualResetEventSlim.Wait()` (which throws `PlatformNotSupportedException` on WASM). Consequence: mixed topology (desktop server `AsyncSegment` + WASM client `Bytes`) works without negotiation or protocol-name variation — client converts incoming chunked wire to its synchronous processing model. diff --git a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md index 865210b..c696f6e 100644 --- a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md +++ b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md @@ -3,6 +3,31 @@ For planned/actionable work see `SIGNALR_BINARY_PROTOCOL_TODO.md`. For higher-level SignalR abstractions see `../SIGNALR/SIGNALR_ISSUES.md`. +## ACCORE-SBP-I-T3W9: AsyncSegment blocking waits on thread-pool → starvation deadlock under concurrent load + +**Severity:** Critical · **Status:** Partially Fixed (deser-side fixed 2026-06-08; send-side TBD) · **Area:** `AcBinaryHubProtocol` AsyncSegment send + receive paths + +### Description + +In `AsyncSegment` mode both directions do **synchronous blocking waits on .NET thread-pool threads**, which starve the pool under concurrent load + real network latency (production-only; never reproduces locally where there is no network and one user). Symptom: clients stuck loading → ~60 s timeout. Hits **all** client types (incl. WASM, which has no `.Wait` of its own — they are victims of server-side starvation). + +- **Receive (deser):** `TryParseChunkData` runs the streaming deser via `Task.Run`, which blocks in `AsyncPipeReaderInput.TryAdvanceSegment` → `ManualResetEventSlim.Wait()` between network-delivered chunks. The unblocking `Feed()`+`Set()` runs on the SignalR receive-continuation — **also** a pool thread → circular dependency → pool starvation, broken only by the runtime's slow thread injection (~1-2/s). +- **Send (flush):** `WriteMessageChunked` → `SyncFlush(pipeWriter.FlushAsync())` (lines ~535, ~1135) blocks a dispatch (pool) thread **per chunk** under client backpressure (slow mobile clients) while streaming multi-MB responses. Forced by the synchronous `IHubProtocol.WriteMessage` API. **This is the dominant production trigger** (proven by the WASM-client hang + multi-MB server→client payloads). + +`Bytes` mode avoids both (no background deser, no per-chunk sync flush) — hence stable as the current hotfix. + +### Fix — deser (receive) side ✅ Fixed (2026-06-08) + +Resolved — `TryParseChunkData` runs the blocking streaming deser off the .NET thread pool via `AcBinaryDeserExecutor` (`AyCode.Services/SignalRs/AcBinaryDeserExecutor.cs`). See that file's doc-comment for the design. + +### Fix — send (flush) side + +**TBD.** The per-chunk `SyncFlush` blocking is partly inherent to streaming-with-bounded-memory under the synchronous `WriteMessage` API. To be designed; entry updated when settled. + +### Related TODO + +(To be opened when the deser-side fix lands and the send-side direction is committed.) + ## ACCORE-SBP-I-F6T2: AsyncSegment send-path unsupported on WebAssembly **Severity:** Major (on WASM) · **Status:** Open · **Area:** `AsyncPipeWriterOutput` / WASM runtime diff --git a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_WASMTHREADING.md b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_WASMTHREADING.md index 2cdd553..297e40e 100644 --- a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_WASMTHREADING.md +++ b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_WASMTHREADING.md @@ -20,16 +20,19 @@ flag (line ~88) that gates the background deserialize task at line ~936: ```csharp if (state.DeserTask == null && !IsBrowser) { - state.DeserTask = Task.Run(() => AcBinaryDeserializer.Deserialize(input2, type, opts)); + state.DeserTask = AcBinaryDeserExecutor.Run(() => AcBinaryDeserializer.Deserialize(input2, type, opts)); } ``` -**Non-browser** (server / Windows-app): the deser-task runs on a real thread. +**Non-browser** (server / Windows-app): the deser-task runs on a dedicated +off-pool thread via `AcBinaryDeserExecutor` (NOT `Task.Run` / the .NET thread +pool — the blocking `ManualResetEventSlim.Wait()` between chunks would starve +the pool, see `SIGNALR_BINARY_PROTOCOL_ISSUES.md#accore-sbp-i-t3w9`). `AsyncPipeReaderInput.TryAdvanceSegment(...)` blocks on `ManualResetEventSlim.Wait()` when out of bytes — true pipeline parallelism between the producer (WebSocket-receive) and consumer (deserializer). -**Browser (WASM single-thread)**: `Task.Run` is skipped. Chunks accumulate in +**Browser (WASM single-thread)**: the background deser is skipped. Chunks accumulate in `AsyncPipeReaderInput` from the WebSocket-receive callback. On `CHUNK_END`, `Input.Complete()` is called → `_completed = true`. The deserializer then runs **synchronously on the current thread** over the fully-buffered payload. diff --git a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md b/AyCode.Services/docs/SIGNALR_DATASOURCE/README.md similarity index 98% rename from AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md rename to AyCode.Services/docs/SIGNALR_DATASOURCE/README.md index 2dcce86..d5a9760 100644 --- a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md +++ b/AyCode.Services/docs/SIGNALR_DATASOURCE/README.md @@ -1,9 +1,9 @@ # SignalR DataSource -Change-tracked real-time collection on top of the SignalR transport layer. Source: `SignalRs/AcSignalRDataSource.cs` in `AyCode.Services.Server`. +Change-tracked real-time collection on top of the SignalR transport layer. Source: `SignalRs/AcSignalRDataSource.cs` in `AyCode.Services`. > For the underlying transport (tag system, wire protocol, client base) see `AyCode.Services/docs/SIGNALR/README.md`. -> For server hub infrastructure see `../SIGNALR/README.md`. +> For server hub infrastructure see `AyCode.Services.Server/docs/SIGNALR/README.md`. ## Overview @@ -253,7 +253,7 @@ Projects can also call the transport directly without DataSource — see `AyCode | Component | Path | |---|---| -| DataSource (abstract base) | `SignalRs/AcSignalRDataSource.cs` (in `AyCode.Services.Server`) | -| Tracking helpers (`JsonClone` / `ReflectionClone`) | `SignalRs/TrackingItemHelpers.cs` (in `AyCode.Services.Server`) | +| DataSource (abstract base) | `SignalRs/AcSignalRDataSource.cs` (in `AyCode.Services`) | +| Tracking helpers (`JsonClone` / `ReflectionClone`) | `SignalRs/TrackingItemHelpers.cs` (in `AyCode.Services`) | | CRUD tags | `SignalRs/SignalRCrudTags.cs` (in `AyCode.Services`) | | Transport client | `SignalRs/AcSignalRClientBase.cs` (in `AyCode.Services`) | diff --git a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_ISSUES.md b/AyCode.Services/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_ISSUES.md similarity index 100% rename from AyCode.Services.Server/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_ISSUES.md rename to AyCode.Services/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_ISSUES.md diff --git a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_TODO.md b/AyCode.Services/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_TODO.md similarity index 53% rename from AyCode.Services.Server/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_TODO.md rename to AyCode.Services/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_TODO.md index f94f497..cee50dd 100644 --- a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_TODO.md +++ b/AyCode.Services/docs/SIGNALR_DATASOURCE/SIGNALR_DATASOURCE_TODO.md @@ -8,7 +8,9 @@ Forward-looking work specific to `AcSignalRDataSource`. Transport-side TODOs liv --- ## ACCORE-SIGDS-T-D9F2: Relocate AcSignalRDataSource + TrackingItemHelpers to AyCode.Services -**Priority:** P2 · **Type:** Architectural cleanup · **Status:** Open +**Priority:** P2 · **Type:** Architectural cleanup · **Status:** Closed (2026-06-07) + +> **Note:** The code relocation is complete and verified (WASM AOT publish succeeds). Doc-link cross-reference cleanup across the workspace tracked in the Resolution below. Both files currently live in `AyCode.Services.Server/SignalRs/`, but **neither has any Server-only dependency**: @@ -21,21 +23,29 @@ Both files currently live in `AyCode.Services.Server/SignalRs/`, but **neither h **Practical impact:** Blazor.WebAssembly cannot reference `AyCode.Services.Server` (its dependency graph pulls in `AyCode.Database`, `KeyDerivation`, `SendGrid` — none WASM-compatible). WASM consumers wanting `AcSignalRDataSource` are currently blocked. ### Acceptance -- [ ] `AcSignalRDataSource.cs` moved: `AyCode.Services.Server/SignalRs/` → `AyCode.Services/SignalRs/`; namespace `AyCode.Services.Server.SignalRs` → `AyCode.Services.SignalRs`. -- [ ] `TrackingItemHelpers.cs` moved + namespace updated. -- [ ] Consumer `using AyCode.Services.Server.SignalRs;` lines pulling these types updated to `using AyCode.Services.SignalRs;`. -- [ ] Doc folder relocated: `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/` → `AyCode.Services/docs/SIGNALR_DATASOURCE/` (3 files: README + ISSUES + TODO). -- [ ] `TOPIC_CODES.md` SIGDS row "Docs location" updated (`AyCode.Services.Server/...` → `AyCode.Services/...`). -- [ ] Doc-index READMEs updated: remove SIGDS row from `Services.Server/docs/README.md`; add to `Services/docs/README.md`. -- [ ] All cross-refs in workspace updated (~15 paths across AyCode.Core + AyCode.Blazor) per the path-rewrite pattern used in LLMP-DEC-66. -- [ ] `AyCode.Services.Server/README.md` + `AyCode.Services.Server/SignalRs/README.md` doc tables: SIGDS rows removed. -- [ ] `AyCode.Services/SignalRs/README.md` doc table: DataSource entry added (currently no Data Source section there). -- [ ] Tests in `AyCode.Services.Server.Tests/SignalRs/SignalRDatasources/` re-evaluated — likely keep in Server.Tests (full hub environment) but update `using` statements; or split into `AyCode.Services.Tests/SignalRs/SignalRDatasources/` if isolated. -- [ ] New `LLMP-DEC` entry filed (SIGDS phase 2 relocation, cross-references `LLMP-DEC-66`). -- [ ] Build green across all consumers (AyCode.Blazor.Components, FruitBankHybridApp, etc.); WASM build verified if applicable. +- [x] `AcSignalRDataSource.cs` moved: `AyCode.Services.Server/SignalRs/` → `AyCode.Services/SignalRs/`; namespace `AyCode.Services.Server.SignalRs` → `AyCode.Services.SignalRs`. +- [x] `TrackingItemHelpers.cs` moved + namespace updated. +- [x] Consumer `using AyCode.Services.Server.SignalRs;` lines pulling these types updated to `using AyCode.Services.SignalRs;`. +- [x] Doc folder relocated: `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/` → `AyCode.Services/docs/SIGNALR_DATASOURCE/` (3 files: README + ISSUES + TODO). +- [x] `TOPIC_CODES.md` SIGDS row "Docs location" updated (`AyCode.Services.Server/...` → `AyCode.Services/...`). +- [x] Doc-index READMEs updated: removed SIGDS row from `Services.Server/docs/README.md`; added to `Services/docs/README.md`. +- [x] All cross-refs updated: AyCode.Core 9 (registries + index READMEs + moved-README internals) + AyCode.Blazor 5 (GLOSSARY + MGGRID docs); FruitBankHybridApp had none (type mentions only, no path refs). +- [x] `AyCode.Services.Server/README.md` + `AyCode.Services.Server/SignalRs/README.md` doc tables: SIGDS rows removed. +- [ ] `AyCode.Services/SignalRs/README.md` doc table: DataSource entry — **deferred** (the `SignalRCrudTags` line cross-refs the spec; no standalone Data Source section added). +- [x] Tests in `AyCode.Services.Server.Tests/SignalRs/SignalRDatasources/` — `SignalRDataSourceTestBase.LoadItem.cs` was missing the `AyCode.Services.SignalRs` using (partial-file using list); **added**. Debug sln build green. (The Release sln config skips MSTest projects — that masked it during Release-only verification.) +- [x] New `LLMP-DEC` entry filed (`LLMP-DEC-68`). +- [x] Build green across consumers (AyCode.Core, AyCode.Blazor.Components, FruitBankHybrid.Web.Client WASM AOT publish verified). + +### Resolution (2026-06-07) +**What:** Moved `AcSignalRDataSource.cs` + `TrackingItemHelpers.cs` → `AyCode.Services/SignalRs/` (namespace `AyCode.Services.SignalRs`); removed orphan `Microsoft.AspNetCore.SignalR.Core 1.2.0` PackageReference from `AyCode.Blazor.Components.csproj`; cleaned `.Server` DLL refs from Web.Client / Shared / Components; consumer usings updated; this doc folder moved here. +**Why:** Unblocks browser-WASM AOT — `FruitBankHybrid.Web.Client` AOT-publish now succeeds (AOT-input 175→165 assemblies, `SignalR.Core` gone, `[164/164]` + emcc link OK). Server stays JIT (IIS in-process compatible). +**Where:** Full detail in `LLMP-DEC-68`. +**Verified:** `dotnet publish FruitBankHybrid.Web.Client -c Release` (RunAOTCompilation=true) — green; **AyCode.Core.sln green in both Debug and Release** (Debug builds the MSTest projects the Release config skips — this caught a missing using in `SignalRDataSourceTestBase.LoadItem.cs`, since fixed). +**Doc-link sync:** ✅ completed this session — all path cross-refs rewritten to `AyCode.Services/docs/SIGNALR_DATASOURCE` (AyCode.Core 9; AyCode.Blazor 5; FruitBankHybridApp 0). **Optional residue (non-blocking):** no standalone DataSource section in `AyCode.Services/SignalRs/README.md`; Server.Tests `SignalRDatasources` usings keep a now-redundant `Server.SignalRs` import (build green). **Decision Log rolling-window:** LLMP-DEC-68 should rotate LLMP-DEC-53 on next `docs-archive` run (per LLMP-DEC-67). ### Related - `LLMP-DEC-66` (SIGDS topic introduction — this TODO completes the architectural side that the docs-side migration left open) +- `LLMP-DEC-68` (this relocation — code side) - Parallels `LLMP-DEC-58` Framework-First retightening principle (consumer-side code shouldn't live in Server-flavored projects) --- diff --git a/docs/CONVENTIONS.md b/docs/CONVENTIONS.md index 9ce1760..a1b75f4 100644 --- a/docs/CONVENTIONS.md +++ b/docs/CONVENTIONS.md @@ -36,7 +36,7 @@ See `AyCode.Services/docs/SIGNALR/README.md` for full architecture documentation - **Single dispatch method** — all communication goes through `OnReceiveMessage(int messageTag, int? requestId, SignalParams signalParams, object data)`. Do not add new hub methods. - **Tag-based routing** — associate methods with integer tags via `[SignalR(tag)]` (server) or `[SignalRSendToClient(tag)]` (client). Tags must be unique across the entire system. -- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md`. +- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See `AyCode.Services/docs/SIGNALR_DATASOURCE/README.md`. - **Binary protocol** — `AyCodeBinaryHubProtocol` (derived from `AcBinaryHubProtocol`) is the transport protocol. Zero-copy write: `AcBinarySerializer.Serialize(value, output)` directly to pipe. Zero-copy read: `SequenceReader` + type-aware deserialization via `SignalParams.SignalDataType`. Three read paths: byte[] fast-path (0x44 tag), IsRawBytesData (raw byte[]), typed deserialization. ### ⚠️ Temporary: JSON-in-Binary Request Parameters @@ -45,6 +45,7 @@ Client→server request parameters use a JSON-inside-Binary envelope — cross-c ## Testing +- ⚠️ **Release config skips test projects (verification trap)** — `AyCode.Core.sln`'s **Release** configuration leaves several MSTest projects unmapped (no `Build.0` in the `.sln` `ProjectConfigurationPlatforms`), so a **Release-only build/publish does NOT compile the tests**. A test-side break (stale `using`, moved/renamed type, broken reference) therefore stays **hidden in Release** and surfaces only in **Debug**. When a change touches types the tests reference (e.g. relocating a class across assemblies — see `LLMP-DEC-68`), build/verify in **Debug** too — a green Release build alone is not proof the tests compile. (The same `.sln` per-project mapping governs the `SGenOnly`/`Product` configs — see `Directory.Build.props`.) - **MSTest** framework across all test projects. - **Abstract test bases** with `AcBase_` prefixed methods for reusable test logic. - **TestDataFactory** for centralized test data creation with ID sequencing. diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index fcd0b18..ee06aee 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -76,7 +76,7 @@ For full architecture see `AyCode.Services/docs/SIGNALR/README.md`. | **SignalParams** | Metadata accompanying each message (separate hub arg). Carries Status, response type, packed parameters, raw-bytes flag. `[AcBinarySerializable]`. Full spec: `AyCode.Services/docs/SIGNALR/README.md`. | | **Message Tag** | Integer identifier mapping to a method via `[SignalR(tag)]` or `[SignalRSendToClient(tag)]` attributes. | | **DynamicMethodRegistry** | Resolves message tags to `MethodInfo` at runtime. Static `ConcurrentDictionary` cache with lazy scan on miss. | -| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE/README.md`. | +| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See `AyCode.Services/docs/SIGNALR_DATASOURCE/README.md`. | | **AcBinaryHubProtocol** | Unsealed base `IHubProtocol` replacing SignalR's JSON+Base64 with `AcBinarySerializer`. Protocol name: `"acbinary"`. Full spec (write/read paths, chunked framing): `AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md`. | | **AyCodeBinaryHubProtocol** | Consumer-derived protocol (per-message header with type resolution, `IsRawBytesData`). Registered via `AddAcBinaryProtocol(...)` DI extension on both server and client. Full spec: `AyCode.Services/docs/SIGNALR/README.md`. | | **AcBinaryHubProtocolOptions** | Mutable config for protocol registration (SerializerOptions, ProtocolMode, BufferSize, FlushTimeout, etc.). `Validate()` + `Clone()` for DI safety. Full spec: `AyCode.Services/docs/SIGNALR/README.md`. | diff --git a/docs/README.md b/docs/README.md index 99685bf..f6cc2b1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -12,8 +12,8 @@ Top-level docs for `AyCode.Core` (Layer 0 — core framework). - `AyCode.Core/docs/` — Logger, Binary serializer (paired topics: LOGGING/, BINARY/) - `AyCode.Core.Server/docs/` — Server-side logger variant (LOGGING/) -- `AyCode.Services/docs/` — Remote logger variant, SignalR, SignalR binary protocol -- `AyCode.Services.Server/docs/` — Server-side SignalR hub (SIGNALR/), SignalR DataSource (SIGNALR_DATASOURCE/) +- `AyCode.Services/docs/` — Remote logger variant, SignalR, SignalR binary protocol, SignalR DataSource (SIGNALR_DATASOURCE/) +- `AyCode.Services.Server/docs/` — Server-side SignalR hub (SIGNALR/) ## Navigation