diff --git a/.github/LLM_PROTOCOL_DECISIONS.md b/.github/LLM_PROTOCOL_DECISIONS.md index 1cdc9cc..2d54a24 100644 --- a/.github/LLM_PROTOCOL_DECISIONS.md +++ b/.github/LLM_PROTOCOL_DECISIONS.md @@ -2,6 +2,65 @@ Append-only log of decisions about the AI AGENT CORE PROTOCOL and related LLM tooling (skills, instruction files, shared conventions) across the 8 known `.github/copilot-instructions.md` files. +--- + +## Current protocol state (quick reference) + +Active as of **2026-04-24**. For the full evolution history, see the dated table near the bottom of this file. Read this summary first to orient yourself before diving into the dated entries. + +**Session start flow**: +- **Primary files**: agent reads `copilot-instructions.md` + 3 SKILL.md files (`docs-discovery`, `docs-check`, `protocol-audit`). First response's `[LOADED_DOCS: ...]` shows 4 files. +- **Inherit files**: agent reads this inherit `copilot-instructions.md` + AyCode.Core's canonical `copilot-instructions.md` (for the numbered rules) + 3 SKILL.md files. First response's `[LOADED_DOCS: ...]` shows 5 files. +- **Why pre-load**: workspace skills are NOT in Claude Code's native skill-registry; pre-loading ensures implicit triggers fire reliably. + +**AI AGENT CORE PROTOCOL** (present verbatim in each primary `.github/copilot-instructions.md`): +1. **MANDATORY OUTPUT PREFIX** — every response starts with `[LOADED_DOCS: N files (+K this turn: )]`. Short-name rule: basename by default; `TOPIC/README.md` for topic-folder READMEs (to disambiguate the many `README.md` files); extend upward for cross-project collisions. Never emit a bare `README.md` in the prefix. +2. **HARD-GATE: DOCS BEFORE CODE** — no `code_search` / source `get_file` until relevant `.md` docs are loaded. Includes **CROSS-REPO HARD-GATE** (applies to every repo entered via `own-dep-repos`) and **PER-QUESTION DOC-FIRST** (check for unloaded `.md` before grepping code). +3. **STRICT NO-RE-READ POLICY** — a doc is "in context" only if actually read via tool call in THIS conversation (session summaries / memory entries do NOT count — lossy compression). +4. **CONTEXT RECOVERY** — auto-detection triggers: session starts with a summary, mid-session compaction, or inability to quote exact content → reset `[LOADED_DOCS: NONE]` and re-read per Rule #2. +5. **EXPLICIT CONSENT FOR MODIFICATIONS** — no rewriting, creating, or deleting *any* file (code, docs, config, memory, or otherwise) without explicit user approval. + +**Docs layout — Option C "folder + topic-prefixed companions"**: +- Paired topics live in `TOPIC/` folders: `TOPIC/README.md` (main, canonical) + `TOPIC/TOPIC_ISSUES.md` + `TOPIC/TOPIC_TODO.md` + optional `TOPIC/TOPIC_.md`. +- Single-file reference docs (`ARCHITECTURE.md`, `CONVENTIONS.md`, `GLOSSARY.md`, `DTOS.md`, etc.) remain flat at the `docs/` root. +- Each `docs/` root has a `README.md` index listing the topics. +- Project-specific variants of framework topics (e.g., the server-side logger in `AyCode.Core.Server`, the Nop bridge in `Mango.Nop.Services`) get their own `TOPIC/README.md` under the variant project's `docs/`. +- `.csproj` files use recursive glob: `` so Pattern B nesting auto-shows in Visual Studio Solution Explorer without manual entries. + +**Instruction files — 8 total**: +- **5 primary** files contain the full numbered AI AGENT CORE PROTOCOL: `AyCode.Core`, `AyCode.Blazor`, `Libraries`, `FruitBank` (Mango/Source), `FruitBankHybridApp`. +- **3 inherit** files reference AyCode.Core's protocol via blockquote (no duplicated numbered rules): `Mango.Nop.Core` (sub-project framework), `Nop.Plugin.Misc.AIPlugin` (plugin source), `Mango.FruitBank` (nopCommerce deployment host). +- Each non-Core file has `## Shared Agent Skills` + `## Protocol History` sections. + +**Issue / TODO / Decision ID format** (globally unique): +- Format: `{TOPIC}-{TYPE}-{N}` (e.g., `LOG-I-5`, `SIG-T-3`, `BIN-B-1`, `XCUT-I-1`, `LLMP-DEC-4`) +- **Topic codes**: `LOG` (logger), `SIG` (SignalR), `SBP` (SignalR binary protocol), `BIN` (binary serializer), `TOON` (Toon serializer), `GRID` (MGGRID), `XCUT` (cross-cutting), `LLMP` (LLM-protocol meta — uses `DEC` type only). +- **Type codes**: `I` = issue, `T` = TODO, `B` = bug, `C` = critical (severity override), `DEC` = decision (LLMP-only). +- **Counter scope**: per (topic, type) pair. `LOG-I-1` and `LOG-T-1` independent. +- **Append-only**: IDs never change once assigned. Full registry + rules: `AyCode.Core/.github/skills/docs-check/references/TOPIC_CODES.md`. + +**Agent skills — all three at `AyCode.Core/.github/skills/`**: +- **`docs-discovery`** — extracts topic tokens from user requests, globs `**/docs/TOPIC/**/*.md` (+ companion patterns), loads the paired main+ISSUES+TODO set before source-code search. Updates LOADED_DOCS via Rule #1 format. +- **`protocol-audit` v2.2** — reads `references/REPOS.md` and audits all 8 instruction files: Common invariants C1-C4 for all (C4 = `## Session Setup` presence, added in LLMP-DEC-33); Primary-only (P1-P10) for the 5 primary; Inherit-only (I1-I3) for the 3 inherit; Cross-cutting (X1, X3) for primary, (X1, X2) for non-Core primary, (X1, X2) for inherit. Reports PASS/FAIL per invariant-file cell, with patch suggestions. Does NOT auto-apply patches. +- **`docs-check` v1.0** — end-of-code-modifying-response skill: evaluates loaded `.md` for drift vs code, missing topic coverage, csproj-glob registration gaps, and new issue/TODO candidates. Emits the `[DOCUMENTATION CHECK]` section (or `[DOCUMENTATION CHECK] None.` single-line). Read-only on loaded docs; all patches as proposals per Rule #5. + +**Docs-sync enforcement (`[DOCUMENTATION CHECK]` at end of code-modifying responses)**: +- **Keep all .md in sync** — if code and loaded docs disagree, notify and ask (do NOT silently edit). +- **Identify missing documentation** — if a frequently-used pattern isn't documented in main topic docs, propose an entry. +- **Topic-based separation** — split logically distinct topics into separate `.md` files; keep concise and LLM-oriented. +- **File registration (scope: outside existing glob only)** — if a new `.md` falls outside the nearest `.csproj`'s recursive globs (`docs\**\*.md`, `**\README.md`), surface the needed glob patch. Common case (doc under `docs/`, folder-README) is zero-cost via csproj globs set up cluster-wide. +- **Issue / TODO surfacing** — surface concrete, non-duplicate, actionable `_ISSUES.md` / `_TODO.md` candidates as draft entries (priority P1-P3, rationale, code ref — **NO ID**, user assigns at apply-time). **Prerequisites (ALL must hold):** companion is in `LOADED_DOCS`; high-confidence (quotable evidence); concrete; not duplicate. **Max 3 items per response.** Also: surface `Status: FIXED` suggestions when a listed issue is fixed in the current response. +- **Empty-state output** — if `[DOCUMENTATION CHECK]` has nothing to report: `[DOCUMENTATION CHECK] None.` single line, no padding. + +**Decision Log governance (this file)**: +- **Append-only.** Never rewrite or delete entries. Reversals and refinements get *new* entries that reference the superseded one by ID (e.g., "LLMP-DEC-26 superseded by LLMP-DEC-27"). +- **Entries have `LLMP-DEC-N` IDs** (topic=LLMP, type=DEC — see TOPIC_CODES.md). Sequential, append-only, never renumber. Reference entries by ID for unambiguous citation from code comments, other `.md` files, or cross-repo. +- **Archive annually** once the log reaches ~50 entries (split to `LLM_PROTOCOL_DECISIONS_.md`). IDs stay unique across archives. +- **`Affected` column** uses the scope codes defined below. +- **User consent required** for every new entry. + +--- + ## Purpose Quick lookup for **why** a rule is the way it is, to avoid: @@ -45,28 +104,48 @@ The "primary" vs "inherit" distinction is defined in `protocol-audit/references/ ## 2026 -| Date | Decision | Rationale | Affected | -|------------|-------------------------------------------------------|---------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------| -| 2026-04-20 | Added CROSS-REPO HARD-GATE to Rule #2 | Docs-before-code was applied only to the own repo, not external deps | `5× primary` | -| 2026-04-20 | Added PER-QUESTION DOC-FIRST to Rule #2 | LLM skipped checking for relevant `.md` before jumping to code search | `5× primary` | -| 2026-04-21 | Expanded Rule #5 scope: "code/files" → "any file..." | Memory / config / docs edits also need consent; old wording too narrow | `5× primary` | -| 2026-04-22 | Added Auto-detection triggers to Rule #4 | "realize" was subjective; post-compaction reset to `[LOADED_DOCS: NONE]` didn't fire reliably | `5× primary` | -| 2026-04-22 | Added "in context" definition to Rule #3 | Summary content was mistaken for actually-loaded docs (lossy compression) | `5× primary` | -| 2026-04-22 | Unified AyCode.Blazor Rule #3 with the other 4 repos | Was custom "CRITICAL TOOL EXECUTION FIREWALL" variant; normalized | `AyCode.Blazor/copilot-instructions.md` | -| 2026-04-22 | "strictly maintain rule X" → "rule 3" | Per-repo numbers varied (15/18/19/20/21); unified reference | `5× primary` | -| 2026-04-22 | Created `protocol-audit` skill at repo-level | Cross-repo consistency check; `.github/skills/` chosen over personal paths (DRY) | `protocol-audit/SKILL.md` + `protocol-audit/references/REPOS.md` | -| 2026-04-22 | Added `## Shared Agent Skills` section (protocol-audit) | Discoverability of AyCode.Core-hosted skills from dependent repos | `4× non-Core primary` | -| 2026-04-23 | Created `docs-discovery` skill (parallel session) | Paired-doc (`main` + `_ISSUES` + `_TODO`) auto-loading before code search | `docs-discovery/SKILL.md` + `AyCode.Core/copilot-instructions.md` | -| 2026-04-23 | Extended `## Shared Agent Skills` with `docs-discovery` (parallel session) | Both skills now listed in each dependent repo | `4× non-Core primary` + `Mango.Nop.Core/copilot-instructions.md` | -| 2026-04-24 | LOADED_DOCS prefix: full list → count+delta format | Long lists became visual noise at scale; delta preserves self-commitment | `5× primary` + `docs-discovery/SKILL.md` | -| 2026-04-24 | Created this Decision Log | Institutional memory for protocol evolution; avoid re-debating resolved choices | `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md` | -| 2026-04-24 | Added `## Protocol History` section | Cross-repo discoverability of the Decision Log | `4× non-Core primary` + `Mango.Nop.Core/copilot-instructions.md` | -| 2026-04-24 | Filled empty Nop.Plugin.Misc.AIPlugin instruction file | Previously 0 bytes; now minimal inherit-pattern referencing AyCode.Core + both skills + log | `Nop.Plugin.Misc.AIPlugin/copilot-instructions.md` | -| 2026-04-24 | Filled empty Mango\FruitBank instruction file | Previously 0 bytes; now minimal inherit-pattern with ⚠️ "purpose TBD" notice | `Mango/FruitBank/copilot-instructions.md` | -| 2026-04-24 | Expanded REPOS.md: 5 repos → 8 (primary + inherit) | Audit scope was incomplete; sub-project & plugin files were missed | `protocol-audit/references/REPOS.md` | -| 2026-04-24 | Fixed Mango.Nop.Core `@repo` path (7 `..` → 6 `..`) | Previous path resolved to `H:\Aycode\...` (non-existent). Corrected relative depth from repo root | `Mango.Nop.Core/copilot-instructions.md` | -| 2026-04-24 | Resolved Mango.FruitBank purpose (nopCommerce host) | Confirmed: directory is a nopCommerce deployment for FruitBank company, hosting Nop.Plugin.Misc.AIPlugin. Layer 4 (host), not TBD. Updated content accordingly. | `Mango/FruitBank/copilot-instructions.md` | -| 2026-04-24 | `protocol-audit` v1.0 → v2.0: primary/inherit invariant split | SKILL.md now applies Common + Primary invariants to rows 1-5, Common + Inherit + Cross-cutting to rows 6-8. New invariants: P3 (Rule #1 count+delta format), X1 (Shared Agent Skills), X2 (Protocol History). | `protocol-audit/SKILL.md` + `protocol-audit/references/REPOS.md` (issues cleared) | +| ID | Date | Decision | Rationale | Affected | +|---------------|------------|-------------------------------------------------------|---------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------| +| LLMP-DEC-1 | 2026-04-20 | Added CROSS-REPO HARD-GATE to Rule #2 | Docs-before-code was applied only to the own repo, not external deps | `5× primary` | +| LLMP-DEC-2 | 2026-04-20 | Added PER-QUESTION DOC-FIRST to Rule #2 | LLM skipped checking for relevant `.md` before jumping to code search | `5× primary` | +| LLMP-DEC-3 | 2026-04-21 | Expanded Rule #5 scope: "code/files" → "any file..." | Memory / config / docs edits also need consent; old wording too narrow | `5× primary` | +| LLMP-DEC-4 | 2026-04-22 | Added Auto-detection triggers to Rule #4 | "realize" was subjective; post-compaction reset to `[LOADED_DOCS: NONE]` didn't fire reliably | `5× primary` | +| LLMP-DEC-5 | 2026-04-22 | Added "in context" definition to Rule #3 | Summary content was mistaken for actually-loaded docs (lossy compression) | `5× primary` | +| LLMP-DEC-6 | 2026-04-22 | Unified AyCode.Blazor Rule #3 with the other 4 repos | Was custom "CRITICAL TOOL EXECUTION FIREWALL" variant; normalized | `AyCode.Blazor/copilot-instructions.md` | +| LLMP-DEC-7 | 2026-04-22 | "strictly maintain rule X" → "rule 3" | Per-repo numbers varied (15/18/19/20/21); unified reference | `5× primary` | +| LLMP-DEC-8 | 2026-04-22 | Created `protocol-audit` skill at repo-level | Cross-repo consistency check; `.github/skills/` chosen over personal paths (DRY) | `protocol-audit/SKILL.md` + `protocol-audit/references/REPOS.md` | +| LLMP-DEC-9 | 2026-04-22 | Added `## Shared Agent Skills` section (protocol-audit) | Discoverability of AyCode.Core-hosted skills from dependent repos | `4× non-Core primary` | +| LLMP-DEC-10 | 2026-04-23 | Created `docs-discovery` skill (parallel session) | Paired-doc (`main` + `_ISSUES` + `_TODO`) auto-loading before code search | `docs-discovery/SKILL.md` + `AyCode.Core/copilot-instructions.md` | +| LLMP-DEC-11 | 2026-04-23 | Extended `## Shared Agent Skills` with `docs-discovery` (parallel session) | Both skills now listed in each dependent repo | `4× non-Core primary` + `Mango.Nop.Core/copilot-instructions.md` | +| LLMP-DEC-12 | 2026-04-24 | LOADED_DOCS prefix: full list → count+delta format | Long lists became visual noise at scale; delta preserves self-commitment | `5× primary` + `docs-discovery/SKILL.md` | +| LLMP-DEC-13 | 2026-04-24 | Created this Decision Log | Institutional memory for protocol evolution; avoid re-debating resolved choices | `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md` | +| LLMP-DEC-14 | 2026-04-24 | Added `## Protocol History` section | Cross-repo discoverability of the Decision Log | `4× non-Core primary` + `Mango.Nop.Core/copilot-instructions.md` | +| LLMP-DEC-15 | 2026-04-24 | Filled empty Nop.Plugin.Misc.AIPlugin instruction file | Previously 0 bytes; now minimal inherit-pattern referencing AyCode.Core + both skills + log | `Nop.Plugin.Misc.AIPlugin/copilot-instructions.md` | +| LLMP-DEC-16 | 2026-04-24 | Filled empty Mango\FruitBank instruction file | Previously 0 bytes; now minimal inherit-pattern with ⚠️ "purpose TBD" notice | `Mango/FruitBank/copilot-instructions.md` | +| LLMP-DEC-17 | 2026-04-24 | Expanded REPOS.md: 5 repos → 8 (primary + inherit) | Audit scope was incomplete; sub-project & plugin files were missed | `protocol-audit/references/REPOS.md` | +| LLMP-DEC-18 | 2026-04-24 | Fixed Mango.Nop.Core `@repo` path (7 `..` → 6 `..`) | Previous path resolved to `H:\Aycode\...` (non-existent). Corrected relative depth from repo root | `Mango.Nop.Core/copilot-instructions.md` | +| LLMP-DEC-19 | 2026-04-24 | Resolved Mango.FruitBank purpose (nopCommerce host) | Confirmed: directory is a nopCommerce deployment for FruitBank company, hosting Nop.Plugin.Misc.AIPlugin. Layer 4 (host), not TBD. Updated content accordingly. | `Mango/FruitBank/copilot-instructions.md` | +| LLMP-DEC-20 | 2026-04-24 | `protocol-audit` v1.0 → v2.0: primary/inherit invariant split | SKILL.md now applies Common + Primary invariants to rows 1-5, Common + Inherit + Cross-cutting to rows 6-8. New invariants: P3 (Rule #1 count+delta format), X1 (Shared Agent Skills), X2 (Protocol History). | `protocol-audit/SKILL.md` + `protocol-audit/references/REPOS.md` (issues cleared) | +| LLMP-DEC-21 | 2026-04-24 | Docs layout: paired topics migrated to `TOPIC/README.md + TOPIC/TOPIC_*.md` pattern (Option C) | Single `docs/` folder with flat `TOPIC_*.md` files was noisy at scale; folder grouping reinforces the "folder navigation rule" and gives each topic a canonical `README.md` entry point. Option C (README.md main + topic-prefixed companions) chosen over pure Pattern B for (a) glob-pattern compatibility with old `**/TOPIC*.md`, (b) unique LOADED_DOCS basenames for companion files, (c) GitHub README auto-render preserved. Single-file topics (ARCHITECTURE, GLOSSARY, CONVENTIONS, DTOS, etc.) kept flat. | 13 `docs/` folders across 5 repos (AyCode.Core, AyCode.Blazor, Libraries, FruitBankHybridApp, Nop.Plugin.Misc.AIPlugin); ~35 file renames/moves; 13 new `README.md` index files; 4 csproj files converted from explicit file-lists to recursive glob (`docs\**\*.md`); `docs-discovery/SKILL.md` Step 2 glob patterns updated to match Pattern B layout | +| LLMP-DEC-22 | 2026-04-24 | Rule #1 LOADED_DOCS format: "basenames only" → "shortest unique short names" | After the Pattern B docs migration, many `README.md` files share the same basename across different topic folders (e.g., `LOGGING/README.md`, `BINARY/README.md`, `SIGNALR/README.md`). A bare `README.md` in the LOADED_DOCS prefix is ambiguous and breaks the self-commitment / no-re-read mechanism. New rule: basename by default (already unique for topic-prefixed companions and flat single-file topics); `TOPIC/README.md` for topic-folder READMEs; further disambiguation upward for cross-project collisions. | `5× primary` + `docs-discovery/SKILL.md` Step 4 | +| LLMP-DEC-23 | 2026-04-24 | Full cross-reference cleanup after Pattern B migration (batch sed + targeted fixes) | After the docs migration, ~45 `.md` files contained outdated paths (flat `TOPIC_*.md` refs that are now `TOPIC/TOPIC_*.md`; renamed mains `LOGGING_SERVER.md`→`LOGGING/README.md` etc.; pre-existing depth bugs like `AyCode.Core/docs/LOGGING.md` that were always wrong depth; cross-folder sibling refs inside `SIGNALR_BINARY_PROTOCOL/` pointing to `SIGNALR/` topics by basename). Three-pass cleanup: (1) batch sed on 44 files for `docs/TOPIC*.md` → Pattern B paths; (2) second sed pass for pre-existing `AyCode.Core/docs/{LOGGING,BINARY,SIGNALR}/` → `AyCode.Core/{AyCode.Core,AyCode.Services}/docs/...` depth correction (with placeholder guard to avoid double-substitution); (3) targeted Edits for cross-folder sibling refs and project-level README doc-listing tables. Final grep: zero old-style references remain. | ~45 `.md` files across all 5 repos; AyCode.Core/copilot-instructions.md rule 8 + 19; AyCode.Blazor rule 14; FruitBankHybridApp rules 15+20; 7 project-level `README.md` "doc listing" tables; 9 MGGRID sibling refs; 5 cross-folder SIGNALR↔SIGNALR_BINARY_PROTOCOL refs | +| LLMP-DEC-24 | 2026-04-24 | Phantom-reference fixes: `docs/ARCHITECTURE.md` from `.github/` + BINARY/README.md serializer-overview target | (1) `AyCode.Core/.github/copilot-instructions.md:96` had `docs/ARCHITECTURE.md#framework-vs-consumer-boundary` which resolved to `.github/docs/ARCHITECTURE.md` (non-existent) relative to the file's own location. Fixed to `../docs/ARCHITECTURE.md#framework-vs-consumer-boundary` so the path resolves to the existing repo-root `docs/ARCHITECTURE.md`. (2) `AyCode.Core/AyCode.Core/docs/BINARY/README.md:22` pointed to `docs/ARCHITECTURE.md` for "Serialization overview (Toon vs AcBinary vs AcJson)". The new `TOON/README.md` (created by a parallel session) uses the more canonical `../../Serializers/README.md` — the serializer-framework-level README that genuinely compares all three formats. Aligned BINARY/README.md to the same target for consistency. | `AyCode.Core/.github/copilot-instructions.md` + `AyCode.Core/AyCode.Core/docs/BINARY/README.md` | +| LLMP-DEC-25 | 2026-04-24 | Added "Current protocol state (quick reference)" section to the top of the Decision Log | The log grows monotonically (append-only); readers landing on it shouldn't have to scan every dated entry to learn the currently-active protocol. The new top-of-file summary captures Rules #1-5, docs layout convention, instruction-file inventory (5 primary + 3 inherit), agent skills, and Decision Log governance. Dated entries remain the source of truth for *why* each decision was made; the summary is the *what* at a glance. Archival plan noted: yearly split at ~50 entries. | `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md` | +| LLMP-DEC-26 | 2026-04-24 | Extended docs-sync rule with two sub-rules: `File registration` + `Issue / TODO surfacing` (initial version; superseded by 2026-04-24 revision below) | (1) **File registration** — every new `.md` must be visible in Visual Studio Solution Explorer, which requires a `` entry (or matching recursive glob) in the nearest `.csproj`. (2) **Issue / TODO surfacing** — surface concrete bugs/TODO candidates in `[DOCUMENTATION CHECK]` when not already in loaded `_ISSUES.md` / `_TODO.md`. Initial calibration: "concrete + non-duplicate + actionable — silence beats noise". **This version was revised after two external LLM critiques — see the next entry.** | `5× primary` (docs-sync rule: AyCode.Core #18, AyCode.Blazor #13, Libraries #16, FruitBank #14, FruitBankHybridApp #14) | +| LLMP-DEC-27 | 2026-04-24 | Revision of the two new sub-rules after two-session LLM review + proactive csproj glob expansion | Two external LLM critiques identified gaps in the initial sub-rule text. Applied revisions: (A) **File registration scope-limit** — check only if the new file falls OUTSIDE an existing recursive glob (zero-cost skip for the common `docs/`-path case). (B) **Issue / TODO surfacing guardrails** — added 4 explicit prerequisites (companion must be in `LOADED_DOCS` for reliable duplicate check; high confidence with quotable evidence; concrete observation; not duplicate). Added **volume cap max 3 per response**. Removed **ID assignment** from draft entries (user assigns at apply-time to avoid parallel-session collisions). Added **status-update-on-fix** clause (fixing an issue listed in loaded `_ISSUES.md` → surface `Status: FIXED` suggestion). (C) **ENFORCEMENT empty-state guard** — if nothing to report: `[DOCUMENTATION CHECK] None.` single line (no prose padding). (D) **Proactive csproj glob expansion** — added `` to 9 csproj files already having `docs/**` glob; added the same pattern (without `docs/**` exclusion) to 5 csproj files lacking any md glob. Together these cover folder-level `README.md` files in code directories (Loggers, Serializers, SignalRs, DbContexts, LogItems, etc.), making them visible in VS Solution Explorer without per-file rule trigger. The rule sub-rule A now mostly fires only for truly unusual locations. Cross-reference added between sub-rule B and the existing "Identify missing documentation" clause (different targets: main docs vs companions). | `5× primary` instruction files (docs-sync rule revision); **14 csproj files** (glob expansion): 9 with `docs/**` + `**/README.md`, 5 with only `**/README.md` | +| LLMP-DEC-28 | 2026-04-24 | Created `docs-check` skill; docs-sync rule → short pointer | Observation: both Claude Code and Copilot interpret and follow *skills* more reliably than numbered rules (explicit activation, step-by-step SKILL.md, description-matching forces deliberate thought). The docs-sync rule had grown into a multi-step procedure (drift detection, missing docs, topic separation, file registration, issue/TODO surfacing with 4 prereqs + volume cap + status-update-on-fix, empty-state handling) — exactly the shape of a skill, not a rule. Created `AyCode.Core/.github/skills/docs-check/SKILL.md` (v1.0) encapsulating the full procedure in 7 steps + edge cases + negative examples. Rule #18 (and equivalents) shortened in each primary file to a single pointer: *"at the end of EVERY code-modifying response, invoke the docs-check skill ..."*. Removed ~40 lines of duplicated procedure per primary file (×5 = ~200 lines of redundant text across the workspace). Skill is read-only on loaded docs (no new `Read`/`Grep` during invocation); all patches surface as proposals (Rule #5). | **1 new skill**: `docs-check/SKILL.md`; **5 primary** copilot-instructions.md (docs-sync rule shortened); **7 non-Core** files (`## Shared Agent Skills` section extended with the `docs-check` bullet: 4 primary non-Core + 3 inherit) | +| LLMP-DEC-29 | 2026-04-24 | `protocol-audit` v2.0 → v2.1 | (a) X1 invariant extended to expect all three skills in `## Shared Agent Skills` (was two: docs-discovery + protocol-audit; now adds docs-check). (b) New **X3** invariant: the docs-sync rule in each primary file points to the `docs-check` skill (checks for backtick-wrapped `` `docs-check` `` and the skill path). (c) Applicability matrix extended: X3 applies to all 5 primary files, N/A for the 3 inherit files (they don't have the numbered docs-sync rule). | `protocol-audit/SKILL.md` | +| LLMP-DEC-30 | 2026-04-24 | Introduced globally-unique issue/TODO/decision ID system + batch-migrated existing entries | Three reasons: (a) make IDs like `ISSUE-01` unambiguous outside their containing file (DB natural-key ready, code-comment friendly, cross-repo clear); (b) remove collision risk across parallel LLM sessions (per-topic+type counter is a smaller concurrency domain than global counter); (c) standardize the inconsistent mix of existing prefixes (`PROTO-`, `DISPATCH-`, `DS-`, `CONN-`, `XCUT-`, `DESER-`, `SER-`, `SGEN-`, `BWO-`, and per-file `ISSUE-NN`/`TODO-NN`). **Format:** `{TOPIC}-{TYPE}-{N}` — topic code from registry, type = `I`(ssue) / `T`(ODO) / `B`(ug) / `C`(ritical severity override) / `DEC` (LLMP-only), N is per-topic-per-type counter starting at 1 (no zero-padding). **Topic codes:** `LOG`, `SIG`, `SBP`, `BIN`, `TOON`, `GRID` (avoids `MG`-prefix-collision with `Mg*` class naming), `XCUT`, `LLMP`. Sub-categories (`PROTO`, `DISPATCH`, `CONN`, etc.) move from ID to body header tag (`## SIG-I-1 [PROTO]: ...`). Registry maintained in `docs-check/references/TOPIC_CODES.md`. Migration: ~50 existing entries renumbered across 10 files (LOGGING, SIGNALR, SBP, BINARY, TOON topic folders) + cross-references updated workspace-wide; found and mapped the previously-uncatalogued `BWO-4` entry → `BIN-I-15`. | `docs-check/references/TOPIC_CODES.md` (new); `docs-check/SKILL.md` (Step 5 draft entry format + topic registry reference); **10 topic doc files** (IDs renumbered); cross-ref updates across ~20 `.md` files in the workspace | +| LLMP-DEC-31 | 2026-04-24 | Consolidated XCUT cross-cutting entries into dedicated `docs/XCUT/` folder | Before: cross-cutting entries (currently just `XCUT-I-1: JSON-in-Binary request parameters`) were duplicated — canonical body in `BINARY_ISSUES.md`, short cross-ref in `SIGNALR_ISSUES.md`. This pattern breaks down as more XCUT entries arrive (every added cross-cutting topic needs duplicate maintenance, and the concept of "which side is canonical" is arbitrary). **Now:** canonical home is `AyCode.Core/AyCode.Core/docs/XCUT/` (topic-folder at project level, consistent with LOGGING/, BINARY/, TOON/ placement). Contains `README.md` (explaining XCUT concept + convention), `XCUT_ISSUES.md` (canonical entries), `XCUT_TODO.md` (template + future entries). Each affected topic's file keeps a **short cross-ref** pointing to the canonical entry; body size per topic drops to ~2 lines. Migrated `XCUT-I-1` — full body now lives in `XCUT_ISSUES.md`, with cross-refs from `BINARY_ISSUES.md` and `SIGNALR_ISSUES.md`. Also updated: `BINARY_TODO.md#bin-t-1` Related-link now points to canonical; `AyCode.Core/docs/CONVENTIONS.md` JSON-in-Binary section references canonical. | New folder `AyCode.Core/AyCode.Core/docs/XCUT/` (3 files); updated `BINARY_ISSUES.md`, `SIGNALR_ISSUES.md` (XCUT sections); `BINARY_TODO.md`, `AyCode.Core/docs/CONVENTIONS.md` (related-link updates); `AyCode.Core/AyCode.Core/docs/README.md` (topic list extended) | +| LLMP-DEC-32 | 2026-04-24 | SKILL.md generalization + workspace-meta-tooling exception documented | Two related concerns raised: (1) SKILL.md files had hardcoded specific repo/project names (e.g., "5 AyCode/Mango repos (AyCode.Core, AyCode.Blazor, Libraries, FruitBank, FruitBankHybridApp)" in protocol-audit's `description`; "`AyCode.Core/AyCode.Core/docs/XCUT/`" in docs-check's Step 5). Skills should describe BEHAVIOR; specific data lives in `references/` files. (2) The `.github/skills/` registry files (`REPOS.md`, `TOPIC_CODES.md`) list higher-layer products (Layer 3 consumers like FruitBank) — a literal Framework-First Design Principle violation if interpreted strictly. **Decisions:** (a) Skills stay **universal** (one copy under `AyCode.Core/.github/skills/`, referenced by all other repos via Shared Agent Skills section) — per-layer skill duplication rejected as maintenance-hostile. (b) SKILL.md files generalized to read from `references/` registries instead of hardcoding names. (c) The `.github/` folder is **explicitly exempted** from Framework-First: it is workspace-configuration, not framework code. Rule added to AyCode.Core/copilot-instructions.md's Framework-First Design Principle section. Scope notes added to `REPOS.md` and `TOPIC_CODES.md` at their tops. **Future escape hatch:** if a skill becomes truly layer-specific (e.g., `fruit-measurement-audit` for FruitBank-only concerns), it goes in that layer's `.github/skills/`, not in AyCode.Core's. | `protocol-audit/SKILL.md` (description + invariant X3 + applicability note generalized); `docs-check/SKILL.md` (Step 5 XCUT path reference generalized); `docs-discovery/SKILL.md` (consumer-specific class-name example swapped for a placeholder); `AyCode.Core/.github/copilot-instructions.md` (Framework-First section + "Exception for workspace meta-tooling" paragraph); `protocol-audit/references/REPOS.md` (scope note header); `docs-check/references/TOPIC_CODES.md` (scope note header) | +| LLMP-DEC-33 | 2026-04-24 | `protocol-audit` v2.1 → v2.2 — added C4 invariant for Session Setup section presence | New common invariant `C4` enforces `## Session Setup` section presence across all 8 files, with references to the three skill SKILL.md files (and for inherit files, also the canonical host's `copilot-instructions.md`). Ensures the new Session Setup rule is mechanically verifiable. Applicability matrix updated: C-invariant range extended from C1-C3 to C1-C4. | `protocol-audit/SKILL.md` (version bump; new C4 invariant; applicability matrix update) | +| LLMP-DEC-34 | 2026-04-24 | Token-economics principle made explicit — amortization over multi-turn sessions is the central design principle | **Observation**: a validation session exposed an LLM reasoning bias — when asked whether the protocol's upfront cost was worth it, the agent's first answer categorized `protocol-audit/SKILL.md` (~6K tokens) as "wasteful for this specific turn" because the turn didn't invoke it. A single user reminder ("the session just started") was enough for the LLM to self-correct and reframe the cost as amortizing over the full session lifetime. **Issue**: the Session Setup rule's rationale said "one-time cost per session" but did NOT explicitly name the single-turn optimization trap. LLMs default to per-turn local optimization and can mis-judge pre-loaded content as wasteful, especially in early session turns or quick one-off queries. **Fix**: extended the `## Session Setup` "Why mandatory" paragraph in all 8 `copilot-instructions.md` files (5 primary + 3 inherit) with an explicit `**Amortization — critical, do NOT re-evaluate per-turn**` note. The note states: (a) cost is measured over the ENTIRE session; (b) the first domain question alone recoups it; (c) the alternative (repeated source-code searches per turn) costs 10-20K tokens per turn with lower quality; (d) the design depends on cross-turn amortization + Rule #3 (no-re-read) + Rule #4 (context recovery as only exception); (e) this is the **central token-economics principle** of the entire stack. Proactively inoculates future sessions against the single-turn-optimization bias that was observed. | `5× primary` + `3× inherit` `copilot-instructions.md` (Session Setup "Why mandatory" paragraph extended with Amortization sub-paragraph) | +| LLMP-DEC-35 | 2026-04-24 | Added mandatory Session Setup — pre-load 3 SKILL.md files at session start | **Problem:** Claude Code does NOT populate workspace skills into its system-reminder / native skill-registry. Previous design relied on instruction-driven invocation (rule pointer → LLM reads SKILL.md on first trigger). This is **fragile**: the agent might not recognize implicit triggers (e.g., "this is a domain question → load docs-discovery"), might forget to load the SKILL.md, or might delay loading until after a code-search has already started. A test session confirmed the risk: an LLM answered a rule-listing question without loading any skill, then retroactively loaded them only when asked directly — the first domain question would have missed the docs-discovery trigger. **Solution:** Added `## Session Setup` section to all 8 `copilot-instructions.md` files (primary and inherit). Rule mandates pre-loading of all three SKILL.md files (`docs-discovery`, `docs-check`, `protocol-audit`) at session start, immediately after reading the main `copilot-instructions.md`. For inherit files, AyCode.Core's `copilot-instructions.md` is also in the mandatory load set (because inherit files do not duplicate Rules #1-5 — the canonical host has them). Expected `[LOADED_DOCS: ...]` prefix on first response: 4 files for primary, 5 for inherit. **Cost:** ~10-13K tokens one-time per session; Rule #3 (no-re-read) prevents repeated reads. **Reliability gain:** implicit triggers fire reliably for the entire session. The `## Shared Agent Skills` section intros (in 7 non-Core files) were also updated to cross-reference the Session Setup mandate. | `5× primary` + `3× inherit` `copilot-instructions.md` (added `## Session Setup` section); `4× non-Core primary` + `3× inherit` = `7× non-Core` (Shared Agent Skills intro updated to cross-reference Session Setup) | +| LLMP-DEC-36 | 2026-04-24 | Migrated Toon docs to `TOON/README.md` + `TOON_*.md` (Option C) | Last AyCode.Core domain still on flat layout. Absorbed the orphan `ToonExtendedInfo.txt` at repo root; added paired `TOON_ISSUES.md` / `TOON_TODO.md` per convention. Verified format spec against actual code (`MetaWriter`, `DataSection`, `TypeDefinitions`, `Descriptions`), not just the legacy marketing-style txt — surfaced richer reality (placeholder system, enum backing-field detection, navigation metadata, type relation constants) absent from the old txt. Serialize-only status recorded as `TOON_TODO.md#toon-t-6`. | `AyCode.Core/docs/TOON/` (8 new files) + `AyCode.Core/docs/README.md` (topic list) + `AyCode.Core/Serializers/Toons/README.md` (stub pointer) + `ToonExtendedInfo.txt` (removed) | +| LLMP-DEC-37 | 2026-04-24 | `docs-discovery` SKILL.md hardening: literal-path tilalom + false-empty guardrail (cross-session validation feedback) | **Incident**: a parallel Copilot session executed `docs-discovery` for a logger-review query but substituted a literal `AyCode.Core/docs/LOGGING/...` path for the spec'd `**/docs/{TOKEN}/**/*.md` glob. The literal path missed AyCode.Core's project-level docs layout (`//docs/LOGGING/`), yielded 0 matches, and Copilot concluded "docs are empty" → fell through to code-only review, skipping the loaded paired-docs set (8 known issues, 11 prioritized TODOs). **Fix scope**: minimal — 2 sentences appended to existing steps, NOT new rules. (a) Step 2 end: explicit prohibition on substituting literal `/docs/...` for the recursive `**/docs/...` glob, with one-line rationale that `**` matches both repo-root and project-level layouts in a single pass. (b) Step 3 end: false-empty guardrail — if glob returns 0 matches OR all matches are 0-byte, re-validate the glob (typo? literal substituted?) and retry once with the same token under a corrected `**/docs/...` pattern (NEVER an ad-hoc path guess) before falling through to code-search. **Rejected alternatives**: an early Copilot draft proposed 5 new numbered rules + 5 new SKILL steps (incl. a Step 0 "load every README.md" pass). Evaluation: most were redundant with existing rules (vs. hard-gate, paired-docs Step 5, Rule #1 prefix), one was based on `**`-semantics misunderstanding (proposed "two-pass" globs where the second pattern is a strict subset of the first), one was actively harmful to token economy (Step 0). Net: ~80% reduction in proposed protocol surface, same bug-class coverage. | `docs-discovery/SKILL.md` (Step 2 end + Step 3 end) | +| LLMP-DEC-38 | 2026-04-24 | Retroactively assigned `LLMP-DEC-N` IDs to existing Decision Log entries | When the globally-unique ID system was introduced (LLMP-DEC-30) with the `LLMP` topic code for LLM-protocol meta-decisions using `DEC` type, the existing ~37 Decision Log entries remained keyed by date alone. This meant the design intention (entries have unambiguous string IDs for cross-referencing) wasn't realized for historical entries. Fix: added an `ID` column as the leftmost in the 2026 table, and populated LLMP-DEC-1 through LLMP-DEC-37 in chronological order (retroactive assignment, this entry is LLMP-DEC-38 as the first ID-aware addition). Entries now referenceable by ID from code comments, other `.md` files, PR descriptions, or future DB migration. The `LLMP-DEC-N` format is append-only (IDs never renumber); superseded decisions get new entries that reference old ones by ID (e.g., "LLMP-DEC-26 superseded by LLMP-DEC-27"). | `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md` (ID column + 37 retroactive IDs) | +| LLMP-DEC-39 | 2026-04-24 | `docs-discovery` SKILL.md Step 2 restructuring — recursive `**/` wildcard requirement moved to prominent CRITICAL section at TOP (second iteration; LLMP-DEC-37 proved insufficient) | **Incident**: a second Copilot session exhibited the same literal-path bug that LLMP-DEC-37 was supposed to prevent — it synthesized `AyCode.Core/docs/LOGGING/README.md` (a path that doesn't exist; the real docs are at `AyCode.Core/AyCode.Core/docs/LOGGING/`, one level deeper at project-level) instead of using the recursive `**/docs/LOGGING/**/*.md` glob. LLMP-DEC-37 had appended a warning to the END of Step 2, which was not prominent enough — Copilot read Step 2, formed its own mental model of "usual docs paths", and never reached the trailing caution. **Fix**: moved the warning to the TOP of Step 2 as a `### ⚠️ CRITICAL` subsection. Content expanded: (a) explicit depth-level enumeration (repo-root `/docs/TOPIC/`, project-level `//docs/TOPIC/`, nested), (b) concrete failure-mode walkthrough showing the specific "know the repo → synthesize shallow path → 0 matches → false-empty conclusion" pattern that fires for Pattern B layouts, (c) correct-vs-wrong form side-by-side examples, (d) absolute rule: NEVER drop the leading `**/` even when the repo is known. The old trailing warning was consolidated to a back-reference to the new top section to keep Step 2 from splitting the message. User rejected intermediate proposals (byte-count verification, flat `**/docs/TOPIC.md` defensive pattern) as "drótozás" (hardcoding) — fix stays structural, no glob pattern additions. | `docs-discovery/SKILL.md` (Step 2 restructuring; supersedes LLMP-DEC-37's Step-2-end placement) | +| LLMP-DEC-40 | 2026-04-24 | Rule #1 clarification: removed false "globally unique by basename" claim for `copilot-instructions.md` | **Observation**: a parallel Claude Code self-audit session noticed a literal contradiction in Rule #1. The short-name rule line 18 said `copilot-instructions.md` is "globally unique by basename" — true for a typical session (1 active repo = 1 loaded copilot-instructions.md) but FALSE for `protocol-audit` sessions, which load all 8 `copilot-instructions.md` files simultaneously. The claim conflicts with the cross-project-disambiguation rule one line above (line 17) whenever the audit use-case fires. The LLM correctly identified the ambiguity and proposed a patch. **Fix**: replaced the parenthetical "globally unique by basename" with an explicit acknowledgment of the `protocol-audit` collision case + pointer to the already-existing cross-project-disambiguation rule above. New wording uses concrete disambiguation examples (`AyCode.Core/copilot-instructions.md`, `FruitBankHybridApp/copilot-instructions.md`). The `.github/` prefix tiltás is retained — now justified as "implicit location" rather than "already unique". **Deferred (explicit YAGNI)**: proposed new `protocol-audit` C5 invariant (semantic validation: Rule #1 basename-uniqueness claims must match the actual file-set in REPOS.md). Rejected for now — one known instance isn't enough to justify invariant surface / false-positive risk; revisit if a second similar contradiction appears. | `5× primary` `copilot-instructions.md` (Rule #1 line 18 patched) | ## Known follow-ups diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 47b3659..4374521 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -4,17 +4,25 @@ You are operating in a multi-repo, documentation-first architecture. You MUST STRICTLY follow this protocol for every response. Failure to do so will break the workspace rules. 1. **MANDATORY OUTPUT PREFIX:** Your response MUST begin on the very first line with this format: - `[LOADED_DOCS: N files (+K this turn: )]` + `[LOADED_DOCS: N files (+K this turn: )]` - `N` = total count of `.md` files currently in your context (across all loaded docs in this conversation) - `K` = count of `.md` files newly loaded during THIS response (may be 0) - - If `K > 0`: list the newly loaded **basenames only** (no paths) after `:` + - If `K > 0`: list the newly loaded files as **shortest unique short names** (see naming rule below) - If `K = 0`: write `[LOADED_DOCS: N files, no new loads]` - If `N = 0`: write `[LOADED_DOCS: NONE]` + **Short-name rule (for each loaded `.md` file):** + - **Default:** use the basename only. Works for all topic-prefixed companions (`LOGGING_ISSUES.md`, `BINARY_FEATURES.md`) and flat single-file topics (`ARCHITECTURE.md`, `GLOSSARY.md`, `CONVENTIONS.md`) — these are already unique. + - **For `README.md` files:** always include the immediate parent folder as prefix → `TOPIC/README.md` (e.g., `LOGGING/README.md`, `BINARY/README.md`). Never write a bare `README.md` in the prefix because multiple folders have one. + - **Cross-project disambiguation:** if two files share the same short name (rare — e.g., `docs/README.md` from two different projects), extend the prefix one more level: `PROJECT/docs/README.md`. Always use the shortest unique form. + - **Never use absolute paths.** Never include the `.github/` prefix for `copilot-instructions.md` — the `.github/` location is implicit. When multiple are loaded simultaneously (typical in `protocol-audit`), apply the cross-project disambiguation above: `AyCode.Core/copilot-instructions.md`, `FruitBankHybridApp/copilot-instructions.md`, etc. + **Examples:** - `[LOADED_DOCS: NONE]` — nothing loaded yet - `[LOADED_DOCS: 1 files, no new loads]` — only `copilot-instructions.md` loaded earlier, nothing new this turn - - `[LOADED_DOCS: 4 files (+3 this turn: LOGGING.md, LOGGING_ISSUES.md, LOGGING_TODO.md)]` — 3 new this turn + - `[LOADED_DOCS: 4 files (+3 this turn: LOGGING/README.md, LOGGING_ISSUES.md, LOGGING_TODO.md)]` — logger topic folder loaded + - `[LOADED_DOCS: 7 files (+3 this turn: SIGNALR/README.md, SIGNALR_ISSUES.md, SIGNALR_TODO.md)]` — SignalR topic loaded (LOGGING was already in context) + - `[LOADED_DOCS: 5 files (+2 this turn: ARCHITECTURE.md, GLOSSARY.md)]` — top-level reference docs loaded (flat, no folder prefix needed) This prefix is MANDATORY on **EVERY** response (not just the first, not just when loading happens). It serves two purposes: **(a)** user-visible compliance signal, and **(b)** self-commitment for the no-re-read rule — in subsequent turns you read your own prior prefix from the conversation to enforce Rule #3. Dropping the prefix breaks both. @@ -50,6 +58,22 @@ You are operating in a multi-repo, documentation-first architecture. You MUST ST 5. **EXPLICIT CONSENT FOR MODIFICATIONS:** NEVER rewrite, create, or delete any file (code, documentation, configuration, memory, or otherwise) without the user's explicit permission. If the user does not specifically request a code modification (e.g., using phrases like "we are just thinking," "what do you think," "let's plan"), you MUST ONLY provide text-based analysis and planning. You are FORBIDDEN from using file-modifying tools (`replace_string_in_file`, `edit_file`, `create_file`, etc.) until the user explicitly says "ok", "go ahead", "implement", or a similar unambiguous approval. +## Session Setup + +**Mandatory reads at session start** — in addition to this `copilot-instructions.md`, the agent MUST load the three workspace skills' `SKILL.md` files: + +- `docs-discovery/SKILL.md` — **reactive** (triggers on any domain question — must be ready BEFORE the first domain query arrives) +- `docs-check/SKILL.md` — **reactive** (triggers at the end of every code-modifying response) +- `protocol-audit/SKILL.md` — **on-demand** (triggers on explicit "audit protocol" command) + +**Path resolution**: if this repo is the canonical protocol host (see `@repo` block below — typically AyCode.Core), the paths are local: `.github/skills//SKILL.md`. Otherwise, prefix with this repo's `own-dep-repos` AyCode.Core path (see the `## Shared Agent Skills` section below for explicit paths). + +**Why mandatory**: workspace skills are NOT in Claude Code's native skill-registry / system-reminder. Without pre-loading their `SKILL.md` content, the agent cannot reliably recognize implicit triggers (e.g., "domain question → invoke docs-discovery" at the moment the question arrives, not retroactively). Pre-loading is a **one-time cost** per session (~10-13K tokens); Rule #3 (no-re-read) prevents repeated reads; re-read only if Rule #4 (Context Recovery) fires. + +**Amortization — critical, do NOT re-evaluate per-turn**: the Session Setup cost is measured over the ENTIRE session, not per single turn. A typical session has many turns; the first domain question alone already recoups the investment (alternative — repeated source-code `Grep`/`Read` per turn — costs 10-20K tokens *per turn* with lower output quality). Do NOT flag pre-loaded content as "wasteful" for turns that don't invoke it — the design depends on cross-turn amortization + Rule #3 (no-re-read) + on-demand specificity of Rule #4 (Context Recovery). This amortization is the **central token-economics principle** of the entire protocol stack. + +The first response's `[LOADED_DOCS: ...]` prefix must reflect **4 files** (this `copilot-instructions.md` + 3 SKILL.md). + ## Documentation-first coding Before running any source-code `Grep` / `get_file` / `code_search` in response to a domain-related user request, invoke the **`docs-discovery`** skill (`.github/skills/docs-discovery/SKILL.md`). It scans `docs/` folders via Glob using topic tokens extracted from the request, loads paired main/`_ISSUES`/`_TODO` .md sets as a unit, and honours the **no-re-read** rule. This saves tokens and prevents reintroducing fixed bugs. The skill contains the full procedure; this rule is just the trigger. @@ -85,7 +109,9 @@ Before running any source-code `Grep` / `get_file` / `code_search` in response t Framework design = **"write the base first, derive the specific later"**. Plan framework placement first, then consumer-specific code. -Full doctrine: `docs/ARCHITECTURE.md#framework-vs-consumer-boundary` +**Exception for workspace meta-tooling:** Files under `.github/` (LLM-agent skills, `copilot-instructions.md`, `LLM_PROTOCOL_DECISIONS.md`, registry data like `REPOS.md` / `TOPIC_CODES.md`) are **workspace-configuration, not framework code**. They may reference any layer — topics, repos, or consumer products — because they describe *this specific workspace's* agent-tooling layout. The Framework-First rule applies to C# code, namespaces, types, and domain `.md` docs (under `docs/`), **NOT** to `.github/` tooling. If AyCode.Core is ever extracted as a standalone framework, the `.github/` folder is workspace-swapped; the framework code itself stays layer-clean. Do NOT "fix" apparent Framework-First violations inside `.github/` tooling — they are intentional. + +Full doctrine: `../docs/ARCHITECTURE.md#framework-vs-consumer-boundary` ## Key Abstractions 1. **IId** is the foundation interface — almost every entity implements it. Always preserve ID integrity. @@ -99,7 +125,7 @@ Full doctrine: `docs/ARCHITECTURE.md#framework-vs-consumer-boundary` ## SignalR 7. **Tag-based transport (no conventional hub methods)** — SignalR communication should generally use the generic methods provided by `AcWebSignalRHubBase` (server) and `AcSignalRClientBase` (client). Request types are conventionally identified by `int` tags. Try to avoid adding custom, business-specific, or conventional string-based Hub methods (e.g., `GetUsers()`). -8. **AcSignalRDataSource** — generic `IList` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. Transport docs: `AyCode.Services/docs/SIGNALR.md`. +8. **AcSignalRDataSource** — generic `IList` with change tracking, CRUD via `SignalRCrudTags`, binary merge, rollback. See `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md`. Transport docs: `AyCode.Services/docs/SIGNALR/README.md`. 9. **JSON-in-Binary tech debt** — client→server request parameters are currently JSON inside a Binary envelope (`SignalPostJsonDataMessage`). Do NOT attempt to fix as a side effect — requires coordinated changes across all consuming projects. ## Critical Warnings @@ -113,10 +139,7 @@ Full doctrine: `docs/ARCHITECTURE.md#framework-vs-consumer-boundary` 15. Session pattern for reads, Transaction pattern for writes. 16. **Target framework is net9.0** (set in AyCode.Core.targets). The SourceGenerator targets netstandard2.0. Consuming projects (AyCode.Blazor, FruitBankHybridApp UI) may target net10.0 but reference AyCode.Core DLLs built as net9.0. 17. **No redundant code** — before writing new logic, check whether similar methods already exist in the current context. Reuse or extract shared logic into smaller methods rather than duplicating. If an existing method does most of what you need, split it into composable parts. -18. **Keep all .md documentation in sync (PASSIVE DETECTION & ASK FIRST)** — If you notice *any contradiction, error, or missing information* between the code and your currently loaded `.md` files, **briefly notify the user and ask for permission before making changes**. Do NOT automatically update `.md` files or trigger new searches yet. Once the user approves, you may actively search, read, and update the necessary docs in the correct layer. -**Identify missing documentation:** If you notice during your task that a frequently used pattern, underlying logic, or important behavior is missing from the docs, **and adding it would improve future LLM context-efficiency (saving searches/tokens)**, briefly notify the user to get approval. -**Topic-based separation:** When creating or expanding documentation, keep logically distinct features or domains in separate `.md` files (e.g., architecture, data model, billing) and only reference them in the main/index documents. Do not cram everything into a single monolithic file. Keep in mind: these `.md` files are primarily for LLM grounding and context providing. Keep them concise, structured, and focused on rules/patterns rather than human-readable prose. -**ENFORCEMENT:** At the end of EVERY code-modifying response, append a **`[DOCUMENTATION CHECK]`** section. Evaluate ONLY the `.md` files *already in your context*. State in 1-2 sentences if you found discrepancies or missing concepts, and ask if they should be documented. **DO NOT trigger searches or tool calls for this initial check**. -19. **Documentation layering** — write `.md` documentation at the **defining layer** (where the code lives). Higher-layer `.md` files reference the base docs (e.g. `see AyCode.Core/docs/SIGNALR.md`) and document only project-specific overrides or extensions. Never duplicate base-layer descriptions in consumer-level docs. +18. **Keep all .md documentation in sync** — at the end of EVERY code-modifying response, invoke the **`docs-check`** skill (`AyCode.Core/.github/skills/docs-check/SKILL.md`). It evaluates loaded `.md` files for drift vs code, missing topic coverage, csproj-glob registration gaps for new `.md` files, and new issue/TODO candidates — then emits the `[DOCUMENTATION CHECK]` section per its procedure (or `[DOCUMENTATION CHECK] None.` single-line when nothing to report). The skill encapsulates the full calibration (4 prerequisites, 3-item volume cap, no-ID rule, status-update-on-fix clause) and empty-state handling. Passive detection + ASK FIRST; all patches require user approval (Rule #5). Do NOT trigger new searches for this check. +19. **Documentation layering** — write `.md` documentation at the **defining layer** (where the code lives). Higher-layer `.md` files reference the base docs (e.g. `see AyCode.Services/docs/SIGNALR/README.md`) and document only project-specific overrides or extensions. Never duplicate base-layer descriptions in consumer-level docs. 20. **Do not re-read .md files** already in your context window. They only change if you modify them yourself (new content is already in context) or if the developer tells you they changed — in that case re-read them once. 21. **Folder navigation** — start from the root `README.md` for solution-level navigation. When you need to understand a folder's contents or find a type/class, read the `README.md` in that folder first — it indexes the local files and sub-folders. Follow this before grepping or reading source files. diff --git a/.github/skills/docs-check/SKILL.md b/.github/skills/docs-check/SKILL.md new file mode 100644 index 0000000..e24f12c --- /dev/null +++ b/.github/skills/docs-check/SKILL.md @@ -0,0 +1,176 @@ +--- +name: docs-check +description: Evaluate loaded `.md` documentation at the end of every code-modifying response — detect drift vs code, missing topic coverage, csproj-glob registration gaps for new `.md` files, and new issue/TODO candidates. Emit the `[DOCUMENTATION CHECK]` section per the protocol (or `[DOCUMENTATION CHECK] None.` when nothing to report). Use at the end of ANY response that creates, modifies, or deletes source code or documentation files. Does NOT trigger new Read/Grep — operates only on files already in LOADED_DOCS context. Read-only; never modifies files (user approves patches per Rule #5). +compatibility: Designed for Claude Code and GitHub Copilot. Invoked at the end of code-modifying responses. Uses only content already in LOADED_DOCS. +metadata: + author: Fullepi + version: "1.0" +--- + +# docs-check + +Evaluate the documentation state at the end of a code-modifying response and emit the `[DOCUMENTATION CHECK]` section. This skill encapsulates the former Rule #18 sub-rules (keep in sync, identify missing, topic separation, file registration, issue/TODO surfacing, ENFORCEMENT). The rule in each repo's `copilot-instructions.md` now points here; the full procedure lives in this file. + +## Before you start + +**Read-only.** This skill MUST NOT: +- Trigger new `Read` / `Grep` / `get_file` / `code_search` calls on source content — evaluate only files already in `LOADED_DOCS`. +- Modify any file — all proposals surface as draft patches for user approval per the active repo's Rule #5 (`EXPLICIT CONSENT FOR MODIFICATIONS`). + +The one allowed exception: `Glob` / `file_search` / `Test-Path` for **metadata checks** (e.g., locating the nearest `.csproj` in Step 4). These don't read content. + +## Step 1 — Triggering condition + +Invoke this skill **only** at the end of a response that: +- Created, modified, or deleted any source code file (`.cs`, `.csproj`, `.razor`, `.cshtml`, `.sln`, etc.) +- Created, modified, or deleted any `.md` file +- Created or updated any `.github/` configuration file (`copilot-instructions.md`, skill, `LLM_PROTOCOL_DECISIONS.md`, etc.) + +Do **NOT** invoke on pure read/analysis/planning responses — the output would be `None.` and consume tokens unnecessarily. + +## Step 2 — Evaluate loaded `.md` for code/docs drift + +For each `.md` file in `LOADED_DOCS`: +- Compare its stated rules/behaviour against the code you touched this turn. +- Flag **concrete discrepancies** only — "doc says X; code does Y" with quotable evidence. +- One line per mismatch. + +Only use evidence from already-loaded docs and the code you modified this turn. Do NOT scan other sources. + +## Step 3 — Identify missing documentation (main topic docs) + +During this turn, did you work with a pattern, class, or behaviour that: +- Is used frequently in the codebase +- Would likely be re-discovered by a future LLM session (wasting tokens) +- Is NOT captured anywhere in the loaded `.md` files + +If yes, propose adding documentation to the appropriate `TOPIC/README.md`, or creating a new topic folder. One line per missing concept. + +**Related but distinct from Step 5** — Step 3 covers main topic docs (`TOPIC/README.md`); Step 5 covers paired companions (`TOPIC_ISSUES.md` / `TOPIC_TODO.md`). + +## Step 4 — File registration (new `.md` outside csproj glob) + +For any `.md` file **created** (or proposed to be created) during this turn: + +1. Identify the nearest `.csproj` (walk up from the file's directory). +2. Check whether the file is already covered by a glob or explicit entry: + - `` — covers Pattern B `docs/` folder contents + - `` — covers folder-level `README.md` files in code directories + - An explicit `` entry +3. If the file is **NOT** covered → propose the csproj patch (narrowest appropriate glob or explicit include). + +Common cases (file under `docs/`, folder-level `README.md`) are already covered cluster-wide → **zero-cost skip**. Surface only for unusual locations. + +Do NOT apply the patch — output it in the `[DOCUMENTATION CHECK]` section. User approves per Rule #5. + +Files missing from the csproj are invisible in Visual Studio Solution Explorer. + +## Step 5 — Issue / TODO surfacing (PASSIVE DETECTION & ASK FIRST) + +If during code reading or modification you observed a concrete: +- **Issue** — bug, broken contract, inconsistent behaviour, observable edge case +- **Planned work** — refactor, missing feature, optimization + +propose a draft entry. + +### Prerequisites (ALL must hold — else stay silent) + +1. **Companion loaded** — the matching `_ISSUES.md` or `_TODO.md` MUST be in `LOADED_DOCS`. Without it, no reliable duplicate check → **SKIP**. +2. **High confidence** — you can quote the offending code line or spec inconsistency. Speculative "might be" / "could be" / "possibly" → **SKIP**. +3. **Concrete** — a developer could act on it without further clarification. Stylistic hunches ("this could be cleaner", "consider renaming") → **SKIP**. +4. **Not duplicate** — if a similar item already exists in the loaded companion, even if phrased differently → **SKIP**. + +### Draft entry format + +- **Proposed ID**: `{TOPIC}-{TYPE}-{N}` per the workspace registry (`references/TOPIC_CODES.md` relative to this SKILL.md). Types: `I` (issue), `T` (TODO), `B` (bug — confirmed broken), `C` (critical — severity override). `N` = (highest existing seq in the loaded companion file) + 1. Marked `[tentative]` since parallel sessions may collide — the user verifies/adjusts at apply-time. +- **Priority**: `P1` (high) / `P2` (important) / `P3` (nice-to-have). Note: `C` type already implies emergency; priority is still stated for precision. +- **Rationale**: 1-2 sentences. What's wrong or what's planned. +- **Affected**: file/class/method reference (with line number if known) +- **Sub-category** (optional): if the entry relates to a known sub-area (e.g., SignalR protocol, logger DI factory), state in body header: `## SIG-I-4 [PROTO]: ...`. Do NOT encode sub-category in the ID itself — ID stays flat. + +**Topic code selection**: +- Pick from the registry (`references/TOPIC_CODES.md`) — the registry is the authoritative list of topic codes and type codes; this skill does not hardcode them. +- If the observation spans ≥2 topics → use the `XCUT` topic code (see the XCUT row in `references/TOPIC_CODES.md` for its "Docs location" — the canonical home of cross-cutting entries), with short cross-refs from each affected topic's file. +- If a new topic is needed → surface a proposal ("suggest new topic code `NEWTOPIC` — reason: ...") in the output, do NOT auto-assign. User adds via Decision Log + registry update. + +### Volume cap + +**Maximum 3 surfaced items per response.** If more candidates exist, triage for highest-confidence + highest-impact. Noise cost > marginal insight. + +### Negative examples (do NOT surface) + +- "This variable could have a better name" — stylistic, skip. +- "This might have a race condition if ..." — speculative, skip. +- "Consider refactoring the whole class to use a strategy pattern" — vague, not actionable without clarification. + +## Step 6 — Status update on fix + +If code changes in this response **fix** a concrete issue that IS listed in a loaded `_ISSUES.md`: +- Surface a suggestion to mark that issue `Status: FIXED` (with commit ref if applicable). +- Same prerequisites + cap apply. +- Counts against the 3-item cap in Step 5. + +Example surface: `LOGGING_ISSUES.md#log-i-1 → mark Status: FIXED (NopLogWriter ctor signature fixed in this response)`. + +## Step 7 — Emit the `[DOCUMENTATION CHECK]` section + +Collect findings from Steps 2-6. Emit at the end of the response (after the main answer content): + +### If any findings exist + +``` +[DOCUMENTATION CHECK] + +**Drift / missing concepts** (if any): +- [concrete statement of what doc says vs what code does] + +**New .md files needing csproj registration** (if any): +- `` → needs `` + +**Issue / TODO draft entries** (capped at 3): +- `[tentative: TOPIC-TYPE-N]` [P1/P2/P3] [rationale] — [affected file:line] + +**Status: FIXED suggestions** (if any): +- `#` → mark FIXED (fix in ) + +Apply which, if any? +``` + +Omit sub-sections that have no content for this turn — keep the output minimal. + +### If NO findings + +Emit the one-liner: + +``` +[DOCUMENTATION CHECK] None. +``` + +No prose padding. No "I checked X, Y, Z and everything looked fine" — just the line. + +## Edge cases + +- **No `.md` files in LOADED_DOCS** — Steps 2 and 5 fall through (no context to evaluate). Step 4 still runs if any `.md` was created. Output usually `None.`. +- **Companion `_ISSUES.md` / `_TODO.md` not loaded** — Step 5 SKIPS surfacing for that topic. Do NOT load it during docs-check (this skill is read-only on loaded context). +- **Massive code modification** — if the response refactored 20+ files, stay focused: Step 5's 3-item cap applies strictly. Report the most important; note in output if triage was needed (e.g., "12 additional minor observations suppressed per volume cap"). +- **Response created 5 new `.md` files** — Step 4 reports all registration gaps (not capped, unlike Step 5). +- **Skill invoked on pure-analysis response** — Step 1 says: do NOT invoke. If already triggered, output `None.` and move on. +- **Ambiguous duplicate match** — if unsure whether an observation is already covered by an existing entry, default to SKIP. Silence beats noise. + +## Do NOT + +- **Trigger new `Read` / `Grep` / `code_search` on source content** — this is a review skill, not a search skill. +- **Apply any patches** — surface as proposals. User approves per Rule #5. +- **Generate verbose prose** — output is telegraphic: bullet lists, one-line-per-item. +- **Fire on every response** — only at the end of code-modifying responses (Step 1). +- **Surface low-confidence hunches** — prerequisites in Step 5 are strict. +- **Assign IDs to draft issue/TODO entries** — the user assigns at apply-time. + +## Tool usage + +This skill is tool-neutral and mostly read-only: +- Reading `LOADED_DOCS` list — implicit (conversation context) +- Verifying `.csproj` path — `Glob` / `file_search` / `Test-Path` (metadata only, no content) +- Output generation — normal response text + +Explicitly forbidden: `get_file` / `Read` for content inspection during this skill, `Grep` / `code_search` for patterns during this skill. diff --git a/.github/skills/docs-check/references/TOPIC_CODES.md b/.github/skills/docs-check/references/TOPIC_CODES.md new file mode 100644 index 0000000..de44fce --- /dev/null +++ b/.github/skills/docs-check/references/TOPIC_CODES.md @@ -0,0 +1,90 @@ +# Topic Codes — registry for globally-unique issue/TODO/decision IDs + +> **Scope note — workspace meta-tooling:** This registry is **workspace-configuration, not framework code**. It lists topics across any layer (framework, shared libraries, consumer products) because LLM agents operate across the full workspace and need one unified topic list. This is an intentional exception to the Framework-First Design Principle — see `AyCode.Core/.github/copilot-instructions.md` → Framework-First Design Principle → Exception for workspace meta-tooling. If AyCode.Core is extracted as a standalone framework, this registry is replaced with the new workspace's topic set (typically only LOG, BIN, TOON, XCUT, LLMP — the framework-layer topics). + +Canonical registry of topic codes used in issue, TODO, bug, critical, and decision entry IDs across the workspace. ID format: `{TOPIC}-{TYPE}-{N}`. + +## Why this registry exists + +To make IDs like `LOG-I-5`, `SIG-B-2`, `XCUT-I-1` **globally unique strings** — unambiguous when referenced from code comments, cross-file, commit messages, or a future DB migration. + +## Topic codes + +| Code | Topic | Scope | Docs location (primary) | +|---------|-----------------------------|-----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------| +| `LOG` | LOGGING | Logger system: levels, writers, config-reading vs DI factory | `AyCode.Core/AyCode.Core/docs/LOGGING/` (+ variants in `AyCode.Core.Server`, `AyCode.Services`, `Mango.Nop.Services`) | +| `SIG` | SIGNALR | SignalR transport: tags, client base, dispatch, session | `AyCode.Core/AyCode.Services/docs/SIGNALR/` (+ variant in `AyCode.Services.Server`) | +| `SBP` | SIGNALR_BINARY_PROTOCOL | Binary wire protocol over SignalR: framing, chunking, argument read | `AyCode.Core/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/` | +| `BIN` | BINARY | AcBinary serializer: features, format, writers, source generator | `AyCode.Core/AyCode.Core/docs/BINARY/` | +| `TOON` | TOON | Toon serializer: LLM-optimized format with @meta/@types/@data sections | `AyCode.Core/AyCode.Core/docs/TOON/` | +| `GRID` | MGGRID (grid component) | MGGRID component family: layout, CRUD, columns, toolbar, rendering | `AyCode.Blazor/AyCode.Blazor.Components/docs/MGGRID/` | +| `XCUT` | cross-cutting | Issues / TODOs that span ≥2 topics — one canonical home, referenced from others | `AyCode.Core/AyCode.Core/docs/XCUT/` *(canonical; entry can be cross-ref'd from each affected topic file)* | +| `LLMP` | LLM-protocol meta | LLM protocol decisions (Decision Log entries only — uses `LLMP-DEC-N` form) | `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md` | + +## Type codes + +| Code | Type | Used in file | Notes | +|-------|----------------|----------------------------------------------|----------------------------------------------------------------------------------------------------| +| `I` | Issue | `{TOPIC}_ISSUES.md` | Concrete concern: spec inconsistency, broken contract, observable edge case | +| `T` | TODO | `{TOPIC}_TODO.md` | Forward-looking planned work: refactor, missing feature, optimization | +| `B` | Bug | `{TOPIC}_ISSUES.md` (alongside `I` entries) | Confirmed broken behaviour, reproducible, needs a code fix | +| `C` | Critical | `{TOPIC}_ISSUES.md` or `{TOPIC}_TODO.md` | **Severity override** — emergency priority, supersedes `I`/`B`/`T` category; body explains type | +| `DEC` | Decision | `LLM_PROTOCOL_DECISIONS.md` | **LLMP-only.** Append-only protocol decision entries. | + +### Distinctions + +- **I vs B**: Both tracked together in `_ISSUES.md`. Use `B` only when the behaviour is confirmed broken with a reproducer. `I` covers concerns, inconsistencies, doc drift, edge cases without an active bug. +- **C (Critical)**: A severity flag, not a category. `LOG-C-1` means "logger critical item 1" — body must state whether it's an underlying bug / issue / todo. Prefer `C` over `I`/`B`/`T` when severity is emergency. Do NOT double-classify (no `LOG-IB-1` or similar). +- **DEC**: LLMP exception — long form because "LLMP-D-1" is unreadable. Decision Log entries only. + +## ID format rules + +1. **Format**: `{TOPIC}-{TYPE}-{N}` — all uppercase, hyphen-separated. +2. **Sequential number**: starts at 1, no zero-padding (`LOG-I-1`, `LOG-I-10`, `LOG-I-123`). +3. **Counter scope**: per (topic, type) pair. `LOG-I-*` and `LOG-T-*` have independent counters. `LOG-B-*` has its own counter (separate from `LOG-I-*`). +4. **Append-only**: once assigned, IDs never change. If an entry is reversed or superseded, add a NEW entry that references the prior one — do not renumber. +5. **Hash anchor** (markdown cross-file refs): lowercase with hyphens preserved (`LOGGING_ISSUES.md#log-i-5` — GitHub auto-converts). +6. **No sub-category in ID**: legacy prefixes like `PROTO-`, `DISPATCH-`, `CONN-`, `DS-` are NOT allowed at ID level. Capture sub-category in the entry body header: `## SIG-I-4 [PROTO]: ...`. + +## Registry maintenance + +To add a new topic code: +1. Propose the code (2-5 uppercase chars), short and mnemonic. +2. Check it doesn't collide with C# class-name prefixes (`Ac*` = AyCode.Core, `Mg*` = Mango-specific) — the code should be visually distinct from those prefixes. +3. Add a row to this registry. +4. Create the topic folder: `docs/{TOPIC_FOLDER_NAME}/` with `README.md`, optional `{TOPIC_FOLDER_NAME}_ISSUES.md`, `{TOPIC_FOLDER_NAME}_TODO.md`. +5. Add a Decision Log entry (`LLMP-DEC-N`) recording the new topic. + +## Collision avoidance with class-name prefixes + +C# code conventions in this workspace: +- `Ac*` — AyCode.Core framework types (e.g., `AcLoggerBase`, `AcBinarySerializer`) +- `Mg*` — Mango company types (e.g., `MgGrid`, `MgDbTableBase`, `MgEntityBase`) + +Topic codes intentionally avoid these 2-char prefixes to prevent visual confusion in mixed content (e.g., `MgGrid.cs → GRID-I-1`, not `MG-I-1`). + +## Examples + +``` +LOG-I-5 # logger issue 5 +LOG-T-3 # logger TODO 3 +LOG-B-1 # logger bug 1 (confirmed broken) +LOG-C-1 # logger CRITICAL — body: underlying bug / issue / todo +SIG-I-4 # SignalR issue 4 (body may note: [PROTO] sub-category) +SBP-T-2 # SignalR Binary Protocol TODO 2 +BIN-B-1 # Binary serializer bug 1 +TOON-I-3 # Toon issue 3 +GRID-T-7 # MgGrid TODO 7 +XCUT-I-1 # cross-cutting issue 1 (affects ≥2 topics) +LLMP-DEC-4 # LLM-protocol decision 4 (Decision Log entry) +``` + +## Cross-references to other files + +- **Reference format** (cross-file in markdown): `LOGGING_ISSUES.md#log-i-5` (filename + hash anchor). Bare ID (`LOG-I-5`) may be used when context is unambiguous (within the same topic). +- **Code comments**: `// See LOG-I-5` — bare ID acceptable since it's globally unique. +- **DB natural key** (future migration): `(topic, type, seq)` tuple; or the full string `LOG-I-5` as a single column. + +## Change history + +See the Decision Log (`../../../LLM_PROTOCOL_DECISIONS.md`) for the introduction of this registry and future topic-code additions. diff --git a/.github/skills/docs-discovery/SKILL.md b/.github/skills/docs-discovery/SKILL.md index a41f261..e344150 100644 --- a/.github/skills/docs-discovery/SKILL.md +++ b/.github/skills/docs-discovery/SKILL.md @@ -19,7 +19,7 @@ This skill READS `.md` files and updates the LLM's `[LOADED_DOCS: ...]` state. I Parse the user's most recent message (and the wider conversation tail if relevant) for concrete concepts. Examples: -- Class / type names: `AcLoggerBase`, `SegmentBufferReader`, `AcBinaryHubProtocol`, `FruitBankSignalRClient` +- Class / type names: `AcLoggerBase`, `SegmentBufferReader`, `AcBinaryHubProtocol`, `SignalRClient` (any derived/consumer-specific type) - Feature areas: "logger", "log writer", "serializer", "SignalR", "hub protocol", "chunked framing", "connection builder", "options" - File hints: `Program.cs`, `AcLoggerBase.cs`, `SIGNALR.md` - Patterns / idioms: "DI factory", "appsettings", "mode negotiation" @@ -35,21 +35,51 @@ Keep the set small (usually 1-3 root tokens). If the request genuinely spans mul ## Step 2 — Map tokens to glob patterns (semantic, not hardcoded) -For each root token, synthesize `.md` filename patterns using common conventions: +### ⚠️ CRITICAL — the recursive `**/` wildcard is MANDATORY in every glob -| Token example | Glob patterns to try | -|---|---| -| `logger`, `log`, `logging` | `**/docs/LOGGING*.md` | -| `binary`, `serializer` | `**/docs/BINARY*.md` | -| `signalr`, `hub` | `**/docs/SIGNALR*.md` | -| `protocol`, `wire`, `chunked` | `**/docs/*PROTOCOL*.md` | -| `grid`, `mggrid` | `**/docs/MGGRID*.md` | +The `**/` is NOT cosmetic. It matches `docs/` at **any depth** in the workspace: +- repo-root: `/docs/TOPIC/` +- project-level: `//docs/TOPIC/` ← **very common for Pattern B layouts** +- nested: `///docs/TOPIC/` -Do NOT require the tokens to match a pre-baked list — construct patterns from the token itself uppercased (e.g., `logger` → `**/docs/LOGGER*.md` + `**/docs/LOGGING*.md`). Natural language variants (logger/logging, serialize/serializer, binary/binaries) should all be attempted. +**Correct form — always**: `/**/docs/{TOKEN}/**/*.md` +**Wrong form — never**: `/docs/{TOKEN}/**/*.md` (missing the leading `**/`) -Also consider suffix patterns: -- `**/docs/*{TOKEN}*.md` (substring match) -- `**/docs/*{TOKEN}_ISSUES.md`, `**/docs/*{TOKEN}_TODO.md` (paired docs) +**Failure mode** (this happens often with Pattern B projects): +- You know the target repo (e.g. via `own-dep-repos`) — say ` = AyCode.Core`. +- You synthesize `/docs/{TOKEN}/...` because "that's where docs usually live". +- Glob returns 0 matches (repo-root `docs/` doesn't contain topic folders — only flat reference docs). +- You conclude "no docs exist" and fall through to code-search. +- Meanwhile the actual docs sit at `//docs/{TOKEN}/` — one level deeper. + +**The rule is absolute**: NEVER drop the leading `**/`, even when you "know" the repo. Let the recursive glob find the actual depth. Relative-path guesses based on "usual" layouts are a reliable source of false-empty conclusions. + +### File layout convention + +(See `LLM_PROTOCOL_DECISIONS.md` entry "Docs migrated to folder+README pattern".) + +Topics with multiple files live in named folders: `docs/TOPIC/README.md` + `docs/TOPIC/TOPIC_ISSUES.md` + `docs/TOPIC/TOPIC_TODO.md` (or other `TOPIC_*.md` companions). Single-file reference docs remain flat at the `docs/` root (e.g., `docs/ARCHITECTURE.md`, `docs/GLOSSARY.md`). + +For each root token, synthesize glob patterns targeting BOTH layouts: + +| Token example | Primary glob (folder) | Companion glob (flat + variants) | +|---|---|---| +| `logger`, `log`, `logging` | `**/docs/LOGGING/**/*.md` | `**/docs/LOGGING_*.md` (legacy/variants) | +| `binary`, `serializer` | `**/docs/BINARY/**/*.md` | `**/docs/BINARY_*.md` | +| `signalr`, `hub` | `**/docs/SIGNALR*/**/*.md` | — (covers SIGNALR + SIGNALR_BINARY_PROTOCOL folders) | +| `protocol`, `wire`, `chunked` | `**/docs/*PROTOCOL*/**/*.md` | — | +| `grid`, `mggrid` | `**/docs/MGGRID/**/*.md` | — | +| `architecture`, `conventions`, `glossary` | — (flat, single-file) | `**/docs/ARCHITECTURE.md`, `**/docs/CONVENTIONS.md`, `**/docs/GLOSSARY.md` | + +Do NOT require tokens to match a pre-baked list — construct patterns from the token itself uppercased: +- Primary: `**/docs/{TOKEN}/**/*.md` (matches everything inside the topic folder) +- Companion/variant: `**/docs/{TOKEN}_*.md` (matches flat files or variant prefix folders like `SIGNALR_BINARY_PROTOCOL`) + +Natural language variants (logger/logging, serialize/serializer, binary/binaries) should all be attempted against both the primary and companion patterns. + +**For README.md discovery** (folder-navigation rule): if a topic folder match is found, the `README.md` in that folder is the entry point and MUST be included in the load set (not just sibling `_ISSUES` / `_TODO` files). + +(See the CRITICAL section at the top of this Step 2 for the full explanation of why the leading `**/` is mandatory — this is the most common cause of false-empty docs conclusions.) ## Step 3 — Execute the Glob and dedupe against already-loaded docs @@ -61,9 +91,11 @@ Run each glob pattern via the host agent's Glob tool. Collect all matching absol If the total match count exceeds 10, narrow the glob pattern (e.g., require domain token near the filename start, not just substring). LLM context is finite. +**False-empty guardrail:** if the glob returns 0 matches OR all matched files are 0-byte, do NOT conclude "docs are empty" — first re-validate the glob (typo? literal path substituted?) and retry once with the same token under a corrected `**/docs/...` pattern (NEVER with an ad-hoc path guess). Only after the validated retry also fails should you fall through to code-search. + ## Step 4 — Load the filtered set -Read all remaining matches in parallel (batch the Read calls in one tool-use block). The newly-loaded basenames will appear in your next response's `[LOADED_DOCS: ...]` prefix under the `+K this turn: ` delta, per the active repo's Rule #1 format. +Read all remaining matches in parallel (batch the Read calls in one tool-use block). The newly-loaded files will appear in your next response's `[LOADED_DOCS: ...]` prefix under the `+K this turn: ` delta, per the active repo's Rule #1 format (basename by default; `TOPIC/README.md` for topic-folder READMEs to disambiguate across the many `README.md` files the Pattern-B docs layout introduces). ## Step 5 — Respect the paired-docs convention diff --git a/.github/skills/protocol-audit/SKILL.md b/.github/skills/protocol-audit/SKILL.md index ae8c18a..985069f 100644 --- a/.github/skills/protocol-audit/SKILL.md +++ b/.github/skills/protocol-audit/SKILL.md @@ -1,15 +1,15 @@ --- name: protocol-audit -description: Audit the 8 AyCode/Mango `.github/copilot-instructions.md` files for protocol consistency. Two file types are recognized — **primary** (5 files with full numbered AI AGENT CORE PROTOCOL: AyCode.Core, AyCode.Blazor, Libraries, FruitBank, FruitBankHybridApp) and **inherit** (3 files that reference AyCode.Core's protocol: Mango.Nop.Core, Nop.Plugin.Misc.AIPlugin, Mango.FruitBank). The skill applies the appropriate invariant set per type. Use when the user asks to "audit protocol", "check instruction consistency", "verify repo rules", "check cross-repo drift", or after modifying the AI AGENT CORE PROTOCOL in any repo. Produces a per-file × invariant table with concrete patch suggestions; does NOT modify any file without explicit consent. -compatibility: Designed for Claude Code and GitHub Copilot (VS). Requires read access to the 8 paths listed in `references/REPOS.md`. +description: Audit all `.github/copilot-instructions.md` files registered in `references/REPOS.md` for protocol consistency. Two file types are recognized per that registry — **primary** (full numbered AI AGENT CORE PROTOCOL) and **inherit** (reference-only: blockquote pointer to the canonical protocol, no duplicated numbered rules). The skill applies the appropriate invariant set per type. Use when the user asks to "audit protocol", "check instruction consistency", "verify repo rules", "check cross-repo drift", or after modifying the AI AGENT CORE PROTOCOL in any repo. Produces a per-file × invariant table with concrete patch suggestions; does NOT modify any file without explicit consent. +compatibility: Designed for Claude Code and GitHub Copilot (VS). Requires read access to the paths listed in `references/REPOS.md`. metadata: author: Fullepi - version: "2.0" + version: "2.2" --- # Protocol Audit -Verify that all 8 known `.github/copilot-instructions.md` files share a consistent AI AGENT CORE PROTOCOL ecosystem. **Primary** files contain the full numbered protocol; **inherit** files reference AyCode.Core's protocol without duplicating the numbered rules. +Verify that all `.github/copilot-instructions.md` files registered in `references/REPOS.md` share a consistent AI AGENT CORE PROTOCOL ecosystem. **Primary** files contain the full numbered protocol; **inherit** files reference the canonical protocol without duplicating the numbered rules. ## Before you start @@ -18,11 +18,13 @@ This skill READS files and REPORTS findings. It MUST NOT modify any file. Patch ## Step 1 — Load the repo list Read `references/REPOS.md` (relative to this SKILL.md). Extract: -- **Primary files table** (rows 1-5) with absolute paths -- **Inherit files table** (rows 6-8) with absolute paths +- **Primary files table** (with absolute paths and classification) +- **Inherit files table** (with absolute paths and classification) - **Expected own-dep-repos** tables (one per file type) - **Known issues** section (pre-flagged expected failures) +The registry file is the **single source of truth** for "which files belong to this workspace's protocol audit". The skill treats its contents as authoritative — it does not hardcode any repo/project names. + ## Step 2 — Read each instruction file For each entry in both tables, read `\.github\copilot-instructions.md` once. Record the full text keyed by the logical name and **file type** (primary / inherit). If a file is empty (0 bytes), record as `EMPTY` and still run the size-sensitive invariants (expected: all fail). @@ -44,7 +46,10 @@ For each `": "` entry, resolve `/` present, where `` is the repo designated as the canonical host in REPOS.md (the file content's literal phrasing must match the host's name — this workspace's files currently say "AyCode.Core"). If REPOS.md designates a different host, both the inherit files and this invariant's expected substring are updated in lockstep. **I2. Does NOT duplicate numbered Rules #1-5** Must NOT contain the header `🛑 AI AGENT CORE PROTOCOL (CRITICAL ENFORCEMENT)` (that belongs to primary files only). If the file has `^\d+\. \*\*MANDATORY OUTPUT PREFIX` or similar, flag as FAIL — the inherit file has leaked primary content. @@ -86,24 +91,30 @@ Must NOT contain the header `🛑 AI AGENT CORE PROTOCOL (CRITICAL ENFORCEMENT)` **I3. Has a link to the Decision Log** Substring `LLM_PROTOCOL_DECISIONS.md` present (via the Protocol History section — see X2 below). -### 3D — Cross-cutting invariants (applied to 4 non-Core primary + 3 inherit = 7 files) +### 3D — Cross-cutting invariants (applied to all files EXCEPT the canonical protocol host) -These invariants apply to files OTHER than AyCode.Core itself — AyCode.Core does not need to reference itself. +The **canonical protocol host** is the repo designated in `references/REPOS.md` as housing the shared skills, Decision Log, and registries (typically the first row labeled the "host" in REPOS.md). That host does not need to reference itself. These X invariants apply to every other file registered in REPOS.md. -**X1. `## Shared Agent Skills` section present** -Header `## Shared Agent Skills` must appear. Both `docs-discovery` and `protocol-audit` bullets must be listed under it. +**X1. `## Shared Agent Skills` section present with all three skills** +Header `## Shared Agent Skills` must appear. All three bullets must be listed: `docs-discovery`, `protocol-audit`, `docs-check`. **X2. `## Protocol History` section present** -Header `## Protocol History` must appear AND it must reference `AyCode.Core/.github/LLM_PROTOCOL_DECISIONS.md`. +Header `## Protocol History` must appear AND it must reference the Decision Log at the canonical host's location (e.g., `/.github/LLM_PROTOCOL_DECISIONS.md`; the concrete path is resolvable from REPOS.md). + +**X3. Docs-sync rule points to `docs-check` skill** *(primary files only)* +In each primary file's docs-sync rule (the numbered rule whose title begins "Keep all .md documentation in sync"), the substring `` `docs-check` `` (backtick-wrapped skill name) must be present, AND a reference to the docs-check skill's `SKILL.md` path (e.g., `.github/skills/docs-check/SKILL.md` or a correct relative variant) must be present. ### Invariant applicability matrix -| Invariant set | AyCode.Core | 4× non-Core primary | 3× inherit | -|---------------|-------------|---------------------|------------| -| Common (C1-C3) | ✓ | ✓ | ✓ | +| Invariant set | Canonical host | Other primary | Inherit | +|---------------|----------------|---------------|---------| +| Common (C1-C4) | ✓ | ✓ | ✓ | | Primary-only (P1-P10) | ✓ | ✓ | N/A (skip) | | Inherit-only (I1-I3) | N/A (skip) | N/A (skip) | ✓ | -| Cross-cutting (X1-X2) | N/A (skip) | ✓ | ✓ | +| Cross-cutting (X1-X2) | N/A (skip — the host does not cross-reference itself) | ✓ | ✓ | +| Cross-cutting (X3) | ✓ | ✓ | N/A (skip — inherit files don't have the numbered docs-sync rule) | + +The primary/inherit classification and the "canonical host" designation both come from `references/REPOS.md`. This skill does not hardcode any specific repo or project name. Use `N/A` in the report cell, not `PASS`, for skipped invariants — so it's obvious the check wasn't applicable. @@ -153,5 +164,5 @@ This skill is tool-neutral. Map these capabilities to the host agent's tools (pe - **Multiple `@repo` blocks in one file:** Audit the first one; flag the duplicate as its own finding. - **Rule order differs** (e.g., Rules #3 and #4 swapped in a primary file): invariants P6 and P8 fail independently — do not try to auto-reorder. - **Unfinished mid-edit:** If a file has obviously truncated content (cut off mid-sentence), record `CORRUPT` and stop that file's audit. -- **Ambiguous file type** (has both `AI AGENT CORE PROTOCOL` header AND `follows the AI Agent Core Protocol defined in AyCode.Core` blockquote): flag as FAIL on I2 and P1 — file has structural identity crisis, user must resolve. +- **Ambiguous file type** (has both `AI AGENT CORE PROTOCOL` header AND the inherit-blockquote substring from I1): flag as FAIL on I2 and P1 — file has structural identity crisis, user must resolve. - **New file not in REPOS.md:** Do NOT audit it automatically. Report it separately as `"Unregistered .github/copilot-instructions.md found at — add to REPOS.md to include in future audits."` diff --git a/.github/skills/protocol-audit/references/REPOS.md b/.github/skills/protocol-audit/references/REPOS.md index 1ba18aa..bbbe96a 100644 --- a/.github/skills/protocol-audit/references/REPOS.md +++ b/.github/skills/protocol-audit/references/REPOS.md @@ -1,18 +1,26 @@ # Repos under protocol-audit -Each row is one `.github/copilot-instructions.md` file that participates in the shared AI AGENT CORE PROTOCOL — either as **Primary** (full numbered-rule protocol) or **Inherit** (references AyCode.Core's protocol, minimal own content). +> **Scope note — workspace meta-tooling:** This file is **workspace-configuration, not framework code**. It may list repos at any layer (including consumer products) because protocol-audit is a workspace-level cross-cutting tool. This is an intentional exception to the Framework-First Design Principle — see `AyCode.Core/.github/copilot-instructions.md` → Framework-First Design Principle → Exception for workspace meta-tooling. If AyCode.Core is extracted as a standalone framework, this file is replaced with the new workspace's layout. + +Each row is one `.github/copilot-instructions.md` file that participates in the shared AI AGENT CORE PROTOCOL — either as **Primary** (full numbered-rule protocol) or **Inherit** (references the canonical host's protocol, minimal own content). + +## Canonical protocol host + +**`AyCode.Core`** — this repo hosts the shared agent skills (`.github/skills/`), the Decision Log (`.github/LLM_PROTOCOL_DECISIONS.md`), and these registry files. All inherit files reference AyCode.Core. Cross-cutting invariants (X1–X2) are skipped for the host itself (it does not cross-reference itself). + +If the host designation is ever moved to a different repo, update this section AND the inherit-file substring checked by invariant I1 in `SKILL.md`. ## Primary protocol files -These files contain the full numbered AI AGENT CORE PROTOCOL (Rules #1-5), `@repo` block, and Conventions. All structural, rule-content, and cross-reference invariants apply. +These files contain the full numbered AI AGENT CORE PROTOCOL (Rules #1-5), `@repo` block, and Conventions. All structural, rule-content, and cross-reference invariants apply. The first row (marked `★`) is the canonical protocol host. -| # | Name | Absolute path | Layer | -|---|---------------------|----------------------------------------------------------------------------|------------------| -| 1 | AyCode.Core | `H:\Applications\Aycode\Source\AyCode.Core` | framework (0) | -| 2 | AyCode.Blazor | `H:\Applications\Aycode\Source\AyCode.Blazor` | framework (1) | -| 3 | Libraries | `H:\Applications\Mango\Source\NopCommerce.Common\4.70\Libraries` | shared libs (2) | -| 4 | FruitBank | `H:\Applications\Mango\Source\FruitBank` | product (3) | -| 5 | FruitBankHybridApp | `H:\Applications\Mango\Source\FruitBankHybridApp` | product (3) | +| # | Name | Absolute path | Layer | Host | +|---|---------------------|----------------------------------------------------------------------------|------------------|------| +| 1 | AyCode.Core | `H:\Applications\Aycode\Source\AyCode.Core` | framework (0) | ★ | +| 2 | AyCode.Blazor | `H:\Applications\Aycode\Source\AyCode.Blazor` | framework (1) | | +| 3 | Mango.Nop Libraries | `H:\Applications\Mango\Source\NopCommerce.Common\4.70\Libraries` | shared libs (2) | | +| 4 | FruitBank | `H:\Applications\Mango\Source\FruitBank` | product (3) | | +| 5 | FruitBankHybridApp | `H:\Applications\Mango\Source\FruitBankHybridApp` | product (3) | | ## Inherit protocol files @@ -35,8 +43,8 @@ The instruction file to audit for each is: `\.github\copilot-instructi | AyCode.Core | — (none) | | AyCode.Blazor | `AyCode.Core: ../AyCode.Core` | | Libraries | `AyCode.Core: ../../../../../Aycode/Source/AyCode.Core` | -| FruitBank | `AyCode.Core: ../../../Aycode/Source/AyCode.Core`, `Libraries: ../NopCommerce.Common/4.70/Libraries` | -| FruitBankHybridApp | `AyCode.Core: ../../../Aycode/Source/AyCode.Core`, `AyCode.Blazor: ../../../Aycode/Source/AyCode.Blazor`, `Libraries: ../NopCommerce.Common/4.70/Libraries` | +| FruitBank | `AyCode.Core: ../../../Aycode/Source/AyCode.Core`, `Mango.Nop Libraries: ../NopCommerce.Common/4.70/Libraries` | +| FruitBankHybridApp | `AyCode.Core: ../../../Aycode/Source/AyCode.Core`, `AyCode.Blazor: ../../../Aycode/Source/AyCode.Blazor`, `Mango.Nop Libraries: ../NopCommerce.Common/4.70/Libraries` | ### Inherit diff --git a/AyCode.Core.Server/AyCode.Core.Server.csproj b/AyCode.Core.Server/AyCode.Core.Server.csproj index f5354fc..2bedbdc 100644 --- a/AyCode.Core.Server/AyCode.Core.Server.csproj +++ b/AyCode.Core.Server/AyCode.Core.Server.csproj @@ -17,6 +17,7 @@ + diff --git a/AyCode.Core.Server/Loggers/README.md b/AyCode.Core.Server/Loggers/README.md index 82d2e51..dc2c47a 100644 --- a/AyCode.Core.Server/Loggers/README.md +++ b/AyCode.Core.Server/Loggers/README.md @@ -2,7 +2,7 @@ Server-side singleton logger for static access across the application. -> For full logging architecture see `docs/LOGGING.md`. For core logger and writer abstractions see `AyCode.Core/Loggers/README.md`. +> For full logging architecture see `docs/LOGGING/README.md`. For core logger and writer abstractions see `AyCode.Core/Loggers/README.md`. ## Key Files diff --git a/AyCode.Core.Server/README.md b/AyCode.Core.Server/README.md index 44eea8e..673feee 100644 --- a/AyCode.Core.Server/README.md +++ b/AyCode.Core.Server/README.md @@ -10,7 +10,7 @@ Server-side extension of AyCode.Core. Provides server-specific implementations t | Document | Topic | |---|---| -| `LOGGING_SERVER.md` | GlobalLogger singleton, server-side logging | +| `LOGGING/README.md` | GlobalLogger singleton, server-side logging | ## Folder Structure diff --git a/AyCode.Core.Server/docs/LOGGING_SERVER.md b/AyCode.Core.Server/docs/LOGGING/README.md similarity index 79% rename from AyCode.Core.Server/docs/LOGGING_SERVER.md rename to AyCode.Core.Server/docs/LOGGING/README.md index e4f462a..63bcbdb 100644 --- a/AyCode.Core.Server/docs/LOGGING_SERVER.md +++ b/AyCode.Core.Server/docs/LOGGING/README.md @@ -1,6 +1,6 @@ # Server Logging -Server-side logging extensions. For core framework (base classes, configuration, LogLevel, ILogger bridge) see `AyCode.Core/docs/LOGGING.md`. For remote writers (HTTP, browser, SignalR) see `AyCode.Services/docs/LOGGING_REMOTE.md`. +Server-side logging extensions. For core framework (base classes, configuration, LogLevel, ILogger bridge) see `AyCode.Core/AyCode.Core/docs/LOGGING/README.md`. For remote writers (HTTP, browser, SignalR) see `AyCode.Services/docs/LOGGING/README.md`. ## GlobalLogger diff --git a/AyCode.Core.Server/docs/README.md b/AyCode.Core.Server/docs/README.md new file mode 100644 index 0000000..f0339ce --- /dev/null +++ b/AyCode.Core.Server/docs/README.md @@ -0,0 +1,16 @@ +# AyCode.Core.Server documentation + +Topic documentation for the `AyCode.Core.Server` project (Layer 0, server-side). + +## Topics + +- [`LOGGING/`](LOGGING/README.md) — Server-side logger (variant of AyCode.Core's base logger) + +## Navigation + +Per the AI Agent Core Protocol (folder navigation rule), start from this README when browsing `docs/`. Each topic folder has its own `README.md` with the main content, plus optional `TOPIC_ISSUES.md` (known issues) and `TOPIC_TODO.md` (planned work). + +## See also + +- **Base logger** (framework): `../../AyCode.Core/AyCode.Core/docs/LOGGING/README.md` +- **Remote logger** (AyCode.Services variant): `../../AyCode.Services/docs/LOGGING/README.md` diff --git a/AyCode.Core/AyCode.Core.csproj b/AyCode.Core/AyCode.Core.csproj index 9bc8c91..8c96535 100644 --- a/AyCode.Core/AyCode.Core.csproj +++ b/AyCode.Core/AyCode.Core.csproj @@ -1,4 +1,4 @@ - + net10.0 @@ -27,6 +27,7 @@ + diff --git a/AyCode.Core/Loggers/README.md b/AyCode.Core/Loggers/README.md index 8aea4f9..118f4af 100644 --- a/AyCode.Core/Loggers/README.md +++ b/AyCode.Core/Loggers/README.md @@ -2,7 +2,7 @@ Custom logging framework with multi-writer fan-out and `Microsoft.Extensions.Logging` integration. This directory contains the **core** logger and writer abstractions — the defining layer for the logging system. -> For full architecture, configuration, all writers, and remote logging flow see `docs/LOGGING.md`. +> For full architecture, configuration, all writers, and remote logging flow see `docs/LOGGING/README.md`. ## Architecture @@ -18,7 +18,7 @@ IAcLogWriterBase └─ [AcLogItemWriterBase in AyCode.Entities — structured writers] ``` -**Two-level filtering:** Logger has a global `LogLevel` gate; each writer has its own `LogLevel`. Both must pass for a log entry to be written. See `docs/LOGGING.md` → Design Overview. +**Two-level filtering:** Logger has a global `LogLevel` gate; each writer has its own `LogLevel`. Both must pass for a log entry to be written. See `docs/LOGGING/README.md` → Design Overview. ## Key Files diff --git a/AyCode.Core/Serializers/Binaries/README.md b/AyCode.Core/Serializers/Binaries/README.md index 4d8789d..b7690d7 100644 --- a/AyCode.Core/Serializers/Binaries/README.md +++ b/AyCode.Core/Serializers/Binaries/README.md @@ -2,9 +2,9 @@ High-performance binary serialization/deserialization. Two-phase processing, multiple wire modes, string interning, source generation. Primary goal: **speed**. -> Implementation details (zero virtual dispatch, buffer management): `../../docs/BINARY_IMPLEMENTATION.md` -> Output writers (ArrayBinaryOutput, BufferWriterBinaryOutput, chunk sizing): `../../docs/BINARY_WRITERS.md` -> Source generation (SGen architecture, hybrid model, bridge methods): `../../docs/BINARY_SGEN.md` +> Implementation details (zero virtual dispatch, buffer management): `../../docs/BINARY/BINARY_IMPLEMENTATION.md` +> Output writers (ArrayBinaryOutput, BufferWriterBinaryOutput, chunk sizing): `../../docs/BINARY/BINARY_WRITERS.md` +> Source generation (SGen architecture, hybrid model, bridge methods): `../../docs/BINARY/BINARY_SGEN.md` ## Architecture @@ -24,7 +24,7 @@ Two root paths in `AcBinarySerializer.Serialize`: | **SGen fast** | `UseGeneratedCode` + `GeneratedWriter != null` | 3 checks → `WriteObject` directly | | **Full runtime** | No GeneratedWriter or `UseGeneratedCode=false` | IQueryable → Expression → TryWritePrimitive → WriteValueNonPrimitive → WriteObject | -SGen fast path skips: `is IQueryable`, `IsExpressionType`, `TryWritePrimitive` (GetTypeCode + 15-case switch), `WriteValueNonPrimitive` (4 interface checks). Wire format identical. Details: `../../docs/BINARY_SGEN.md`. +SGen fast path skips: `is IQueryable`, `IsExpressionType`, `TryWritePrimitive` (GetTypeCode + 15-case switch), `WriteValueNonPrimitive` (4 interface checks). Wire format identical. Details: `../../docs/BINARY/BINARY_SGEN.md`. ### Wire Format @@ -43,7 +43,7 @@ SGen fast path skips: `is IQueryable`, `IsExpressionType`, `TryWritePrimitive` ( | 144+ | **Headers** — Metadata, RefHandling, CacheCount | | 192–255 | **Tiny ints** — single-byte -16..47 | -Full spec: `docs/BINARY_FORMAT.md` +Full spec: `docs/BINARY/BINARY_FORMAT.md` ## Key Files @@ -93,7 +93,7 @@ Key wire-format options: `WireMode` (Compact/Fast), `ReferenceHandling` (None/On `ReferenceHandling=None` + `UseStringInterning=None` = no scan pass (single-phase, fastest). -Presets: `Default`, `FastMode`, `ShallowCopy`, `WasmOptimized`. Details: `docs/BINARY_OPTIONS.md`. +Presets: `Default`, `FastMode`, `ShallowCopy`, `WasmOptimized`. Details: `docs/BINARY/BINARY_OPTIONS.md`. ## Dependencies diff --git a/AyCode.Core/Serializers/Toons/AcToonSerializer.DataSection.cs b/AyCode.Core/Serializers/Toons/AcToonSerializer.DataSection.cs index 53dec06..8c61916 100644 --- a/AyCode.Core/Serializers/Toons/AcToonSerializer.DataSection.cs +++ b/AyCode.Core/Serializers/Toons/AcToonSerializer.DataSection.cs @@ -26,6 +26,7 @@ public static partial class AcToonSerializer context.CurrentIndentLevel++; WriteValue(value, type, context, 0); + context.WriteLine(); context.CurrentIndentLevel--; context.WriteLine("}"); diff --git a/AyCode.Core/Serializers/Toons/README.md b/AyCode.Core/Serializers/Toons/README.md index f1b078f..db8290a 100644 --- a/AyCode.Core/Serializers/Toons/README.md +++ b/AyCode.Core/Serializers/Toons/README.md @@ -1,76 +1,19 @@ -# Toons +# Toons (code folder) -Token-Oriented Object Notation (Toon) — an LLM-optimized serialization format with separate schema (@meta) and data (@data) sections. The primary goal is **LLM accuracy**: maximizing the precision and correctness of LLM responses by providing structured, unambiguous context. Designed for human and LLM readability with minimal token usage. +Source code for `AcToonSerializer` — Token-Oriented Object Notation, the LLM-optimized serialization format. -## Architecture +> **Domain documentation:** [`../../docs/TOON/README.md`](../../docs/TOON/README.md). This file is only a local file index. -### Serialization Modes +## Files in this folder -| Mode | Description | -|---|---| -| `Full` | Both @meta (schema) and @data (values) — default | -| `MetaOnly` | Only @meta section (send schema once, reuse across conversation) | -| `DataOnly` | Only @data section (when schema already sent) | +- `AcToonSerializer.cs` — main entry point (public `Serialize` / `SerializeTypeMetadata` / `SerializeMetadata` API). +- `AcToonSerializer.*.cs` — 14 partial class files split topically (MetaWriter, DataSection, TypeDefinitions, Descriptions, Attributes, AttributeExtraction, Placeholders, TopologicalSort, Navigation, ForeignKeys, Validation, ToonSerializationContext, ToonSerializeTypeMetadata). +- `AcToonSerializerOptions.cs` — options POCO, `ToonSerializationMode` enum, preset factories. +- `AcToonContextBase.cs` — shared base for serialization contexts. +- `ToonDescriptionAttribute.cs` — `[ToonDescription]` attribute + `ToonRelationType` enum. +- `ToonTypeRelation.cs` — type relation string constants (`BaseOf`, `DtoOf`, `ModelOf`, etc.). -### Output Sections +## Cross-references -- **@meta** — Type definitions, property descriptions, navigation info, constraints. -- **@types** — Per-type schema with constraints and examples. -- **@data** — Actual object values with optional type hints. - -### Key Features - -- Triple-quote syntax for multi-line strings. -- Topological sorting for complex type relationships. -- Navigation property tracking (foreign keys and relationships). -- Type relation understanding (inheritance, interfaces). - -## Key Files - -### Core -- **`AcToonSerializer.cs`** — Main serializer entry point and orchestration. -- **`AcToonSerializer.ToonSerializationContext.cs`** — Serialization context. -- **`AcToonSerializer.ToonSerializeTypeMetadata.cs`** — Cached type metadata. -- **`AcToonSerializerOptions.cs`** — Configuration and presets. -- **`AcToonContextBase.cs`** — Base context. - -### Output Generation (partial classes of AcToonSerializer) -- **`AcToonSerializer.MetaWriter.cs`** — @meta section generation. -- **`AcToonSerializer.TypeDefinitions.cs`** — @types section generation. -- **`AcToonSerializer.DataSection.cs`** — @data section generation. -- **`AcToonSerializer.Descriptions.cs`** — Property description generation. - -### Type Analysis -- **`AcToonSerializer.TopologicalSort.cs`** — Dependency-aware type ordering. -- **`AcToonSerializer.Navigation.cs`** — Navigation property discovery. -- **`AcToonSerializer.ForeignKeys.cs`** — Foreign key relationship detection. -- **`AcToonSerializer.Validation.cs`** — Output validation. -- **`AcToonSerializer.Placeholders.cs`** — Placeholder value generation. - -### Attributes & Metadata -- **`AcToonSerializer.Attributes.cs`** — Attribute processing. -- **`AcToonSerializer.AttributeExtraction.cs`** — Attribute value extraction. -- **`ToonDescriptionAttribute.cs`** — Per-property description attribute. -- **`AcNavigationPropertyInfo.cs`** — Navigation property metadata. -- **`ToonTypeRelation.cs`** — Type relationship tracking (inheritance, interfaces). - -## Configuration - -| Option | Default | Description | -|---|---|---| -| `Mode` | Full | Full/MetaOnly/DataOnly | -| `UseIndentation` | true | Readability control | -| `UseInlineTypeHints` | false | Type hints in data section | -| `UseInlineComments` | false | Comments in data section | -| `ShowCollectionCount` | true | Collection sizes | -| `UseMultiLineStrings` | true | Triple-quote long strings | -| `UseEnhancedMetadata` | true | Rich property metadata | -| `OmitDefaultValues` | true | Skip null/default values | -| `WriteTypeNames` | true | Root type names in data | - -**Presets:** `Default`, `MetaOnly`, `DataOnly`, `Compact`, `Verbose`. - -## Dependencies - -- Base classes from parent `Serializers/` folder -- Expression utilities from `Expressions/` folder (for queryable serialization) +- Shared serializer infrastructure (metadata cache, reference tracking, `IdentityMap`): [`../README.md`](../README.md). +- Full Toon docs: [`../../docs/TOON/`](../../docs/TOON/README.md). diff --git a/AyCode.Core/docs/BINARY_FEATURES.md b/AyCode.Core/docs/BINARY/BINARY_FEATURES.md similarity index 100% rename from AyCode.Core/docs/BINARY_FEATURES.md rename to AyCode.Core/docs/BINARY/BINARY_FEATURES.md diff --git a/AyCode.Core/docs/BINARY_FORMAT.md b/AyCode.Core/docs/BINARY/BINARY_FORMAT.md similarity index 100% rename from AyCode.Core/docs/BINARY_FORMAT.md rename to AyCode.Core/docs/BINARY/BINARY_FORMAT.md diff --git a/AyCode.Core/docs/BINARY_IMPLEMENTATION.md b/AyCode.Core/docs/BINARY/BINARY_IMPLEMENTATION.md similarity index 76% rename from AyCode.Core/docs/BINARY_IMPLEMENTATION.md rename to AyCode.Core/docs/BINARY/BINARY_IMPLEMENTATION.md index e6d9784..da34b84 100644 --- a/AyCode.Core/docs/BINARY_IMPLEMENTATION.md +++ b/AyCode.Core/docs/BINARY/BINARY_IMPLEMENTATION.md @@ -141,3 +141,18 @@ Two-phase: - `WriteObjectFullMarkerIId` / `WriteObjectFullMarkerAll`: `wrapper.Metadata` cached at entry, reused in ref-handling and non-ref branches - `GetWrapper(type, slot)` is O(1) array index after first call, but `value.GetType()` is a virtual call — avoid repeating it +## Metadata Lifecycle & Cold-Start (planned: BIN-T-3 / BIN-T-4) + +Today `BinarySerializeTypeMetadata` and `BinaryDeserializeTypeMetadata` are built lazily in `GetWrapperSlow` via `GlobalMetadataCache.GetOrAdd(type, MetadataFactory)`. The factory runs reflection property enumeration, attribute scans, and `Expression.Compile` per property — the dominant first-call cost for SGen types (see `BINARY_ISSUES.md#bin-i-10`). + +**Planned evolution** (`BINARY_TODO.md#bin-t-3`): + +- **`GeneratedMetadataRegistry`**: `ModuleInit` registers pre-built metadata per `[AcBinarySerializable]` type alongside the existing `GeneratedWriterRegistry` / `GeneratedReaderRegistry` entries. Generator passes references to its static `s_typeNameHash` / `s_propertyHashes` fields — single source of truth, no duplicate computation, no hot-path indirection (generator keeps using its own static fields). +- **Metadata ctor split**: a second ctor on `BinarySerializeTypeMetadata` / `BinaryDeserializeTypeMetadata` accepts pre-computed values (hashes, `MinWriteSize`, `ComplexPropertyCount`, `IsIId`, `IdAccessorType`, flags). No reflection in this ctor. +- **Lazy `RuntimeInit`**: `TypeMetadataBase` gets `volatile bool _runtimeInitialized` + `internal void RuntimeInit()`. `GetWrapperSlow` calls it only when `wrapper.GeneratedWriter == null || !Options.UseGeneratedCode` — i.e. for runtime-only types and the `UseGeneratedCode=false` edge case. SGen types skip it. Thread-safe by idempotence + `volatile` (no lock). +- **Hybrid safety**: SGen root path (`WriteObjectProperties` → `generatedWriter.WriteProperties`) never touches the SGen type's own property accessors; non-SGen child types come through the `MetadataFactory` path as today. + +**Follow-up** (`BINARY_TODO.md#bin-t-4`): after BIN-T-3 removes reflection + `Expression.Compile` from the cold path, JIT of generated methods becomes dominant — mitigated via `[AggressiveOptimization]`, background `RuntimeHelpers.PrepareMethod`, and/or R2R (consumer publish config). + +Wire format unchanged; `UseGeneratedCode=false` fallback continues to work identically (triggers `RuntimeInit` for SGen types on demand). + diff --git a/AyCode.Core/docs/BINARY_ISSUES.md b/AyCode.Core/docs/BINARY/BINARY_ISSUES.md similarity index 64% rename from AyCode.Core/docs/BINARY_ISSUES.md rename to AyCode.Core/docs/BINARY/BINARY_ISSUES.md index 1f34921..a9f29b4 100644 --- a/AyCode.Core/docs/BINARY_ISSUES.md +++ b/AyCode.Core/docs/BINARY/BINARY_ISSUES.md @@ -2,7 +2,7 @@ ## Deserialization -### DESER-1: Non-array-backed memory — per-segment copy +### BIN-I-1: Non-array-backed memory — per-segment copy **Status:** By design **Affects:** `SequenceBinaryInput` @@ -12,7 +12,7 @@ When `ReadOnlySequence` segments are backed by native memory (not managed **Impact:** Negligible. Non-array-backed `ReadOnlyMemory` is extremely rare (custom `MemoryManager` with native memory, memory-mapped files). All standard .NET pools (`ArrayPool`, `MemoryPool.Shared`, Kestrel pipe) are array-backed. -### DESER-2: Cross-boundary scratch buffer is not pooled across calls +### BIN-I-2: Cross-boundary scratch buffer is not pooled across calls **Status:** Acceptable **Affects:** `SequenceBinaryInput._scratchBuffer` @@ -23,14 +23,14 @@ The scratch buffer is `ArrayPool.Rent`-ed on first cross-boundary read and reuse **Possible optimization:** Store the scratch buffer on the pooled `BinaryDeserializationContext` and reuse across deserializations. Low priority — `ArrayPool` overhead is negligible. -### DESER-3: ReadBytes always copies +### BIN-I-3: ReadBytes always copies **Status:** By design **Affects:** `BinaryDeserializationContext.ReadBytes(int length)` `ReadBytes` allocates a new `byte[]` and copies from the buffer. This is unavoidable because the caller owns the returned array, and the source buffer (pipe segment or serialized data) may be recycled. -### DESER-4: ReadStringUtf8 requires contiguous buffer +### BIN-I-4: ReadStringUtf8 requires contiguous buffer **Status:** By design **Affects:** `BinaryDeserializationContext.ReadStringUtf8(int length)` @@ -41,14 +41,14 @@ The scratch buffer is `ArrayPool.Rent`-ed on first cross-boundary read and reuse ## Serialization -### SER-1: BufferWriterBinaryOutput fallback path allocates per-chunk +### BIN-I-5: BufferWriterBinaryOutput fallback path allocates per-chunk **Status:** Acceptable **Affects:** `BufferWriterBinaryOutput.AcquireChunk` fallback -When `MemoryMarshal.TryGetArray` fails on `IBufferWriter.GetMemory()` (native memory-backed writer), a `byte[]` is rented from `ArrayPool` per chunk and copied to the writer on `Grow`/`Flush`. Same as DESER-1 — non-array-backed writers are extremely rare. +When `MemoryMarshal.TryGetArray` fails on `IBufferWriter.GetMemory()` (native memory-backed writer), a `byte[]` is rented from `ArrayPool` per chunk and copied to the writer on `Grow`/`Flush`. Same as BIN-I-1 — non-array-backed writers are extremely rare. -### SER-2: AsyncPipeWriterOutput uses sync GetResult() for backpressure +### BIN-I-6: AsyncPipeWriterOutput uses sync GetResult() for backpressure **Status:** By design (v1) **Affects:** `AsyncPipeWriterOutput.Grow()` — `_lastFlush.GetAwaiter().GetResult()` @@ -59,75 +59,92 @@ When the previous `PipeWriter.FlushAsync()` hasn't completed by the next `Grow() **Possible optimization:** `AsyncSegment` mode (future) with a custom async `WriteMessageAsync` protocol interface, enabling `await` on flush instead of `GetResult()`. -### SER-3: AsyncPipeWriterOutput fallback path — same as SER-1 +### BIN-I-7: AsyncPipeWriterOutput fallback path — same as BIN-I-5 **Status:** Acceptable **Affects:** `AsyncPipeWriterOutput.AcquireChunk` fallback -Same `TryGetArray` fallback as `BufferWriterBinaryOutput` (SER-1). Kestrel `PipeWriter.GetMemory()` always returns array-backed memory — fallback is for non-standard `PipeWriter` implementations only. +Same `TryGetArray` fallback as `BufferWriterBinaryOutput` (BIN-I-5). Kestrel `PipeWriter.GetMemory()` always returns array-backed memory — fallback is for non-standard `PipeWriter` implementations only. ## Deserialization (PipeReader) -### DESER-5: PipeReaderBinaryInput uses sync ReadAsync().GetResult() +### BIN-I-8: PipeReaderBinaryInput uses sync ReadAsync().GetResult() **Status:** By design (v1) **Affects:** `PipeReaderBinaryInput.Initialize()` and `TryAdvanceSegment()` -Same constraint as SER-2 — `IBinaryInputBase` interface is synchronous. `ReadAsync().GetAwaiter().GetResult()` blocks when waiting for more data from the pipe. Currently not used in production (SignalR delivers complete messages via `TryParseMessage`). Reserved for future direct-pipe deserialization scenarios. +Same constraint as BIN-I-6 — `IBinaryInputBase` interface is synchronous. `ReadAsync().GetAwaiter().GetResult()` blocks when waiting for more data from the pipe. Currently not used in production (SignalR delivers complete messages via `TryParseMessage`). Reserved for future direct-pipe deserialization scenarios. ## Source Generator (SGen) -### SGEN-1: CS8625 warnings for non-nullable reference types +### BIN-I-9: CS8625 warnings for non-nullable reference types **Status:** Known **Affects:** Generated reader code The source generator emits `null` assignments for non-nullable reference type properties during deserialization (before the value is read from the stream). This produces CS8625 warnings. Functionally harmless — the property is always assigned before use. -## Buffer Writer (BWO) +### BIN-I-10: First-run cold-start overhead -### BWO-1: Struct copy semantics +**Status:** Active — mitigation planned (see `BINARY_TODO.md#bin-t-3`, `BINARY_TODO.md#bin-t-4`) +**Affects:** First `Serialize`/`Deserialize` per `[AcBinarySerializable]` type, per process -**Status:** By design -**Affects:** `BufferWriterBinaryOutput` value-type assignment +Cold-start cost chain on first use of an SGen type (before BIN-T-3 lands): -Assigning a `BufferWriterBinaryOutput` value creates an independent copy. State changes (e.g. `_committedBytes` via `Grow`/`Flush`) are not reflected in the original. Copy back after use if needed. +1. `BinarySerializeTypeMetadata` ctor — reflection property enumeration + `GetCustomAttribute` scans +2. `Expression.Compile` per property accessor (dynamic getter + typed getters) — **dominant cost** +3. `TypeMetadataWrapper` ctor — `GeneratedWriterRegistry` + `GeneratedReaderRegistry` lookups, tracking state init +4. JIT of `WriteObject` / `WriteObjectProperties` / scan pass +5. JIT of generated `WriteProperties` / `ScanObject` / `ScanForDuplicates` (size scales with property count) +6. Cascade: each referenced child type repeats steps 1–5 -### BWO-2: Initialize resets tracking +Subsequent calls hit cached metadata/wrappers → only Tier 0→1 JIT transition remains (background, async). -**Status:** By design -**Affects:** `BufferWriterBinaryOutput.Initialize` (context mode) +**Dominant cost today:** #1–#2 (reflection + `Expression.Compile`). After BIN-T-3, the dominant residual cost shifts to #4–#5 (JIT), addressed by BIN-T-4. -`Initialize` sets `_committedBytes = 0`. Standalone bytes written before are lost if the BWO is then passed to a context. Call `FlushAndReset()` first, or track standalone bytes separately. +**Impact:** Measurable first-call latency — larger for types with many properties or deep graphs. For SignalR workloads the first message per entity type pays this tax. -### BWO-3: Constructor acquires chunk - -**Status:** Acceptable (not a leak) -**Affects:** `BufferWriterBinaryOutput` ctor - -`AcquireChunk` runs in ctor for standalone readiness. Redundant if only context mode is used (context `Initialize` acquires its own). Not a leak — consecutive `GetMemory` without `Advance` returns overlapping memory. - -### BWO-4: No mode mixing - -**Status:** By design -**Affects:** `BufferWriterBinaryOutput` — context vs standalone mode - -A single instance must not use context + standalone modes simultaneously — buffer states desynchronize. One mode per lifecycle phase; `FlushAndReset()` as boundary between modes. - -### SGEN-2: Consumer entity with `new` Id shadowing — excluded from SGen +### BIN-I-11: Consumer entity with `new` Id shadowing — excluded from SGen **Status:** Workaround-in-place (compiled-expression fallback) **Affects:** Any consumer entity whose base class hides `BaseEntity.Id` with `readonly new int Id { get; }` pattern (e.g. `DiscountProductMapping` in Mango.Nop.Core) When the base class shadows `Id` with a setter-less `new int Id { get; }`, SGen can't emit a setter without CS0200. Runtime falls back to compiled-expression serialization for these types. Low priority — affects a small number of consumer entities. -**Related TODO:** `BINARY_TODO.md#todo-02` +**Related TODO:** `BINARY_TODO.md#bin-t-2` -## Cross-cutting (also tracked in SignalR-side docs) +## Buffer Writer (BWO) -### XCUT-1: JSON-in-Binary request parameters +### BIN-I-12: Struct copy semantics -**Status:** Major tech debt, planned replacement (coordinated) -**Affects:** SignalR client→server request parameter path +**Status:** By design +**Affects:** `BufferWriterBinaryOutput` value-type assignment -Request parameters currently use JSON inside a Binary envelope (`SignalPostJsonDataMessage`). Response path already uses pure Binary. Planned migration is a cross-project coordinated change — see `AyCode.Services/docs/SIGNALR_ISSUES.md` PROTO section and `BINARY_TODO.md#todo-01` for the broader picture. Do NOT attempt as a side-effect of unrelated work. +Assigning a `BufferWriterBinaryOutput` value creates an independent copy. State changes (e.g. `_committedBytes` via `Grow`/`Flush`) are not reflected in the original. Copy back after use if needed. + +### BIN-I-13: Initialize resets tracking + +**Status:** By design +**Affects:** `BufferWriterBinaryOutput.Initialize` (context mode) + +`Initialize` sets `_committedBytes = 0`. Standalone bytes written before are lost if the BWO is then passed to a context. Call `FlushAndReset()` first, or track standalone bytes separately. + +### BIN-I-14: Constructor acquires chunk + +**Status:** Acceptable (not a leak) +**Affects:** `BufferWriterBinaryOutput` ctor + +`AcquireChunk` runs in ctor for standalone readiness. Redundant if only context mode is used (context `Initialize` acquires its own). Not a leak — consecutive `GetMemory` without `Advance` returns overlapping memory. + +### BIN-I-15: No mode mixing + +**Status:** By design +**Affects:** `BufferWriterBinaryOutput` — context vs standalone mode + +A single instance must not use context + standalone modes simultaneously — buffer states desynchronize. One mode per lifecycle phase; `FlushAndReset()` as boundary between modes. + +## Cross-cutting (canonical home: `../XCUT/`) + +### XCUT-I-1: JSON-in-Binary request parameters — cross-ref + +Canonical entry: **`../XCUT/XCUT_ISSUES.md#xcut-i-1`**. Summary: client→server request parameters currently use JSON inside a Binary envelope (`SignalPostJsonDataMessage`); response path is already pure Binary. Planned migration is tracked in `BINARY_TODO.md#bin-t-1` but requires coordinated client+server+consumer changes. Do NOT attempt as a side-effect. diff --git a/AyCode.Core/docs/BINARY_OPTIONS.md b/AyCode.Core/docs/BINARY/BINARY_OPTIONS.md similarity index 100% rename from AyCode.Core/docs/BINARY_OPTIONS.md rename to AyCode.Core/docs/BINARY/BINARY_OPTIONS.md diff --git a/AyCode.Core/docs/BINARY_SGEN.md b/AyCode.Core/docs/BINARY/BINARY_SGEN.md similarity index 83% rename from AyCode.Core/docs/BINARY_SGEN.md rename to AyCode.Core/docs/BINARY/BINARY_SGEN.md index ca87114..5ab34ad 100644 --- a/AyCode.Core/docs/BINARY_SGEN.md +++ b/AyCode.Core/docs/BINARY/BINARY_SGEN.md @@ -164,3 +164,14 @@ void ScanObject(object value, BinarySerializationContext ctx, - **Large object graphs**: deep nesting benefits most from zero-dispatch property access - **Small payloads**: root fast path eliminates dispatch overhead that dominates at small sizes - **Not needed for**: one-off serialization, 3rd-party types, Expression/IQueryable types + +## Cold-Start Overhead & Planned Mitigation + +First use of an SGen type per process incurs reflection + `Expression.Compile` in the `BinarySerializeTypeMetadata` / `BinaryDeserializeTypeMetadata` constructors — dominant first-call cost today. Subsequent calls reuse cached metadata and wrappers. + +See `BINARY_ISSUES.md#bin-i-10` for the full cost chain and `BINARY_TODO.md#bin-t-3` / `#todo-04` for planned work: + +- **BIN-T-3 — Compile-time metadata generation.** Source generator emits pre-built `BinarySerializeTypeMetadata` (and deserialize counterpart) with hashes, flags, and `MinWriteSize` baked in. Registered from `ModuleInit` into a `GeneratedMetadataRegistry`; `GetWrapperSlow` consults it before falling back to the reflection `MetadataFactory`. A lazy `TypeMetadataBase.RuntimeInit()` builds the `Expression.Compile` property accessors only when needed (runtime-only type, or `UseGeneratedCode=false`). SGen types never touch their own runtime accessors → `RuntimeInit` skipped. Hybrid mode is unaffected: non-SGen child types continue to run the reflection factory. +- **BIN-T-4 — JIT Tier 1 warmup.** After BIN-T-3, JIT of generated `WriteProperties` / `ScanObject` / `ScanForDuplicates` becomes the residual first-call cost. Candidates to evaluate (measure first): `[MethodImpl(MethodImplOptions.AggressiveOptimization)]` on generated methods, background `RuntimeHelpers.PrepareMethod` from `ModuleInit`, ReadyToRun (R2R) in consuming-project publish config, code chunking for oversized methods. + +Wire format is unchanged by both TODOs — this is dispatch / startup optimization only. diff --git a/AyCode.Core/docs/BINARY/BINARY_TODO.md b/AyCode.Core/docs/BINARY/BINARY_TODO.md new file mode 100644 index 0000000..6e517a7 --- /dev/null +++ b/AyCode.Core/docs/BINARY/BINARY_TODO.md @@ -0,0 +1,57 @@ +# AcBinarySerializer — TODO + +## Priority legend +- **P0** blocker · **P1** important · **P2** nice-to-have · **P3** idea + +--- + +## BIN-T-1: Replace JSON-in-Binary request parameters +**Priority:** P1 · **Type:** Refactor · **Related:** `../XCUT/XCUT_ISSUES.md#xcut-i-1` (canonical), `AyCode.Services/docs/SIGNALR/SIGNALR_TODO.md` + +Migrate client→server request parameters from JSON-in-Binary envelope to direct Binary serialization (matching response path). Coordinated change across client, server, and all consuming projects. Do NOT attempt as side-effect of unrelated work. + +**Acceptance:** `SignalPostJsonDataMessage` replaced by a `SignalPostBinaryDataMessage` (or equivalent); no JSON round-trip on the wire for request params; benchmarks confirm no regression. + +## BIN-T-2: Re-evaluate DiscountProductMapping SGen exclusion +**Priority:** P3 · **Type:** Investigation · **Related:** `BINARY_ISSUES.md#bin-i-11` + +Investigate whether the `new int Id` shadowing pattern can be handled by SGen (via base-class introspection, property-setter lookup on the base) to eliminate the runtime compiled-expression fallback for this entity class. + +## BIN-T-3: Generate `BinarySerializeTypeMetadata` / `BinaryDeserializeTypeMetadata` at compile time +**Priority:** P1 · **Type:** Performance · **Related:** `BINARY_ISSUES.md#bin-i-10` + +Eliminate the dominant first-call cost (reflection + `Expression.Compile` in metadata ctor) for SGen types by emitting pre-built metadata from the source generator. + +**Design outline:** + +- `TypeMetadataBase` / `BinarySerializeTypeMetadata` / `BinaryDeserializeTypeMetadata` get a second constructor that accepts pre-computed values (hashes, `MinWriteSize`, `ComplexPropertyCount`, flags, `IsIId`, `IdAccessorType`, etc.). No reflection executes in this ctor. +- Source generator keeps its existing `s_typeNameHash` / `s_propertyHashes` static fields (hot-path access stays static, zero indirection) and passes the **same references** to the metadata — single source of truth, no duplicate computation. +- `ModuleInit` registers both the writer/reader **and** the pre-built metadata into a `GeneratedMetadataRegistry`. `GetWrapperSlow` consults this registry first, falling back to the reflection-based `MetadataFactory` for runtime-only types. +- Lazy `RuntimeInit()` pattern for `Expression.Compile` property accessors: + - `TypeMetadataBase` gets `volatile bool _runtimeInitialized` + `internal void RuntimeInit()` (idempotent, no lock needed). + - `GetWrapperSlow` calls `metadata.RuntimeInit()` only when `wrapper.GeneratedWriter == null || !Options.UseGeneratedCode` — SGen types skip it entirely (they never touch runtime accessors on their own metadata; non-SGen child types have their own metadata and run the factory path normally). + - Hybrid mode stays correct: an SGen type on the SGen path never uses its own property accessors; a non-SGen child type's metadata runs the reflection ctor as today. +- `volatile` guards the flag; multiple contexts may race into `RuntimeInit`, second run is a no-op. + +**Thread safety:** `GlobalMetadataCache` is `ConcurrentDictionary`; generated metadata is registered once at `ModuleInit`; wrapper construction is per-context and unchanged. + +**Acceptance:** +- Cold benchmark: first `Serialize` of a fresh SGen type shows no reflection / `Expression.Compile` on the call stack. +- Runtime fallback (`UseGeneratedCode=false`) still produces identical wire output and uses the full metadata accessors. +- Deserialize side has parity (same approach for `BinaryDeserializeTypeMetadata`). +- Existing tests pass; wire format unchanged. + +## BIN-T-4: JIT Tier 1 warmup for generated hot methods +**Priority:** P2 · **Type:** Performance · **Related:** `BINARY_ISSUES.md#bin-i-10` + +After BIN-T-3 lands, JIT of generated `WriteProperties` / `ScanObject` / `ScanForDuplicates` becomes the dominant residual first-call cost for SGen types. Options to evaluate (benchmark before committing): + +- **`[MethodImpl(MethodImplOptions.AggressiveOptimization)]`** on the generated hot methods — skips Tier 0, compiles directly at Tier 1. Simple generator change. Trade-off: larger one-time JIT cost in exchange for eliminating the Tier 0→1 recompile step. +- **Background prewarm from `ModuleInit`**: `Task.Run(() => RuntimeHelpers.PrepareMethod(handle))` for each registered writer/reader method. Parallelizes JIT with app startup. Keep it opt-in (option flag) to avoid surprising consumers with extra startup threads. +- **ReadyToRun (R2R)** in consuming projects' publish config — pre-compiles IL to native at publish time. External to SGen, complementary. Document as a recommended publish setting. +- **Code chunking** (split generated methods exceeding a property threshold into sub-methods, e.g. `WriteProperties_Part1` / `_Part2`) — **measure first**. Only beneficial for unusually large types (20+ properties / nested collections). Call overhead can offset gains; JIT inliner may already handle reasonably-sized methods well. +- **Native AOT** — out of scope for this TODO; separate architectural decision with deployment-model implications. + +**Acceptance:** +- Benchmark a realistic entity graph (≥ 3 referenced child types) and show first-call time within ~10% of steady-state after BIN-T-3 + chosen mitigation(s). +- Document which combination is recommended for SignalR hot-path workloads vs. batch serialization. diff --git a/AyCode.Core/docs/BINARY_WRITERS.md b/AyCode.Core/docs/BINARY/BINARY_WRITERS.md similarity index 100% rename from AyCode.Core/docs/BINARY_WRITERS.md rename to AyCode.Core/docs/BINARY/BINARY_WRITERS.md diff --git a/AyCode.Core/docs/BINARY/README.md b/AyCode.Core/docs/BINARY/README.md new file mode 100644 index 0000000..63775fa --- /dev/null +++ b/AyCode.Core/docs/BINARY/README.md @@ -0,0 +1,24 @@ +# BINARY — AcBinary serializer + +Reference documentation for the AcBinary serialization system. Primary goal: **speed** (two-phase scan+serialize, reference tracking, string interning). + +## Files in this folder + +- [`BINARY_FEATURES.md`](BINARY_FEATURES.md) — High-level features and capabilities +- [`BINARY_FORMAT.md`](BINARY_FORMAT.md) — Wire format specification +- [`BINARY_OPTIONS.md`](BINARY_OPTIONS.md) — Configuration options (`AcBinaryOptions`) +- [`BINARY_IMPLEMENTATION.md`](BINARY_IMPLEMENTATION.md) — Internal implementation details +- [`BINARY_WRITERS.md`](BINARY_WRITERS.md) — Writer internals (streaming, buffering) +- [`BINARY_SGEN.md`](BINARY_SGEN.md) — Source generator (`AyCode.Core.Serializers.SourceGenerator`) +- [`BINARY_ISSUES.md`](BINARY_ISSUES.md) — Known issues and limitations +- [`BINARY_TODO.md`](BINARY_TODO.md) — Planned work / open tickets + +## Start here + +For a first read-through, start with [`BINARY_FEATURES.md`](BINARY_FEATURES.md) for the overview, then dive into [`BINARY_FORMAT.md`](BINARY_FORMAT.md) for wire-level details. [`BINARY_SGEN.md`](BINARY_SGEN.md) explains how the code-gen integrates at build time. + +## Cross-references + +- **Serialization overview** (Toon vs AcBinary vs AcJson, shared infrastructure): `../../Serializers/README.md` +- **SignalR binary transport** (uses this serializer): `../../AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md` +- **Glossary terms**: `../../../docs/GLOSSARY.md` diff --git a/AyCode.Core/docs/BINARY_TODO.md b/AyCode.Core/docs/BINARY_TODO.md deleted file mode 100644 index cedcb83..0000000 --- a/AyCode.Core/docs/BINARY_TODO.md +++ /dev/null @@ -1,18 +0,0 @@ -# AcBinarySerializer — TODO - -## Priority legend -- **P0** blocker · **P1** important · **P2** nice-to-have · **P3** idea - ---- - -## TODO-01: Replace JSON-in-Binary request parameters -**Priority:** P1 · **Type:** Refactor · **Related:** `BINARY_ISSUES.md#issue-01`, `AyCode.Services/docs/SIGNALR_TODO.md` - -Migrate client→server request parameters from JSON-in-Binary envelope to direct Binary serialization (matching response path). Coordinated change across client, server, and all consuming projects. Do NOT attempt as side-effect of unrelated work. - -**Acceptance:** `SignalPostJsonDataMessage` replaced by a `SignalPostBinaryDataMessage` (or equivalent); no JSON round-trip on the wire for request params; benchmarks confirm no regression. - -## TODO-02: Re-evaluate DiscountProductMapping SGen exclusion -**Priority:** P3 · **Type:** Investigation · **Related:** `BINARY_ISSUES.md#issue-02` - -Investigate whether the `new int Id` shadowing pattern can be handled by SGen (via base-class introspection, property-setter lookup on the base) to eliminate the runtime compiled-expression fallback for this entity class. diff --git a/AyCode.Core/docs/LOGGING_ISSUES.md b/AyCode.Core/docs/LOGGING/LOGGING_ISSUES.md similarity index 69% rename from AyCode.Core/docs/LOGGING_ISSUES.md rename to AyCode.Core/docs/LOGGING/LOGGING_ISSUES.md index 8af7335..83ed985 100644 --- a/AyCode.Core/docs/LOGGING_ISSUES.md +++ b/AyCode.Core/docs/LOGGING/LOGGING_ISSUES.md @@ -2,7 +2,7 @@ For planned/actionable work see `LOGGING_TODO.md`. -## ISSUE-01: NopLogWriter ctor signature mismatch (consumer-specific but framework-exposed) +## LOG-I-1: NopLogWriter ctor signature mismatch (consumer-specific but framework-exposed) **Severity:** Minor (caught, non-blocking, but noisy) · **Status:** Open · **Area:** Writer-instantiation contract (`AcLoggerBase(string)` config-reading ctor) @@ -20,9 +20,9 @@ Two logger-construction paradigms coexist: Console.Error noise tolerated. Alternatively, consumer uses DI-based `AddAcLoggerFactory` (see LOGGING.md) instead of the config-reading ctor — this path doesn't touch `LogWriters[]`. ### Related TODO -`LOGGING_TODO.md#todo-01` +`LOGGING_TODO.md#log-t-1` -## ISSUE-02: AcEnv.AppConfiguration is filesystem-bound, MAUI/WASM-unsafe +## LOG-I-2: AcEnv.AppConfiguration is filesystem-bound, MAUI/WASM-unsafe **Severity:** Minor · **Status:** Documented limitation · **Area:** `AyCode.Core.Consts.AcEnv` @@ -36,9 +36,9 @@ Design predates `IOptions`/DI pattern. Consumer avoids the config-reading `AcLoggerBase(string)` ctor on these platforms. DI-based `AddAcLoggerFactory` + `services.Configure(...)` is the replacement (see LOGGING.md). ### Related TODO -`LOGGING_TODO.md#todo-02` +`LOGGING_TODO.md#log-t-2` -## ISSUE-03: Two parallel logger-setup patterns +## LOG-I-3: Two parallel logger-setup patterns **Severity:** Minor (confusion, not functional) · **Status:** Documented · **Area:** LOGGING.md / consumer code @@ -47,17 +47,17 @@ Two ways to construct a logger coexist: 1. **Config-reading:** `new Logger(categoryName)` → `AcLoggerBase` reads `AyCode:Logger` section via `AcEnv.AppConfiguration`, instantiates writers via `Activator.CreateInstance` 2. **DI factory:** `services.Configure(...)` + `services.AddAcLoggerFactory()` → `Func` resolved from DI, writers pulled from DI -Consumer picks per scenario; no automatic bridge. Risk: mixing patterns causes subtle failures (e.g. `MissingMethodException` — see ISSUE-01). +Consumer picks per scenario; no automatic bridge. Risk: mixing patterns causes subtle failures (e.g. `MissingMethodException` — see LOG-I-1). ### Related TODO -`LOGGING_TODO.md#todo-03` +`LOGGING_TODO.md#log-t-3` -## ISSUE-04: Default LogLevel diverges across the two setup paths +## LOG-I-4: Default LogLevel diverges across the two setup paths **Severity:** Minor (surprise, not broken) · **Status:** Open · **Area:** `AcLoggerBase` field initializer vs `AcLoggerOptions` ### Description -The two logger-setup paths (see ISSUE-03) ship with **different default LogLevels**: +The two logger-setup paths (see LOG-I-3) ship with **different default LogLevels**: - `AcLoggerBase.cs:18` — field initializer: `LogLevel = Error` - `AcLoggerOptions.cs:27` — DI options default: `LogLevel = Info` @@ -71,9 +71,9 @@ Same consumer code, same "no configuration" state, two different log volumes dep Historical: field initializer predates the Options class; Options was added as part of the DI-factory refactor with a developer-friendly `Info` default. ### Related TODO -`LOGGING_TODO.md#todo-05` +`LOGGING_TODO.md#log-t-5` -## ISSUE-05: `AcConsoleLogWriter.Initialize()` runs twice on parameterless ctor +## LOG-I-5: `AcConsoleLogWriter.Initialize()` runs twice on parameterless ctor **Severity:** Minor (wasted work, not broken) · **Status:** Open · **Area:** `AcConsoleLogWriter` ctor chain @@ -97,9 +97,9 @@ Chain `: this(null)` invokes the `(string?)` ctor, which calls `Initialize()`. C Remove the second `Initialize()` call in the parameterless ctor body (the chain already covered it). ### Related TODO -`LOGGING_TODO.md#todo-06` +`LOGGING_TODO.md#log-t-6` -## ISSUE-06: `ILogger.IsEnabled(MsLogLevel.None)` incorrectly reports enabled +## LOG-I-6: `ILogger.IsEnabled(MsLogLevel.None)` incorrectly reports enabled **Severity:** Low (semantic bug, rare path) · **Status:** Open · **Area:** `AcLoggerBase.IsEnabled` + `MapFromMsLogLevel` @@ -125,9 +125,9 @@ Either: - **(b)** Change the comparison to strict `<` with `Disabled` as sentinel (larger refactor — affects semantic of `Disabled` elsewhere) ### Related TODO -`LOGGING_TODO.md#todo-07` +`LOGGING_TODO.md#log-t-7` -## ISSUE-07: Misleading inline comment in `AcLoggerBase.Log` +## LOG-I-7: Misleading inline comment in `AcLoggerBase.Log` **Severity:** Trivial (doc-only) · **Status:** Open · **Area:** `AcLoggerBase.cs:210-211` @@ -143,7 +143,34 @@ Comment says fallback is `null` (empty display), the code assigns `"Log"`. Contr Update comment to match code: `// Use eventId.Name:eventId.Id if Name is set, otherwise fallback to "Log" per LOGGING.md convention`. ### Related TODO -Folded into `LOGGING_TODO.md#todo-08` (cleanup batch). +Folded into `LOGGING_TODO.md#log-t-8` (cleanup batch). + +## LOG-I-8: Server-side NopCommerce plugin still uses legacy config-reading Logger ctor + +**Severity:** Minor (works, but inconsistent with modern pattern + triggers LOG-I-1 noise) · **Status:** Open · **Area:** Consumer adoption gap in `Nop.Plugin.Misc.AIPlugin/Infrastructure/PluginNopStartup.cs` + +### Description +Client side (`FruitBankHybridApp.*` — Web, Web.Client, MAUI) was migrated to the DI-factory pattern: `services.Configure(...)` + `services.AddAcLoggerFactory()`. The server-side plugin was NOT migrated — it still: + +1. Directly constructs a logger via `new Logger(nameof(AyCodeBinaryHubProtocol))` (`PluginNopStartup.cs:169`) and passes it as `opts.Logger` to the protocol — bypasses `ILogger` DI resolution that `AcSignalRProtocolExtensions.BuildProtocol` already implements. +2. Does NOT call `services.Configure(configuration.GetSection("AyCode:Logger"))`. +3. Does NOT call `services.AddAcLoggerFactory()`. +4. Writer registration exists (`services.AddScoped()` + `NopLogWriter`) but those DI-registered singletons are NOT the instances the `new Logger(...)` ctor sees — the legacy ctor creates a parallel set via `Activator.CreateInstance`. + +The legacy config-reading ctor DOES find the appsettings `AyCode:Logger` section via `AcEnv.AppConfiguration` (filesystem-backed, works on server) — so logging functions. But every `new Logger(...)` call: +- Triggers LOG-I-1 (NopLogWriter ctor mismatch → Console.Error noise) +- Reconstructs writer instances via `Activator` (not singleton-shared with DI-registered writers) +- Is inconsistent with the client-side pattern → two mental models for the same framework + +### Fix direction +See `LOGGING_TODO.md#log-t-11`. + +### Related +- `LOG-I-1` (trigger — NopLogWriter ctor mismatch, currently causing Console.Error noise) +- `LOG-I-3` (root cause — two coexisting setup patterns) +- `LOG-I-4` (consequence — different defaults between paths; server legacy path silently ships with `Error` default unless `AyCode:Logger:LogLevel` is set) +- Sibling gap: `../SIGNALR/SIGNALR_ISSUES.md#sig-i-7` (same server-side plugin, protocol-options adoption gap) +- Plugin doc drift: `Nop.Plugin.Misc.AIPlugin/docs/SIGNALR/README.md:22` still documents the pre-migration `new AcBinaryHubProtocol()` registration (actual code uses `.AddAcBinaryProtocol(opts => {...})`). Update needed. ## Evaluated review findings — NOT bugs (by-design) diff --git a/AyCode.Core/docs/LOGGING_TODO.md b/AyCode.Core/docs/LOGGING/LOGGING_TODO.md similarity index 61% rename from AyCode.Core/docs/LOGGING_TODO.md rename to AyCode.Core/docs/LOGGING/LOGGING_TODO.md index 28dfd43..a28cb3d 100644 --- a/AyCode.Core/docs/LOGGING_TODO.md +++ b/AyCode.Core/docs/LOGGING/LOGGING_TODO.md @@ -5,8 +5,8 @@ --- -## TODO-01: Fix the writer-ctor mismatch for DI-injected writers -**Priority:** P2 · **Type:** Bug fix · **Related:** `LOGGING_ISSUES.md#issue-01` +## LOG-T-1: Fix the writer-ctor mismatch for DI-injected writers +**Priority:** P2 · **Type:** Bug fix · **Related:** `LOGGING_ISSUES.md#log-i-1` Options: - **A)** Provide a fallback `(AppType, LogLevel, string?)` ctor on consumer writers that have DI-heavy primary ctors, with DI resolution via service-locator @@ -15,20 +15,20 @@ Options: Eliminate the per-startup Console.Error noise regardless. -## TODO-02: Expose `AcEnv.AppConfiguration` setter for consumer init -**Priority:** P2 · **Type:** Feature · **Related:** `LOGGING_ISSUES.md#issue-02` +## LOG-T-2: Expose `AcEnv.AppConfiguration` setter for consumer init +**Priority:** P2 · **Type:** Feature · **Related:** `LOGGING_ISSUES.md#log-i-2` Allow `AcEnv.SetConfiguration(IConfiguration)` so consumer `Program.cs` can do `AcEnv.SetConfiguration(builder.Configuration)` at startup. Enables config-reading pattern on MAUI/WASM without filesystem assumptions. Backward-compat: fall back to filesystem if no explicit set. -## TODO-03: Unify or clearly separate config-reading and DI-based patterns -**Priority:** P2 · **Type:** Docs / Refactor · **Related:** `LOGGING_ISSUES.md#issue-03` +## LOG-T-3: Unify or clearly separate config-reading and DI-based patterns +**Priority:** P2 · **Type:** Docs / Refactor · **Related:** `LOGGING_ISSUES.md#log-i-3` Decide the canonical direction: - **(a)** Deprecate config-reading pattern → all consumers migrate to DI factory - **(b)** Keep both, with compile-time guidance (analyzer / XML doc `[Obsolete]` hints / decision tree in LOGGING.md) - **(c)** Merge: DI-factory internally falls back to config-reading when `TLogger` doesn't match the `Activator` ctor shape -## TODO-04: Per-writer LogLevel via appsettings +## LOG-T-4: Per-writer LogLevel via appsettings **Priority:** P2 · **Type:** Feature Extend `AcLoggerOptions` with per-writer LogLevel overrides. Example shape: @@ -42,25 +42,25 @@ Extend `AcLoggerOptions` with per-writer LogLevel overrides. Example shape: ``` Factory applies overrides when constructing writers. Currently writer-LogLevel is hardcoded in writer ctors. -## TODO-05: Unify default LogLevel across setup paths -**Priority:** P1 · **Type:** Behaviour decision + cleanup · **Related:** `LOGGING_ISSUES.md#issue-04` +## LOG-T-5: Unify default LogLevel across setup paths +**Priority:** P1 · **Type:** Behaviour decision + cleanup · **Related:** `LOGGING_ISSUES.md#log-i-4` Decide which default wins (`AcLoggerBase.cs:18` = `Error` vs `AcLoggerOptions.cs:27` = `Info`) and make both paths consistent. Recommendation: **`Info`** — matches the DI/Options path already in use for modern consumers (MAUI, WASM, ASP.NET Core) and is the common developer expectation. Change: drop the field initializer in `AcLoggerBase` (or set it to `Info`). Update LOGGING.md with a single "Default LogLevel = Info" line so this can't drift again. -## TODO-06: Fix `AcConsoleLogWriter.Initialize()` double-run -**Priority:** P3 · **Type:** Bug fix · **Related:** `LOGGING_ISSUES.md#issue-05` +## LOG-T-6: Fix `AcConsoleLogWriter.Initialize()` double-run +**Priority:** P3 · **Type:** Bug fix · **Related:** `LOGGING_ISSUES.md#log-i-5` Remove the redundant `Initialize()` call from the parameterless ctor body — the `: this(null)` chain already invokes it. No behaviour change; removes wasted `Console.ForegroundColor` assignment. -## TODO-07: Fix `ILogger.IsEnabled(MsLogLevel.None)` wrongly reporting enabled -**Priority:** P2 · **Type:** Bug fix · **Related:** `LOGGING_ISSUES.md#issue-06` +## LOG-T-7: Fix `ILogger.IsEnabled(MsLogLevel.None)` wrongly reporting enabled +**Priority:** P2 · **Type:** Bug fix · **Related:** `LOGGING_ISSUES.md#log-i-6` Add `if (logLevel == MsLogLevel.None) return false;` at the top of `AcLoggerBase.IsEnabled`. One-line fix. Optionally add a unit test covering all six MS LogLevel values → expected bool. -## TODO-08: Cleanup batch (low-risk micro-refactor, one commit) -**Priority:** P3 · **Type:** Cleanup · **Related:** `LOGGING_ISSUES.md#issue-07` +## LOG-T-8: Cleanup batch (low-risk micro-refactor, one commit) +**Priority:** P3 · **Type:** Cleanup · **Related:** `LOGGING_ISSUES.md#log-i-7` Single commit covering: - Remove unused usings in `AcLoggerBase.cs` (`System.Security.AccessControl`, `System.Net.Mime.MediaTypeNames`) @@ -68,14 +68,14 @@ Single commit covering: - Fix misleading comment at `AcLoggerBase.cs:210` — current comment says "fallback null" but code assigns `"Log"`. Make comment match code. - Decide on the commented-out batch block in `AcLogItemWriterBase.cs:90-119`: either delete (git history preserves) or convert to a single `// TODO: see LOGGING_TODO.md#todo-XX` marker with a new TODO entry for the batch work. -## TODO-09: Fail-fast ctor validation in `AddAcLoggerFactory` +## LOG-T-9: Fail-fast ctor validation in `AddAcLoggerFactory` **Priority:** P2 · **Type:** Feature `AcLoggerServiceExtensions.AddAcLoggerFactory` uses `Activator.CreateInstance(typeof(TLogger), AppType, LogLevel, categoryName, writers)` only on first logger resolution. Ctor-mismatch therefore surfaces only at first log call, not at `services.BuildServiceProvider()` time. Add a one-time reflection check at registration: `typeof(TLogger).GetConstructor([typeof(AppType), typeof(LogLevel), typeof(string), typeof(IAcLogWriterBase[])])` — if null, throw `InvalidOperationException` with the expected signature in the message. Cost: one reflection call per registration. Benefit: app fails to start (loud) instead of logging being silently broken (quiet). -## TODO-10: Writer exception isolation +## LOG-T-10: Writer exception isolation **Priority:** P2 · **Type:** Resilience Currently `AcLoggerBase.Info(...)` and friends do `LogWriters.ForEach(x => x.Info(...))`. A throwing writer (e.g. `SignaRClientLogItemWriter` during network blip, `AcDbLogItemWriter` during a DB outage) takes down the fan-out → subsequent writers in the list never see the message. @@ -83,3 +83,41 @@ Currently `AcLoggerBase.Info(...)` and friends do `LogWriters.ForEach(x => x.Inf Wrap each writer invocation in `try { ... } catch (Exception ex) { Console.Error.WriteLine($"Writer {x.GetType().Name} failed: {ex.Message}"); }`. Optional: escalate to a circuit-breaker pattern (N failures → disable that writer for M seconds) to avoid Console.Error flood on persistent outages. Must NOT log to `AcLoggerBase` itself inside the catch (reentrancy / infinite loop risk). + +## LOG-T-11: Migrate server-side NopCommerce plugin logger setup to DI-factory +**Priority:** P2 · **Type:** Consumer refactor · **Related:** `LOGGING_ISSUES.md#log-i-8`, `../../../AyCode.Services/docs/SIGNALR/SIGNALR_TODO.md#log-t-5` + +Align the server-side plugin (`Nop.Plugin.Misc.AIPlugin/Infrastructure/PluginNopStartup.cs`) with the client-side setup pattern already used by MAUI / Web / Web.Client consumers. + +### Target diff +```csharp +// In PluginNopStartup.ConfigureServices: + +// 1. Bind Logger options from appsettings.json (replaces AcEnv.AppConfiguration path) +services.Configure(configuration.GetSection("AyCode:Logger")); + +// 2. Register writers (already present) +services.AddScoped(); +services.AddScoped(); + +// 3. Register the logger factory — replaces direct `new Logger(...)` calls +services.AddAcLoggerFactory(); + +// 4. In the SignalR registration — drop the manual Logger construction +.AddAcBinaryProtocol(opts => +{ + opts.ProtocolMode = BinaryProtocolMode.AsyncSegment; + // Remove: opts.Logger = new Logger(nameof(AyCodeBinaryHubProtocol)); + // — AcSignalRProtocolExtensions.BuildProtocol auto-resolves ILogger from DI +}); +``` + +### Consequences / checklist +- [ ] Consumers of `new Logger(...)` elsewhere in the plugin code must switch to injecting `Func` (DI factory). +- [ ] LOG-I-1 Console.Error noise disappears once the config-reading path is no longer invoked. +- [ ] Legacy `AyCode:Logger:LogWriters[]` array in `appsettings.json` becomes unused — decide: **(a)** remove (breaking if any other consumer still reads it), **(b)** keep for back-compat with a comment "superseded by DI, see LOGGING.md". +- [ ] Update `Nop.Plugin.Misc.AIPlugin/docs/SIGNALR/README.md:22` — current text still describes `services.AddSingleton(new AcBinaryHubProtocol())` which has NOT been the actual registration for months. +- [ ] Verify server startup log — `AcLoggerOptions` must bind cleanly (no "missing section" warnings from `IOptionsMonitor`). + +### Why this is a separate TODO from LOG-T-3 +LOG-T-3 is about the **framework-level decision** (keep both patterns vs deprecate legacy). LOG-T-11 is the **consumer-level execution** of that decision for the server plugin — independent of whether LOG-T-3 ultimately deprecates the legacy path or keeps it alive. Migrating the server plugin removes LOG-I-1 noise regardless of LOG-T-3's outcome. diff --git a/AyCode.Core/docs/LOGGING.md b/AyCode.Core/docs/LOGGING/README.md similarity index 98% rename from AyCode.Core/docs/LOGGING.md rename to AyCode.Core/docs/LOGGING/README.md index d27fa21..3810a13 100644 --- a/AyCode.Core/docs/LOGGING.md +++ b/AyCode.Core/docs/LOGGING/README.md @@ -2,8 +2,8 @@ Custom logging framework with multi-writer fan-out and `Microsoft.Extensions.Logging` integration. Source: `Loggers/` in this project. -> For server-side GlobalLogger see `AyCode.Core.Server/docs/LOGGING_SERVER.md`. -> For remote writers (HTTP, browser console, SignalR) see `AyCode.Services/docs/LOGGING_REMOTE.md`. +> For server-side GlobalLogger see `AyCode.Core.Server/docs/LOGGING/README.md`. +> For remote writers (HTTP, browser console, SignalR) see `AyCode.Services/docs/LOGGING/README.md`. ## Design Overview @@ -120,7 +120,7 @@ Each writer's constructor signature must accept `(AppType, LogLevel, string?)`. ## DI-Based Factory Pattern -Modern, framework-first alternative to the config-reading `AcLoggerBase(string)` ctor. No runtime reflection over writer config; concrete writer types resolved from DI. Recommended for all modern projects (MAUI, WASM, ASP.NET Core) — the config-reading path is filesystem-bound and unsuitable for MAUI/WASM (see `LOGGING_ISSUES.md#issue-02`). +Modern, framework-first alternative to the config-reading `AcLoggerBase(string)` ctor. No runtime reflection over writer config; concrete writer types resolved from DI. Recommended for all modern projects (MAUI, WASM, ASP.NET Core) — the config-reading path is filesystem-bound and unsuitable for MAUI/WASM (see `LOGGING_ISSUES.md#log-i-2`). ### Consumer setup in Program.cs @@ -186,7 +186,7 @@ Use the two-arg overload when the consumer separates client-only and server-only ### Companion extension: AddAcDefaults (SignalR) -For SignalR client setup that wires the same `AcLoggerBase` instance into Microsoft.Extensions.Logging, use `AcSignalRConnectionExtensions.AddAcDefaults(builder, logger, connectionOptions)` — it bundles `AddAcConnection` + `ConfigureLogging(AddAcLogger)` in a single call. See `AyCode.Services/docs/SIGNALR.md`. +For SignalR client setup that wires the same `AcLoggerBase` instance into Microsoft.Extensions.Logging, use `AcSignalRConnectionExtensions.AddAcDefaults(builder, logger, connectionOptions)` — it bundles `AddAcConnection` + `ConfigureLogging(AddAcLogger)` in a single call. See `AyCode.Services/docs/SIGNALR/README.md`. ### Comparison: config-reading ctor vs DI-based factory @@ -199,7 +199,7 @@ For SignalR client setup that wires the same `AcLoggerBase` instance into Micros | Typical use | Legacy / server-side config-driven | Modern DI: MAUI, WASM, ASP.NET Core | | MAUI/WASM-safe | ❌ (filesystem-bound `AcEnv.AppConfiguration`) | ✅ | -Both patterns can coexist — consumer picks per scenario. Related issues: `LOGGING_ISSUES.md`. Planned unification: `LOGGING_TODO.md#todo-03`. +Both patterns can coexist — consumer picks per scenario. Related issues: `LOGGING_ISSUES.md`. Planned unification: `LOGGING_TODO.md#log-t-3`. ## Core Components diff --git a/AyCode.Core/docs/README.md b/AyCode.Core/docs/README.md new file mode 100644 index 0000000..02e882c --- /dev/null +++ b/AyCode.Core/docs/README.md @@ -0,0 +1,14 @@ +# AyCode.Core documentation + +Topic documentation for the `AyCode.Core` project (Layer 0 framework). + +## Topics + +- [`BINARY/`](BINARY/README.md) — AcBinary serializer: features, format, writers, source generator +- [`LOGGING/`](LOGGING/README.md) — Logger system: levels, writers, config-reading vs DI factory +- [`TOON/`](TOON/README.md) — Toon serializer: LLM-optimized format with @meta/@types/@data sections +- [`XCUT/`](XCUT/README.md) — Cross-cutting issues/TODOs that span ≥2 topics (canonical home; referenced from each affected topic) + +## Navigation + +Per the AI Agent Core Protocol (folder navigation rule), start from this README when browsing `docs/`. Each topic folder has its own `README.md` with the main content, plus optional `ISSUES.md` (known issues) and `TODO.md` (planned work). diff --git a/AyCode.Core/docs/TOON/README.md b/AyCode.Core/docs/TOON/README.md new file mode 100644 index 0000000..66ca573 --- /dev/null +++ b/AyCode.Core/docs/TOON/README.md @@ -0,0 +1,118 @@ +# TOON — Token-Oriented Object Notation + +LLM-optimized serializer. Primary goal: **maximize LLM accuracy** via explicit schema/data separation, rich metadata, and unambiguous structure boundaries. Serialize-only (no deserializer — see `TOON_TODO.md#toon-t-6`). + +Source: `Serializers/Toons/` in this project. + +## Design goals + +1. **Zero ambiguity** — explicit `{}` / `[]` / `"""` boundaries; no indentation-only scoping. +2. **Rich semantic context** — every property carries description, purpose, constraints, examples in the schema section. +3. **Token efficiency** — schema sent once (`@meta`/`@types`), data streamed thereafter (`@data`); enum values numeric rather than by name. +4. **Smart defaults** — useful output without any attributes; `ToonDescriptionAttribute` for precision when conventions fall short. + +## Three-section output + +| Section | Contents | +|---|---| +| `@meta` | Format version, source language, type list, optional domain context. | +| `@types` | Per-type schema — description, purpose, constraints, examples, navigation metadata, enum value maps. | +| `@data` | Object instance values; nested with inline references for shared/circular objects. | + +## Three modes (`ToonSerializationMode`) + +| Mode | When to use | +|---|---| +| `Full` (default) | First serialization — LLM needs the full schema + data context. | +| `MetaOnly` | Send schema once at conversation start; subsequent sends can skip it. | +| `DataOnly` | Subsequent sends when the LLM already has the schema — 30-50% token savings. | + +Details, preset breakdown, all options: `TOON_OPTIONS.md`. + +## Quick start + +```csharp +using AyCode.Core.Serializers.Toons; + +var person = new Person { Id = 1, Name = "Alice", Email = "alice@example.com" }; +string toon = AcToonSerializer.Serialize(person); +``` + +Output (Full mode, abbreviated): + +```toon +@meta { + version = "1.0" + format = "toon" + source-code-language = "C#" + types = ["Person"] +} + +@types { + Person: "Object of type Person" + Id: int32 + description: "Unique identifier for Person" + purpose: "Primary key / unique identification" + constraints: "required" + Name: string + description: "Name of the Person" + constraints: "required" + Email: string + description: "Email address" + constraints: "required, email-format" +} + +@data { + Person { + Id = 1 + Name = "Alice" + Email = "alice@example.com" + } +} +``` + +## Multi-turn pattern + +```csharp +// Turn 1 — send schema only +string schema = AcToonSerializer.Serialize(person, AcToonSerializerOptions.MetaOnly); + +// Turn 2..N — send data only (LLM already has the schema from Turn 1) +string data = AcToonSerializer.Serialize(person, AcToonSerializerOptions.DataOnly); +``` + +## Public API entry points + +| Method | Purpose | +|---|---| +| `AcToonSerializer.Serialize(T value)` | Default options (Full mode). | +| `AcToonSerializer.Serialize(T value, AcToonSerializerOptions)` | Custom options. | +| `AcToonSerializer.Serialize(T value, string domainDescription, AcToonSerializerOptions)` | Adds `context = "..."` to `@meta`. | +| `AcToonSerializer.SerializeTypeMetadata()` / `SerializeTypeMetadata(Type)` | Schema only (MetaOnly preset internally). | +| `AcToonSerializer.SerializeMetadata(IEnumerable)` / `SerializeMetadata(params Type[])` | Schema for multiple types without an instance. | + +## Choosing between Toon, AcBinary, AcJson + +| Use case | Choose | +|---|---| +| Wire format for LLM context / prompts | **Toon** | +| High-throughput wire format for non-LLM transport | `AcBinary` — see `../BINARY/README.md` | +| Interop with external JSON consumers | `AcJson` | + +Full serializer comparison: see `../../Serializers/README.md`. + +## Files in this folder + +- `README.md` — this file (overview + navigation). +- `TOON_FORMAT.md` — wire format specification (exact grammar). +- `TOON_OPTIONS.md` — `AcToonSerializerOptions` reference, presets, API overloads. +- `TOON_INFERENCE.md` — smart description/constraint auto-generation from property names. +- `TOON_ATTRIBUTES.md` — `ToonDescriptionAttribute` API + placeholder system. +- `TOON_IMPLEMENTATION.md` — internals, partial class breakdown, context pattern. +- `TOON_ISSUES.md` — known limitations. +- `TOON_TODO.md` — planned work. + +## Cross-references + +- Shared serializer infrastructure (metadata cache, reference tracking, `IdentityMap`): `../../Serializers/README.md`. +- Folder-README-first navigation rule: `../../../.github/copilot-instructions.md` rule #21. diff --git a/AyCode.Core/docs/TOON/TOON_ATTRIBUTES.md b/AyCode.Core/docs/TOON/TOON_ATTRIBUTES.md new file mode 100644 index 0000000..007e329 --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_ATTRIBUTES.md @@ -0,0 +1,194 @@ +# Toon — `ToonDescriptionAttribute` + Placeholder System + +Manual metadata override for types and properties. Definition: `Serializers/Toons/ToonDescriptionAttribute.cs`. + +Use when smart inference (`TOON_INFERENCE.md`) is insufficient — domain-specific constraints, business rules, or navigation metadata. + +## Attribute signature + +```csharp +[AttributeUsage(AttributeTargets.Property | AttributeTargets.Class, + AllowMultiple = false, Inherited = true)] +public sealed class ToonDescriptionAttribute : Attribute +``` + +## Text properties + +| Property | Emitted in `@types` as | Fallback if empty | +|---|---|---| +| `Description` | `description: "..."` | MS `[Description]` → smart inference (property) / smart inference (class) | +| `Purpose` | `purpose: "..."` | Smart inference (property only; classes get empty) | +| `Constraints` | `constraints: "..."` | MS DataAnnotations → type-derived → smart inference | +| `Examples` | `examples: "..."` | empty (no automatic fallback) | +| `BusinessRule` | `business-logic: "..."` | empty | + +## Structural properties — class level + +| Property | Emitted as | Notes | +|---|---|---| +| `TableName` | `table-name: "..."` | Fallback: EF `[Table]` / Linq2Db `[Table]` → class name. | +| `TypeRelation` | `related-type: "{relation} {types}"` | Use the `ToonTypeRelation` string constants (below). | +| `RelatedTypes` | (combined with `TypeRelation`) | `Type[]` — related type names joined by `, `. | + +## Structural properties — property level (navigation) + +| Property | Emitted as | Notes | +|---|---|---| +| `IsPrimaryKey` | `primary-key: true` | Convention: a property named `Id` is auto-detected without this. | +| `ForeignKey` | `foreign-key: "..."` | Convention: `CompanyId` ↔ `Company` pair auto-detected. | +| `Navigation` | `navigation: "many-to-one"` | `ToonRelationType` enum; emitted kebab-case. | +| `InverseProperty` | `inverse-property: "..."` | For bidirectional relationships. | + +## Constructors + +```csharp +[ToonDescription] // Named-args only +[ToonDescription("human-readable description")] // Positional Description +``` + +## `ToonTypeRelation` string constants + +Use with the `TypeRelation` property. Values serialize literally: + +| Constant | String value | Example | +|---|---|---| +| `None` | `""` | — | +| `BaseOf` | `"base-of"` | `Order` is base-of `OrderDto` | +| `Entity` | `"entity"` | — | +| `DerivedFrom` | `"derived-from"` | `OrderDto` is derived-from `Order` | +| `DtoOf` | `"dto-of"` | `OrderDto` is dto-of `Order` | +| `ModelOf` | `"model-of"` | `OrderModel` is model-of `Order` | +| `ViewModelOf` | `"view-model-of"` | `OrderViewModel` is view-model-of `Order` | +| `ProjectionOf` | `"projection-of"` | `OrderSummary` is projection-of `Order` | + +## `ToonRelationType` enum + +For the `Navigation` property: + +```csharp +public enum ToonRelationType { ManyToOne, OneToMany, OneToOne, ManyToMany } +``` + +Emitted kebab-case: `many-to-one`, `one-to-many`, `one-to-one`, `many-to-many`. + +## Placeholder system + +Placeholders of the form `[#Name]` inside `Description` / `Purpose` / `Constraints` / `Examples` resolve at serialization time against Microsoft DataAnnotations or smart inference. Implementation: `Serializers/Toons/AcToonSerializer.Placeholders.cs`. + +### Available placeholders + +**For `Description`, `Purpose`:** + +| Placeholder | Resolves to | +|---|---| +| `[#Description]` | MS `[System.ComponentModel.Description]` value. | +| `[#DisplayName]` | MS `[DisplayName]` value. | +| `[#SmartDescription]` | Auto-inferred description (see `TOON_INFERENCE.md`). | +| `[#SmartPurpose]` | Auto-inferred purpose (properties only; empty for classes). | + +**For `Constraints`:** + +| Placeholder | Resolves to | +|---|---| +| `[#Required]` | `required` (when `[Required]` is present). | +| `[#Range]` | `range: min-max` (from `[Range(min, max)]`). | +| `[#MaxLength]` | `max-length: N`. | +| `[#MinLength]` | `min-length: N`. | +| `[#StringLength]` | `length: min-max` (from `[StringLength]`). | +| `[#EmailAddress]` | `email-format`. | +| `[#Phone]` | `phone-format`. | +| `[#Url]` | `url-format`. | +| `[#CreditCard]` | `credit-card-format`. | +| `[#RegularExpression]` | `pattern: ...`. | +| `[#SmartTypeConstraints]` | Type-derived constraints (nullable, numeric). | +| `[#SmartInferenceConstraints]` | Name-based inferred constraints. | + +**For `Examples`:** + +| Placeholder | Resolves to | +|---|---| +| `[#SmartGeneratedExample]` | Auto-generated example value. | + +### Merge vs replace modes + +The presence of a placeholder in `Constraints` flips the merge behavior: + +- **Replace mode** — `Constraints` contains **no** `[#...]` → MS attributes ignored, smart inference ignored, the literal string is used as-is. +- **Merge mode** — `Constraints` contains **any** `[#...]` → placeholder resolved, then merged with type-derived + smart-inferred constraints. + +```csharp +[Range(0, 150)] +[ToonDescription(Constraints = "custom-validation-only")] // REPLACE — Range ignored +public int Score { get; set; } + +[Range(0, 150)] +[ToonDescription(Constraints = "[#Range], verified")] // MERGE — "range: 0-150, verified" +public int Age { get; set; } +``` + +`Description`, `Purpose`, `Examples` use straight substitution (no merge mode). + +## Usage modes + +### Full custom + +```csharp +[ToonDescription("User email address", + Purpose = "Authentication and notifications", + Constraints = "required, email-format, unique", + Examples = "user@example.com")] +public string Email { get; set; } +``` + +### Merge with DataAnnotations + +```csharp +[Description("Contact email")] +[Required] +[EmailAddress] +[ToonDescription(Constraints = "[#Required], [#EmailAddress], unique")] +public string Email { get; set; } +// description: "Contact email" +// constraints: "required, email-format, unique" +``` + +### Partial (some properties only, rest auto) + +```csharp +[Description("Contact email")] +[EmailAddress] +[ToonDescription(Constraints = "[#EmailAddress], unique")] +public string Email { get; set; } +// description: "Contact email" (from [Description]) +// purpose: (inferred, empty) +// constraints: "required, email-format, unique" +// examples: (empty) +``` + +### Full automatic (no `[ToonDescription]`) + +Relies on MS DataAnnotations + smart inference. See `TOON_INFERENCE.md`. + +### Class-level with inheritance + +```csharp +[ToonDescription("Base user entity")] +public class User { } + +[ToonDescription("Admin user account")] // own description +public class AdminUser : User { } + +public class SuperUser : AdminUser { } // inherits AdminUser's description (Inherited = true) +``` + +## When to use which approach + +| Scenario | Approach | +|---|---| +| Generic DTO with conventional property names | Nothing — smart inference covers it. | +| Existing MS DataAnnotations | Smart inference — auto-pulls from annotations. | +| Domain-specific constraint not covered by conventions | `[ToonDescription(Constraints = "...")]` replace mode. | +| Domain rule plus standard validation | `[ToonDescription(Constraints = "[#Required], [#Range], domain-rule")]` merge mode. | +| Navigation property relationship | `[ToonDescription(Navigation = ..., InverseProperty = ..., ForeignKey = ...)]`. | +| Entity metadata (tables, DTO-of mappings) | Class-level `[ToonDescription(TableName = ..., TypeRelation = ..., RelatedTypes = ...)]`. | +| Cross-property business logic | `[ToonDescription(BusinessRule = "this >= NetWeight")]` — descriptive only, not executed. | diff --git a/AyCode.Core/docs/TOON/TOON_FORMAT.md b/AyCode.Core/docs/TOON/TOON_FORMAT.md new file mode 100644 index 0000000..c990d5e --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_FORMAT.md @@ -0,0 +1,200 @@ +# Toon — Wire Format Specification + +Exact grammar of Toon output. Needed by LLMs parsing Toon back to objects, and for verifying generated output against expectations. + +Format version: `"1.0"` (constant `AcToonSerializer.FormatVersion`, emitted as `version` in `@meta`). + +## Top-level structure + +Output contains up to three sections in this order: + +``` +@meta { ... } +@types { ... } +@data { ... } +``` + +Section presence per `ToonSerializationMode`: + +| Mode | `@meta` | `@types` | `@data` | +|---|---|---|---| +| `Full` | ✓ | ✓ | ✓ | +| `MetaOnly` | ✓ | ✓ | — | +| `DataOnly` | — | — | ✓ | + +In `Full` mode with `UseIndentation = true`, a blank line separates `@meta/@types` from `@data`. + +## `@meta` section + +``` +@meta { + version = "1.0" + format = "toon" + source-code-language = "C#" + context = "optional domain description" + types = ["Type1", "Type2", "Type3"] +} +``` + +- `version` — Toon format version. +- `format` — always `"toon"`. +- `source-code-language` — always `"C#"`. +- `context` — emitted only when the `domainDescription` overload was invoked. +- `types` — type short names in **topological order** (dependencies before dependents). Excludes primitives, strings, framework collection generics (`List<>`, `Dictionary<>`, `IEnumerable<>`, `ICollection<>`), and generic type definitions. + +The `@types` section body (when emitted) follows the same topological order. + +## `@types` section + +``` +@types { + TypeName: "description" + [table-name: "..."] + [related-type: "dto-of OtherType"] + [purpose: "..."] + PropertyName: typeHint + [description: "..."] + [purpose: "..."] + [business-logic: "..."] + [constraints: "..."] + [examples: "..."] + [primary-key: true] + [foreign-key: "..."] + [other-key: "..."] + [navigation: "many-to-one"] + [inverse-property: "..."] +} +``` + +- Property metadata indented one level below its property header. +- Optional lines emitted only when the value is non-empty. +- `navigation` values are kebab-case of `ToonRelationType`: `many-to-one`, `one-to-many`, `one-to-one`, `many-to-many`. +- `typeHint` uses **C# short names** — primitives: `int`, `long`, `short`, `byte`, `sbyte`, `uint`, `ulong`, `ushort`, `double`, `float`, `decimal`, `bool`, `string`, `char`; special types: `DateTime`, `DateTimeOffset`, `TimeSpan`, `Guid`; collections: `Person[]`, `List`, `Dictionary`; class / enum names as-is. Nullable suffix: `int?`, `DateTime?`, `TaxDisplayType?`. +- ⚠️ **Two naming conventions coexist.** The `@types` section uses C# short names (above). The `@data` inline type hints (when `UseInlineTypeHints = true`) use full-width tokens like `int32`, `float64`, `datetime` — see the Inline type hints subsection below. + +### Enum types in `@types` + +``` +EnumName: enum + description: "..." + purpose: "..." + underlying-type: "int" + default-value: 0 + values: + MemberName = numericValue + description: "..." + OtherMember = numericValue +``` + +- Members emitted as `Name = numericValue`; per-member description is an indented sub-property. +- When `UseEnhancedMetadata = false`, only the `MemberName = numericValue` list is emitted (no description, purpose, underlying-type, default-value). + +## `@data` section + +### Object syntax + +``` +TypeName { + PropertyName = value + OtherProperty = value +} +``` + +- `TypeName` prefix emitted when `WriteTypeNames = true` (default). +- Properties written one per line; `=` has surrounding spaces only when `UseIndentation = true`, else tight (`prop=value`). +- Properties whose value equals `default(T)` / `null` / `0` / `false` skipped when `OmitDefaultValues = true`. + +### Primitive values + +| Kind | Representation | +|---|---| +| `string` | `"escaped"` inline, or triple-quote for long strings. | +| `int32`/`int64`/`int16`/`byte`/`sbyte`/`uint16`/`uint32`/`uint64` | Plain number, `CultureInfo.InvariantCulture`. | +| `bool` | `true` / `false`. | +| `double` | `G17` format; `NaN`/`±Infinity` → `null`. | +| `float` | `G9` format; `NaN`/`±Infinity` → `null`. | +| `decimal` | Invariant culture. | +| `DateTime` | ISO 8601 in quotes: `"2026-04-24T10:30:00.0000000"` (`"O"` format). | +| `DateTimeOffset` | ISO 8601 with offset in quotes. | +| `TimeSpan` | `"c"` format in quotes. | +| `Guid` | `"D"` format (36-char dashed) in quotes. | +| `char` | single-char string in quotes. | +| `enum` | **Numeric** value of the underlying type (not the member name). `@types` holds the name→value map. | +| `null` | literal `null` (no quotes). | + +### String escaping + +JSON-like escapes: `\"`, `\\`, `\n`, `\r`, `\t`, `\uXXXX` for chars `< 32`. If no escape applies, raw double-quoted. + +### Multi-line strings (triple-quote) + +Automatic when `UseMultiLineStrings = true` AND `string.Length > MultiLineStringThreshold` (default 80): + +``` +Bio = """ + Line 1 + Line 2 + + Line 4 +""" +``` + +- Each content line indented one level deeper than the owning property. +- Closing `"""` at the property's own indent level. +- Line terminators inside the string normalized to platform newline during split. + +### Collection syntax + +``` +PropertyName = (count: N) [ + item1 + item2 +] +``` + +- ` (count: N) ` header emitted only when `ShowCollectionCount = true` (default). +- One item per line. +- Items serialize by runtime type (polymorphic collections supported). + +### Dictionary syntax + +``` +PropertyName = > (count: N) { + key => value + otherKey => otherValue +} +``` + +- Header shows the specific generic type when derivable (via `IsDictionaryType`), else ``. +- Key/value separator is literal ` => ` regardless of indentation setting. + +### Reference syntax + +When `ReferenceHandling != None` and an object is referenced more than once: + +- **First occurrence:** `@N TypeName { ... }` — `N` is the reference ID. +- **Subsequent occurrences:** `@ref:N`. + +Circular references safe — the second visit emits `@ref:N` instead of recursing. When the reference system is disabled (`ReferenceHandling = None`), shared objects serialize redundantly and true cycles cause depth-truncation to `null`. + +### Inline type hints + +When `UseInlineTypeHints = true`: + +``` +Age = 30 +Name = "Alice" +``` + +Hint tokens (full-width naming — **distinct** from the C# short names used in `@types`): `string`, `int32`, `int64`, `int16`, `byte`, `sbyte`, `uint16`, `uint32`, `uint64`, `bool`, `float32`, `float64`, `decimal`, `char`, `datetime`, `datetimeoffset`, `timespan`, `guid`, `enum`. Disabled by default (the same information lives in `@types`). + +## Compact mode quirks + +When `UseIndentation = false`: + +- No leading indent whitespace. +- Property assignments use `=` with no surrounding spaces. +- Enum member list uses `=` with no surrounding spaces. +- No blank line between `@meta` and `@data`. +- `@types` section drops the trailing blank line between type definitions. +- Closing braces/brackets remain on their own lines (structure is still line-based). diff --git a/AyCode.Core/docs/TOON/TOON_IMPLEMENTATION.md b/AyCode.Core/docs/TOON/TOON_IMPLEMENTATION.md new file mode 100644 index 0000000..00a78b0 --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_IMPLEMENTATION.md @@ -0,0 +1,127 @@ +# Toon — Implementation Architecture + +Internal design of `AcToonSerializer`. Read this when modifying the serializer. End users should stay in `TOON_OPTIONS.md` and `TOON_ATTRIBUTES.md`. + +## Partial class breakdown + +`AcToonSerializer` is a `static partial class` split across topical files for cohesion. Each file contributes methods to the same class — no separate types, no inheritance between partials. + +| File | Responsibility | +|---|---| +| `AcToonSerializer.cs` | Public API (`Serialize`, `SerializeTypeMetadata`, `SerializeMetadata`), primitive fast-path, reference scanning, per-type metadata cache. | +| `AcToonSerializer.ToonSerializationContext.cs` | Per-serialization mutable state: buffer, indent level, reference maps. | +| `AcToonSerializer.ToonSerializeTypeMetadata.cs` | Per-type cached reflection data: properties, display names, navigation. | +| `AcToonSerializer.MetaWriter.cs` | `@meta` + `@types` orchestration, type collection, enum backing-field detection. | +| `AcToonSerializer.TypeDefinitions.cs` | Per-type schema emission — property metadata, navigation, enum definitions. | +| `AcToonSerializer.DataSection.cs` | `@data` emission, value dispatcher, object/collection/dict/primitive writers, multi-line strings. | +| `AcToonSerializer.Descriptions.cs` | Smart-inference pattern catalog, fallback chain resolvers. | +| `AcToonSerializer.Attributes.cs` | Attribute processing entry points. | +| `AcToonSerializer.AttributeExtraction.cs` | DataAnnotations → constraint string extraction. | +| `AcToonSerializer.Placeholders.cs` | `[#Name]` placeholder resolution. | +| `AcToonSerializer.TopologicalSort.cs` | Dependency-first type ordering. | +| `AcToonSerializer.Navigation.cs` | Navigation property metadata (conventions + attributes). | +| `AcToonSerializer.ForeignKeys.cs` | FK property detection, inverse property matching. | +| `AcToonSerializer.Validation.cs` | Output validation / sanity checks. | + +Supporting non-partial files: + +| File | Role | +|---|---| +| `AcToonSerializerOptions.cs` | Options POCO + preset factories + mode enum. | +| `AcToonContextBase.cs` | Shared base between serialization contexts. | +| `ToonDescriptionAttribute.cs` | Attribute definition + `ToonRelationType` enum. | +| `ToonTypeRelation.cs` | Type relation string constants. | + +## Context pattern + +``` +AcSerializerContextBase (Serializers/ root) + └─ AcToonContextBase (Toons/ shared base) + └─ ToonSerializationContext (sealed, per-call) +``` + +`ToonSerializationContext` responsibilities: +- Owns the output `StringBuilder` and current indent level. +- Tracks reference IDs via typed `IdentityMap` instances (small-int bitmap + chained hash for larger keys). +- Pooled via `ToonSerializationContextPool.Get(options)` / `Return(context)` — reused across serializations after `ResetTracking()`. + +`ToonSerializeTypeMetadata` (per-type, cached globally in `AcToonSerializer.MetadataCache`): +- Property accessors (`ToonPropertyAccessor`), display names, navigation metadata. +- `CustomDescription` — the `ToonDescriptionAttribute` instance if present. +- Flags: `IsCollection`, `IsDictionary`, `ElementType`. + +Shared infrastructure (metadata cache semantics, `IdentityMap`, `ReferenceTracker`): see `../../Serializers/README.md`. Not duplicated here. + +## Serialization flow + +``` +Serialize(value, options) + ├─ null check → "null" + ├─ TrySerializePrimitiveDirect (fast-path, no context allocated) + ├─ Get context from pool + ├─ ScanReferences (if ReferenceHandling != None) ← marks multi-referenced objects + ├─ switch Mode + │ ├─ MetaOnly → WriteMetaSectionOnly + │ ├─ DataOnly → WriteDataSectionOnly + │ └─ Full → WriteMetaSection + blank line + WriteDataSection + └─ Return context to pool +``` + +## Type collection (`CollectTypes`) + +Recursive traversal starting from the root type: + +- Enums: added, not further traversed. +- Primitives and strings: skipped. +- Already-visited types: skipped (`HashSet` dedup). +- Dictionaries: traverse key type + value type. +- Collections (non-string `IEnumerable`): traverse element type. +- Objects: traverse each declared property's type. + +Additional pass: `DetectAndCollectEnumBackingFields` scans for the `Status` + `StatusId` pattern (see `TOON_INFERENCE.md#enum-backing-field-detection`). + +After collection, `TopologicalSortTypes` orders the set so dependencies come before dependents. Filter applied before sorting: +- Drops primitives, strings. +- Drops `IEnumerable`-implementing non-string types (framework collection machinery). +- Drops generic type definitions. +- Drops well-known collection prefixes (`List`, `ICollection`, `IEnumerable`, `Dictionary`). + +The sorted order is used both for the `@meta.types` list and for the `@types` body emission. + +## Reference scanning + +Pre-pass when `ReferenceHandling != None`. Walks every reachable object exactly once via `TrackForScanning`. An object visited a second time is flagged as "multi-referenced" and assigned an ID. + +During `@data` emission: +- First visit of a multi-referenced object → `@N TypeName { ... }`. +- Subsequent visits → `@ref:N`. + +Circular cycles are safe — the second visit aborts with the `@ref:` form. When reference handling is disabled, shared subgraphs serialize redundantly and true cycles truncate at `MaxDepth` to `null`. + +## Enum output + +Enum values serialize as their **numeric underlying-type value** (not the member name) for token efficiency. The name ↔ value map lives in the `@types` `values:` sub-list; LLMs must cross-reference. + +When `UseEnhancedMetadata = false`, only the bare name/value list is emitted — no description, purpose, underlying-type, or default-value fields. + +## Buffer management + +The context's `StringBuilder` is shared with write helpers: + +- `Write(string)` / `Write(char)` — append, no newline. +- `WriteLine(string)` / `WriteLine()` — append with `\r\n`. +- `WriteIndent()` — emit `CurrentIndentLevel × IndentString`. +- `WriteIndentedLine(string)` — indent + content + newline. +- `WriteProperty(name, value)` — `@meta` helper for `name = value` lines (honors `UseIndentation` for `=` spacing). + +`GetResult()` returns the final string and resets the context for pool reuse. + +## Key design decisions + +- **`static partial class`** — no instance state; per-call state lives only in `ToonSerializationContext`. +- **Sealed context classes** — enables JIT direct calling on hot paths (no virtual dispatch). +- **Context pooling** — repeat serializations allocate no context. +- **Cached per-type metadata** — reflection runs once per type, then hot cache. +- **Parallel code paths for `UseIndentation`** — compact mode avoids space emission on `=` and skips blank lines at write time (not a post-process strip). +- **Fast-path for primitive values** — `Serialize(value)` with a primitive returns without pool access or reflection. +- **No deserialization** — Toon is intentionally one-way. See `TOON_TODO.md#toon-t-6`. diff --git a/AyCode.Core/docs/TOON/TOON_INFERENCE.md b/AyCode.Core/docs/TOON/TOON_INFERENCE.md new file mode 100644 index 0000000..0eef282 --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_INFERENCE.md @@ -0,0 +1,109 @@ +# Toon — Smart Inference + +When no `ToonDescriptionAttribute` is present, Toon auto-generates description, purpose, and constraints by inspecting property names and types. Implemented in `Serializers/Toons/AcToonSerializer.Descriptions.cs`. + +Inference **fills the `@types` section**. It has no effect on `@data` values — those serialize by runtime reflection regardless. + +## Fallback chains + +Per-field, the first non-empty value wins: + +| Field | Priority order | +|---|---| +| `description` | `ToonDescription.Description` (with placeholders resolved) → `System.ComponentModel.[Description]` → inferred-from-name → `null` (line skipped). | +| `purpose` | `ToonDescription.Purpose` (with placeholders) → inferred-from-name → empty (line skipped). | +| `constraints` | `ToonDescription.Constraints` (merge if placeholder, else replace) → MS DataAnnotations (`[Required]`, `[Range]`, `[EmailAddress]`, etc.) → type-derived small-int ranges → name-based inference → post-process append of `readonly` / `not-mapped` / `enum-type`. **Redundancy-conscious**: no auto `nullable` / `required` — the type hint already communicates this. | +| `examples` | `ToonDescription.Examples` (placeholders resolved) → empty (no auto-fallback). | + +Attribute precedence details: `TOON_ATTRIBUTES.md`. + +## Name-based description patterns + +| Property name pattern | Description emitted | +|---|---| +| `Id` (exact, case-insensitive) | `Unique identifier for {TypeName}` | +| `Name` (exact) | `Name of the {TypeName}` | +| contains `Email` | `Email address` | +| contains `Phone` | `Phone number` | +| contains `Address` | `Physical or mailing address` | +| contains `Date` OR type is `DateTime`/`DateTime?` | `Date/time value for {PropertyName}` | +| starts with `Is` + `bool` | `Boolean flag indicating {rest after "Is"}` | +| starts with `Has` + `bool` | `Boolean flag indicating possession of {rest after "Has"}` | +| ends with `Count` + integer type | `Count of {rest before "Count"}` | +| collection (element type known, not `object`) | `Collection of {ElementType} for {TypeName}` | +| dictionary | `Dictionary mapping for {PropertyName} in {TypeName}` | +| fallback | `Property {name} of type {baseType}[ (nullable)]` | + +Matching is case-insensitive where noted and uses `string.Contains` or `StartsWith` as indicated. + +## Name-based purpose patterns + +| Property name pattern | Purpose emitted | +|---|---| +| `Id` (exact) | `Primary key / unique identification` | +| contains `CreatedAt` | `Timestamp when entity was created` | +| contains `UpdatedAt` OR `ModifiedAt` | `Timestamp of last update` | +| contains `DeletedAt` | `Soft delete timestamp` | +| starts with `Is` | `Status flag` | +| contains `Version` | `Version tracking / concurrency control` | +| (fallback) | empty (line skipped) | + +## Type-derived and name-based constraints + +Constraints join with `, `. Inference is **redundancy-conscious** — Toon skips constraints already implied by the property's type hint (visible on the `PropertyName: typeHint` line) to save LLM tokens. Source of this design decision: inline code comments in `AcToonSerializer.AttributeExtraction.cs` → `ExtractTypeConstraints`. + +### Not emitted automatically + +- **`nullable` / `required`** — the type hint conveys this (`int?` vs `int`, `DateTime?` vs `DateTime`, reference type vs non-nullable value type). `required` IS emitted when the property carries an explicit `[Required]` DataAnnotation — that signals intent beyond what the type says. +- **Large-range numeric bounds** — no auto range on `int`, `long`, `ulong`, `float`, `double`, `decimal`. The type name already bounds the value; `range: -2147483648-2147483647` would be noise. +- **Enum property constraints** — none. The `@types` enum definition holds the name ↔ numeric value map. + +### Emitted automatically — small-integer ranges + +Useful extra info because LLMs can misjudge narrow integer type sizes: + +| Type | Added constraint | +|---|---| +| `byte` | `range: 0-255` | +| `sbyte` | `range: -128-127` | +| `short` (`int16`) | `range: -32768-32767` | +| `ushort` (`uint16`) | `range: 0-65535` | +| `uint` (`uint32`) | `range: 0-4294967295` | + +### Emitted automatically — name-based additions + +| Condition | Appended constraint | +|---|---| +| `string` + name contains `Email` | `email-format` | +| `string` + name contains `Url` | `url-format` | +| integer type + name contains `Age` | `range: 0-150` | +| integer type + name ends with `Count` | `non-negative` | +| property has `[NotMappedAttribute]` or `[NotColumnAttribute]` | `not-mapped` | +| read-only property (no public setter) | `readonly` | +| enum backing field detected (see below) | `enum-type: {EnumName}` | + +> ⚠️ **`Age` pattern false positives** — the rule uses `string.Contains("Age", OrdinalIgnoreCase)`, which is not word-boundary-aware. Hits `LanguageId`, `Package*`, `Image*`, `Page*`, `Message*`, `Storage*` and similar. See `TOON_ISSUES.md#toon-i-2` for detected cases and fix options. Lesser risk applies to the `Email`, `Phone`, `Address`, `Url`, `Date`, `Version`, `CreatedAt`, `UpdatedAt`, `DeletedAt` patterns too. + +## Enum backing field detection + +Special pass in `AcToonSerializer.MetaWriter.cs`. + +Trigger conditions — all must hold on the same declaring type: +- An **enum property** (e.g. `Status`) marked with `[NotMapped]` / `[NotColumn]` / `[JsonIgnore]`. +- A **backing property** named `{EnumPropertyName}Id` (e.g. `StatusId`). +- The backing property's type matches the enum's underlying type (`int`/`int?`, `byte`/`byte?`, etc.). + +Effects: +- The enum type is collected and documented in `@types`. +- The backing `StatusId` property's constraint list gains `enum-type: Status`. + +This supports the common "store enum as int in DB, expose as enum on the model" pattern without requiring manual annotation. + +## What Toon does NOT infer + +- **Ranges beyond `Age`** — no `[Range]` equivalent inferred from `Percent`, `Rating`, `Score`, etc. +- **Length limits** — no `max-length` inferred from `Name`, `Description`. +- **Uniqueness** — never inferred without `[ToonDescription(Constraints = "unique")]`. +- **Business logic** — `BusinessRule` requires explicit `[ToonDescription]`. + +For anything beyond the catalogs above, use `ToonDescriptionAttribute` — see `TOON_ATTRIBUTES.md`. diff --git a/AyCode.Core/docs/TOON/TOON_ISSUES.md b/AyCode.Core/docs/TOON/TOON_ISSUES.md new file mode 100644 index 0000000..11097f8 --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_ISSUES.md @@ -0,0 +1,181 @@ +# Toon — Known Issues + +For planned/actionable work see `TOON_TODO.md`. + +## TOON-I-1: `Compact` preset XML doc contradicts code behaviour + +**Severity:** Trivial (doc-only, no runtime effect) · **Status:** Open (direction not decided) · **Area:** `Serializers/Toons/AcToonSerializerOptions.cs` + +### Description +The XML doc on `AcToonSerializerOptions.Compact` claims *"Minimal output, no meta, no indentation"*, but the property initializer sets `UseIndentation = true`. Real output is DataOnly + no type names + no reference handling **with** indentation whitespace. Consumers reading IntelliSense expect a tighter output than they receive. + +```csharp +/// +/// Compact mode: Minimal output, no meta, no indentation. ← doc says no indent +/// ... +/// +public static AcToonSerializerOptions Compact => new() +{ + Mode = ToonSerializationMode.DataOnly, + UseMeta = false, + UseIndentation = true, ← code sets true + ... +}; +``` + +### Root cause +Likely the XML comment predates a later preset revision that flipped `UseIndentation` to `true`. The comment was not updated to match. + +### Fix options (undecided) +- **(a) Fix the doc** — update the XML comment to match current behaviour. "Compact" then means "DataOnly, no type names, no refs" — not literal whitespace-free. +- **(b) Fix the code** — flip `UseIndentation` to `false` in the preset to honour original intent. Output-breaking change for anyone relying on current `Compact` formatting. + +### Known workaround +Build a custom options instance when truly token-minimal output is needed: +```csharp +var trulyCompact = new AcToonSerializerOptions { + Mode = ToonSerializationMode.DataOnly, + UseMeta = false, + UseIndentation = false, + OmitDefaultValues = true, + WriteTypeNames = false, + ReferenceHandling = ReferenceHandlingMode.None +}; +``` + +`TOON_OPTIONS.md` already notes this divergence next to the `Compact` preset; decision here governs whether it persists or gets resolved. + +### Related TODO +None yet — fix direction not decided. + +## TOON-I-2: `Age` inference substring false positive + +**Severity:** Minor (constraint-only, no runtime effect) · **Status:** Open · **Area:** `Serializers/Toons/AcToonSerializer.Descriptions.cs` → `GetPropertyConstraints` / `GetInferredConstraints` + +### Description +Integer properties whose name **contains** the substring `age` (case-insensitive, not word-bounded) erroneously receive the `range: 0-150` constraint from the `Age` inference rule. False positives observed: `LanguageId`, and by pattern `Package*`, `Image*`, `Page*`, `Message*`, `Storage*`, `Stage*`. + +Observed in a 2026-04-24 Toon dump: +``` +LanguageId: int? + constraints: "range: 0-150" +``` + +### Root cause +```csharp +if (propertyName.Contains("Age", StringComparison.OrdinalIgnoreCase)) + constraints.Add("range: 0-150"); +``` + +`"LanguageId".Contains("Age", OrdinalIgnoreCase)` returns `true` — the substring `age` sits at position 5-7 inside `Language`. The rule's intent was to match an `Age`-named property, but `Contains` has no word-boundary awareness. + +### Fix options +- **(a)** Exact match: `propertyName.Equals("Age", OrdinalIgnoreCase)` — loses legitimate matches like `CustomerAge`, `MinAge`. +- **(b)** Suffix: `propertyName.EndsWith("Age", OrdinalIgnoreCase)` — still matches `Language` (ends in `age`). +- **(c)** Word-boundary regex — accurate but adds a regex pass per property. +- **(d)** `EndsWith("Age")` + preceded-by-uppercase guard — `CustomerAge` ok (`r`→`A` capital transition), `Language` rejected (`u`→`a` lowercase transition). + +Similar risk applies to other substring-based rules in `Descriptions.cs` (`Email`, `Phone`, `Address`, `Url`, `Date`, `Version`, `CreatedAt`, `UpdatedAt`, `DeletedAt`) — audit them under the same word-boundary lens. + +### Known workaround +Explicit `[ToonDescription(Constraints = "...")]` on the affected property (replace mode — no placeholder). + +### Related TODO +None yet. + +## TOON-I-3: Property override duplicated on inheritance in `@types` + +**Severity:** Minor (schema correctness; LLM may misinterpret) · **Status:** Open · **Area:** Property enumeration — likely `AcToonSerializer.ToonSerializeTypeMetadata.cs` or the shared base in `Serializers/` root. + +### Description +When a derived class uses `override` on a property inherited from its base, both the derived and the base version appear in the `@types` schema — two entries for the same logical member, each with its own `business-logic` string. + +Observed in a 2026-04-24 Toon dump on `OrderItemPallet`: +``` +OrderItemPallet: + ... + MeasuringStatus: MeasuringStatus + business-logic: "get => IsAudited ? MeasuringStatus.Audited : base.MeasuringStatus" + constraints: "readonly, not-mapped" + ... + MeasuringStatus: MeasuringStatus + business-logic: "get => IsMeasured ? MeasuringStatus.Finnished : Id > 0 ? ..." + constraints: "readonly, not-mapped" +``` + +Only affected when a class uses `override` on a property. Sibling non-overriding classes (`ShippingItemPallet`, `StockTakingItemPallet`) emit the property once as expected. + +### Root cause (likely) +`Type.GetProperties(BindingFlags.Public | BindingFlags.Instance)` returns overridden properties twice — once per declaring type in the inheritance chain. The Toon property enumeration does not dedupe on `Name`, so both versions reach the writer. + +### Fix options +- **(a)** Filter in the metadata build step: group by `Name`, keep the most-derived override (lowest inheritance distance from the target type). +- **(b)** Walk the inheritance chain with `BindingFlags.DeclaredOnly` per level, collecting names into a set; skip if already seen. + +### Known workaround +None — the schema is wrong as-is. Downstream consumers must tolerate the duplicate entry. + +### Related TODO +None yet. + +## TOON-I-4: Property order in `@types` is not pure alphabetical + +**Severity:** Minor (readability / predictability) · **Status:** Open · **Area:** Property ordering — likely `AcToonSerializer.ToonSerializeTypeMetadata.cs`, inherited from the shared base in `Serializers/` root. + +### Description +Per-class property emission in `@types` is **hierarchy-aware derived→base**, with alphabetical order within each inheritance level. The inherited `Id` property therefore appears at the very end of every class definition, instead of its alphabetical slot between `H…` and `J…`. + +Observed in a 2026-04-24 Toon dump on `Customer` (simplified): +``` +Customer: + Active, AdminComment, ..., ZipPostalCode, ← own properties, alphabetical + Id ← inherited, placed LAST +``` + +And on `OrderItemPallet` (which has multi-level inheritance): +``` +OrderItemPallet: + AverageWeight, IsAudited, MeasuringStatus, ..., TrayQuantity, ← derived-class props + Created, CreatorId, ForeignKey, ..., TrayQuantity, ← base class props + Id ← base-base entity +``` + +### Expected behavior +**Pure alphabetical order across all properties** of the type, regardless of inheritance level. For Toon's LLM-readability goal, a predictable flat order matters more than hierarchy preservation. + +### Docs inconsistency +`Serializers/README.md` "Property ordering" section states: *"Hierarchy-aware (base→derived) then alphabetical. Ensures stable property indices across type versions."* Two mismatches with actual behavior: + +1. **Direction reversed** — real output is `derived→base`, not `base→derived` as the doc claims. +2. **Rationale irrelevant to Toon** — "stable property indices across type versions" is a wire-format concern for AcBinary. Toon is a descriptive schema format where property order is cosmetic, not semantic. + +### Fix options +- **(a)** Override the property ordering specifically in `ToonSerializeTypeMetadata` to plain `OrderBy(p => p.Name)` — Toon diverges from the Binary/JSON shared ordering. Recommended: AcBinary needs the hierarchy-aware order for wire stability; Toon does not, and can opt out. +- **(b)** Fix the direction in the shared base (`base→derived` as the docs claim) — partial fix, still not pure alphabetical. +- **(c)** Accept current behavior and update `Serializers/README.md` to match observed reality — reject the issue. Not recommended: the user expectation is pure alphabetical, and the docs-vs-code gap is real either way. + +### Known workaround +None — consumers must tolerate the current order. + +### Related TODO +None yet. Fix also requires reconciling `Serializers/README.md` with the chosen direction. + +## Issue entry template + +``` +## ISSUE-NN: Short title + +**Severity:** Trivial / Low / Minor / Major · **Status:** Open / Documented / Mitigated · **Area:** + +### Description +What breaks, and under what conditions. + +### Root cause +Why it happens (code location + design mismatch). + +### Known workaround +Steps a consumer can take until fixed. + +### Related TODO +`TOON_TODO.md#todo-NN` (if applicable). +``` diff --git a/AyCode.Core/docs/TOON/TOON_OPTIONS.md b/AyCode.Core/docs/TOON/TOON_OPTIONS.md new file mode 100644 index 0000000..d14f7cc --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_OPTIONS.md @@ -0,0 +1,139 @@ +# Toon — Options & Presets + +`AcToonSerializerOptions` controls every Toon output behavior. Definition: `Serializers/Toons/AcToonSerializerOptions.cs`. + +## Toon-specific options + +| Option | Type | Default | Purpose | +|---|---|---|---| +| `Mode` | `ToonSerializationMode` | `Full` | Which sections to emit. | +| `UseMeta` | `bool` | `true` | Include `@meta`/`@types` sections. Ignored when `Mode = DataOnly`. | +| `UseIndentation` | `bool` | `true` | Pretty-print with indentation. When `false`, `=` has no surrounding spaces. | +| `IndentString` | `string` | `" "` (2 spaces) | Indent unit when `UseIndentation = true`. | +| `UseInlineTypeHints` | `bool` | `false` | Emit `value ` hints in `@data` (schema already has type info). | +| `UseInlineComments` | `bool` | `false` | Emit property descriptions as inline comments. | +| `ShowCollectionCount` | `bool` | `true` | Emit ` (count: N) ` header on arrays/dictionaries. | +| `UseMultiLineStrings` | `bool` | `true` | Triple-quote format for long strings. | +| `MultiLineStringThreshold` | `int` | `80` | Character count above which a string becomes multi-line. | +| `UseEnhancedMetadata` | `bool` | `true` | Full per-property metadata (description + purpose + constraints + examples + navigation) vs compact single-line form. | +| `OmitDefaultValues` | `bool` | `true` | Skip properties whose value equals `default(T)` / `null` / `0` / `false`. | +| `WriteTypeNames` | `bool` | `true` | Emit the type name before `{` on `@data` objects. | +| `MaxExampleStringLength` | `int` | `50` | Truncate example values in `@types` beyond this length. | + +## Inherited from `AcSerializerOptions` + +| Option | Type | Notes | +|---|---|---| +| `MaxDepth` | `byte` | Recursion guard; when exceeded, `null` is emitted. | +| `ReferenceHandling` | `ReferenceHandlingMode` | `None` / `OnlyId` / `All` — required for circular or shared-reference graphs. | +| `SerializerType` | `AcSerializerType` | Fixed to `Toon`. | + +## `ToonSerializationMode` enum + +```csharp +public enum ToonSerializationMode : byte +{ + Full = 0, // @meta + @types + @data (default) + MetaOnly = 1, // @meta + @types only + DataOnly = 2, // @data only +} +``` + +## Presets + +Each preset is a fresh instance (static property with `new` expression — no shared state between retrievals). + +### `Default` + +``` +Mode = Full, UseMeta = true, UseIndentation = true, +OmitDefaultValues = true, WriteTypeNames = true +``` + +First-time serialization with full context for the LLM. + +### `MetaOnly` + +``` +Mode = MetaOnly, UseMeta = true, UseIndentation = true, +UseInlineComments = true +``` + +Send schema once at conversation start. `UseInlineComments = true` adds clarity since no `@data` follows. + +### `DataOnly` + +``` +Mode = DataOnly, UseMeta = false, UseIndentation = true, +OmitDefaultValues = true, WriteTypeNames = true +``` + +Subsequent sends once the LLM already has the schema. + +### `Compact` + +``` +Mode = DataOnly, UseMeta = false, UseIndentation = true, +OmitDefaultValues = true, WriteTypeNames = false, +ReferenceHandling = None +``` + +Data-only, no type names, no reference tracking. Note: `UseIndentation = true` here — truly token-minimal output requires manually setting `UseIndentation = false` on a custom options instance. + +### `Verbose` + +``` +Mode = Full, UseMeta = true, UseIndentation = true, +UseInlineTypeHints = true, UseInlineComments = true, +OmitDefaultValues = false, WriteTypeNames = true +``` + +Debugging / documentation — every hint turned on, defaults included. Bloats tokens; avoid for production LLM contexts. + +## Factory helpers + +```csharp +AcToonSerializerOptions.WithMaxDepth(10) +AcToonSerializerOptions.WithoutReferenceHandling() +``` + +Short-hand for common one-off customizations. + +## Custom options + +All properties are `init;` — assemble once per serialization call: + +```csharp +var opts = new AcToonSerializerOptions +{ + Mode = ToonSerializationMode.Full, + UseIndentation = false, // tight token-minimal output + ShowCollectionCount = true, + UseMultiLineStrings = false, // keep long strings inline + OmitDefaultValues = true, + MaxDepth = 20 +}; +string toon = AcToonSerializer.Serialize(value, opts); +``` + +## Public API overloads + +| Method | Purpose | +|---|---| +| `Serialize(T value)` | Default options. | +| `Serialize(T value, AcToonSerializerOptions)` | Custom options. | +| `Serialize(T value, string domainDescription, AcToonSerializerOptions)` | Adds `context = "..."` line to `@meta`. | +| `SerializeTypeMetadata()` / `SerializeTypeMetadata(Type)` | Schema only — uses `MetaOnly` internally. | +| `SerializeMetadata(IEnumerable)` | Schema for multiple types (no instance required). | +| `SerializeMetadata(params Type[])` | Schema for multiple types (params syntax). | +| `SerializeMetadata(string domainDescription, params Type[])` | Multi-type schema with domain context in `@meta`. | + +## Picking a preset / customization + +| Scenario | Choice | +|---|---| +| First prompt to an LLM in a new conversation | `Default` | +| Send schema once, many data round-trips | `MetaOnly` then `DataOnly` | +| Token-minimal streaming after schema is established | Custom: `Mode = DataOnly` + `UseIndentation = false` + `WriteTypeNames = false` | +| Debugging, local inspection | `Verbose` | +| Cyclic object graph (entity with backrefs) | Any preset, but ensure `ReferenceHandling = OnlyId` or `All` on a custom options instance | diff --git a/AyCode.Core/docs/TOON/TOON_TODO.md b/AyCode.Core/docs/TOON/TOON_TODO.md new file mode 100644 index 0000000..14b0d91 --- /dev/null +++ b/AyCode.Core/docs/TOON/TOON_TODO.md @@ -0,0 +1,131 @@ +# Toon — TODO + +## Priority legend +- **P0** blocker · **P1** important · **P2** nice-to-have · **P3** idea + +--- + +## TOON-T-1: Tabular array mode (CSV-style headered lists) +**Priority:** P2 · **Type:** Feature (opt-in) · **Origin:** 2026-04-24 LLM-accuracy proposal + +Emit arrays of homogeneous flat objects with a single column header instead of repeating property names on every element: + +``` +OrderItems: OrderItem[] = [ + [Id, Quantity, ProductName] + [120, 10, "Blueberries"] + [121, 5, "Oranges"] +] +``` + +Activation conditions (all must hold): +- Every element's runtime type equals the declared element type (no polymorphism). +- All element properties are primitives/strings (no nested objects). +- `OmitDefaultValues = false` (otherwise empty columns are ambiguous), or a separate empty-cell encoding chosen. + +When any condition fails → fall back to the current named syntax. + +Rationale: the CSV pattern is heavily represented in LLM training data; a single header avoids per-row key repetition. Estimated 30-50% token savings on arrays with 50+ rows. Must remain OFF in `Verbose` preset for debug clarity. + +## TOON-T-2: Schema-derived type elision in `@data` +**Priority:** P2 · **Type:** Feature · **Origin:** 2026-04-24 LLM-accuracy proposal + +Omit type names on `@data` list elements when: +1. The containing property's declared type is a concrete class (not interface/abstract). +2. The collection is non-polymorphic at runtime. +3. The element is not the `@data` root. + +``` +// Currently +GenericAttributes = [ + GenericAttributeDto { Id = 99, Key = "NetWeight" } +] + +// Proposed (when conditions hold) +GenericAttributes = [ + { Id = 99, Key = "NetWeight" } +] +``` + +Must stay explicit in `Verbose` mode and in polymorphic collections (type name is the discriminator). The main design tension: this slightly erodes the "explicit is better" principle — schema drift in a long conversation could cause silent misinterpretation. Pair with TOON-T-5 to mitigate. + +## TOON-T-3: Schema-declared default values +**Priority:** P2 · **Type:** Feature · **Origin:** 2026-04-24 LLM-accuracy proposal + +Currently `OmitDefaultValues = true` removes fields equal to `default(T)`, but the LLM must guess what the default is. Extend `@types` with explicit per-property defaults so absence becomes unambiguous: + +``` +@types { + Product: + Stock: int32 + description: "Available inventory" + default-value: 0 + constraints: "required, non-negative" +} +``` + +Implementation: during `@types` emission, detect the property's `default(T)` value and emit `default-value: ...` when `OmitDefaultValues = true`. `@data` omission then carries documented semantics ("use the schema default"), not implicit guessing. + +Deserializer impact: N/A — no deserializer yet (see TOON-T-6). + +## TOON-T-4: Reference-handling preset audit +**Priority:** P3 · **Type:** Configuration review · **Origin:** 2026-04-24 LLM-accuracy proposal + +The `@N` / `@ref:N` syntax exists and works (see `TOON_FORMAT.md#reference-syntax`). Audit whether the presets activate it aggressively enough: + +- `Default` preset inherits `ReferenceHandling` from the base `AcSerializerOptions` default — verify the default is `OnlyId` or `All`, not `None`, for IId-bearing entity graphs. +- `Compact` explicitly sets `None` — intentional (token-minimal) but may duplicate shared objects. +- Document explicitly which preset to choose when the object graph has repeated entities. + +Deliverable: a "When to choose which ReferenceHandling mode" section in `TOON_OPTIONS.md` with concrete examples. + +## TOON-T-5: Schema hash in `@meta` +**Priority:** P3 · **Type:** Feature (defense-in-depth for TOON-T-2, TOON-T-3) · **Origin:** 2026-04-24 + +After TOON-T-2 and TOON-T-3 ship, `@data` payloads rely more heavily on a specific `@types` schema. In multi-turn conversations, the LLM's cached schema may drift out of sync with the sender's. + +Emit a compact schema fingerprint in `@meta`: + +``` +@meta { + version = "1.0" + schema-hash = "a3f9b2" + types = [...] +} +``` + +The prompt template instructs the LLM: *"only use cached schema when the `schema-hash` matches; otherwise request the full schema again."* Cost: ~20-30 tokens per payload. Benefit: silent schema drift becomes detectable. + +## TOON-T-6: Deserialization support +**Priority:** P3 · **Type:** Feature (major) · **Origin:** 2026-04-24 during docs migration + +Toon is currently **serialize-only** — there is no `AcToonDeserializer`. An LLM can produce Toon-formatted responses, but the framework cannot parse them back into C# objects. + +Decision needed before implementation: +- Is Toon deserialization in-scope? (Primary goal is LLM accuracy — deserialization is mostly useful for LLM-generated outputs.) +- Or is `AcBinary`/`AcJson` the intended deserialization path, with Toon strictly a presentation format for LLM inputs? + +If in-scope, the parser must handle: `@meta`/`@types`/`@data` sections, reference resolution (`@N` / `@ref:N`), multi-line strings, numeric enum values (requires `@types` for name mapping), tabular arrays (if TOON-T-1 lands), polymorphic collections. + +## TOON-T-7: Validate benchmark claims before re-adding to docs +**Priority:** P3 · **Type:** Doc correctness · **Origin:** 2026-04-24 during `ToonExtendedInfo.txt` removal + +The original `ToonExtendedInfo.txt` (deleted during migration) claimed: +- First-time Full mode: ~85% of JSON speed. +- Subsequent DataOnly: ~95% of JSON speed. +- With attributes: ~90% of JSON speed. + +No benchmark harness was found in `AyCode.Core.Tests` or `AyCode.Benchmark` backing these numbers. Either: +- (a) Run real benchmarks and replace with measured values in `TOON_OPTIONS.md`. +- (b) Drop the claims entirely — marketing filler without provenance is worse than no claim. + +## TODO entry template + +``` +## TODO-NN: Short title +**Priority:** P0 / P1 / P2 / P3 · **Type:** Bug fix / Feature / Cleanup / Docs · **Related:** `TOON_ISSUES.md#issue-NN` (if applicable) + +Description of what and why, including the trigger (user request, audit finding, incident). + +Options / sub-tasks / acceptance criteria. +``` diff --git a/AyCode.Core/docs/XCUT/README.md b/AyCode.Core/docs/XCUT/README.md new file mode 100644 index 0000000..df7ae56 --- /dev/null +++ b/AyCode.Core/docs/XCUT/README.md @@ -0,0 +1,42 @@ +# XCUT — Cross-cutting issues and TODOs + +Canonical home for issues, TODOs, bugs, and critical items that span **two or more topics** (e.g., BINARY + SIGNALR, LOGGING + SIGNALR, BINARY + TOON). + +## When to use `XCUT-*-N` vs a topic-specific ID + +- **Single topic** (concern, bug, or planned work inside one domain) → `{TOPIC}-{TYPE}-{N}` in that topic's `{TOPIC}_ISSUES.md` / `{TOPIC}_TODO.md` (e.g., `LOG-I-5` in `LOGGING_ISSUES.md`). +- **Two or more topics** (coordinated change needed across domains, or the issue's root cause spans multiple subsystems) → `XCUT-{TYPE}-{N}` here, with cross-references from each affected topic's file. + +## Examples of cross-cutting items + +- **`XCUT-I-1: JSON-in-Binary request parameters`** — affects both the BINARY serializer (wire format) and SIGNALR transport (envelope). Neither side can fix it alone; it needs a coordinated client+server+consumer migration. +- *(hypothetical)* `XCUT-T-N` for a shared infrastructure refactor that touches Logger, SignalR, and SourceGenerator simultaneously. +- *(hypothetical)* `XCUT-B-N` for a bug where the symptom appears in one topic but the root cause lives in another. + +## Files in this folder + +- [`XCUT_ISSUES.md`](XCUT_ISSUES.md) — cross-cutting known issues (canonical entries) +- [`XCUT_TODO.md`](XCUT_TODO.md) — cross-cutting planned work (canonical entries) + +## Cross-reference convention + +1. **Canonical entry** lives here. Full body: status, affects (list of topics), rationale, migration plan. +2. **Each affected topic** adds a short `### XCUT-{TYPE}-{N}: — cross-ref` section in its own `_ISSUES.md` / `_TODO.md`, pointing back to this canonical entry. +3. The **topic-specific cross-ref** body is one sentence: "See canonical entry at `../XCUT/XCUT_ISSUES.md#xcut-{type}-{n}`" + a 1-2 line summary of what this topic contributes. + +## How `XCUT-*-N` IDs are numbered + +Per the `TOPIC_CODES.md` registry: +- Counter is `XCUT` + type (`I` / `T` / `B` / `C`). +- `XCUT-I-1`, `XCUT-I-2`, ... share one counter across all cross-cutting issues regardless of which topics they span. +- `XCUT-T-1`, `XCUT-T-2`, ... share another counter for cross-cutting TODOs. +- Append-only — once assigned, IDs never change. + +## Historical note + +Before this folder existed, cross-cutting entries were duplicated in each affected topic's file (e.g., `XCUT-1` appeared both in `BINARY_ISSUES.md` and `SIGNALR_ISSUES.md`). The canonical-home pattern here replaces that — one authoritative entry, short cross-refs from each affected topic. + +## See also + +- **Registry** (topic codes, type codes, ID format rules): `../../../.github/skills/docs-check/references/TOPIC_CODES.md` +- **Decision Log** (why this folder exists): `../../../.github/LLM_PROTOCOL_DECISIONS.md` — entry "XCUT consolidation to dedicated folder". diff --git a/AyCode.Core/docs/XCUT/XCUT_ISSUES.md b/AyCode.Core/docs/XCUT/XCUT_ISSUES.md new file mode 100644 index 0000000..4644671 --- /dev/null +++ b/AyCode.Core/docs/XCUT/XCUT_ISSUES.md @@ -0,0 +1,45 @@ +# XCUT — Cross-cutting Known Issues + +Canonical entries for known issues that span two or more topics. See `README.md` in this folder for the cross-cutting convention. + +For planned cross-cutting work, see `XCUT_TODO.md`. + +--- + +## XCUT-I-1: JSON-in-Binary request parameters + +**Status:** Major tech debt, planned replacement (coordinated) +**Affects:** BINARY serializer (wire format) ↔ SIGNALR transport (envelope) ↔ all consuming projects (caller code) + +### Description + +Client→server request parameters currently travel as JSON inside a Binary envelope: +- `SignalPostJsonDataMessage<T>` is the current envelope +- Response path already uses pure Binary (no JSON) +- Asymmetry: request is JSON-in-Binary, response is Binary + +### Why it spans multiple topics + +- **BINARY side**: the serializer has to special-case JSON payloads inside a Binary stream; ideal Binary-only flow is broken on the request side. +- **SIGNALR side**: the transport carries the JSON-wrapped Binary payload; `SignalPostJsonDataMessage<T>` is a SignalR-level concept. +- **Consumer side**: every caller sends requests via this asymmetric path; changing the wire format requires coordinated client + server + all-consuming-project updates. + +### Planned migration (tracked in TODO) + +- `BINARY_TODO.md#bin-t-1` — "Replace JSON-in-Binary request parameters" +- Acceptance: `SignalPostJsonDataMessage<T>` replaced by `SignalPostBinaryDataMessage<T>` (or equivalent); no JSON round-trip on the wire for request params; benchmarks confirm no regression. + +### Do NOT attempt as a side-effect + +Any client or server change in isolation will break the other side. Requires: +1. Binary envelope for both request and response defined +2. Client code updated to serialize via Binary +3. Server code updated to deserialize via Binary +4. All consuming projects rebuilt against new API +5. Version bump coordinated + +### Cross-references + +- **BINARY_ISSUES.md** (`../BINARY/BINARY_ISSUES.md#xcut-i-1`): cross-ref pointing here +- **SIGNALR_ISSUES.md** (`../../../AyCode.Services/docs/SIGNALR/SIGNALR_ISSUES.md#xcut-i-1`): cross-ref pointing here +- **BINARY_TODO.md#bin-t-1**: migration plan diff --git a/AyCode.Core/docs/XCUT/XCUT_TODO.md b/AyCode.Core/docs/XCUT/XCUT_TODO.md new file mode 100644 index 0000000..ab10055 --- /dev/null +++ b/AyCode.Core/docs/XCUT/XCUT_TODO.md @@ -0,0 +1,37 @@ +# XCUT — Cross-cutting TODO + +Canonical entries for planned work that spans two or more topics. See `README.md` in this folder for the cross-cutting convention. + +For known cross-cutting issues, see `XCUT_ISSUES.md`. + +## Priority legend +- **P0** blocker · **P1** important · **P2** nice-to-have · **P3** idea + +--- + +*(No cross-cutting TODO entries yet. When adding one, use the `XCUT-T-N` format and follow the canonical + per-topic cross-ref pattern documented in `README.md`.)* + +## Entry template + +``` +## XCUT-T-N: Short title +**Priority:** P1/P2/P3 · **Type:** Refactor/Feature/etc. · **Related:** `<topic>_TODO.md#<id>`, `<topic>_ISSUES.md#<id>` + +**Affects:** <topic-1> ↔ <topic-2> [ ↔ consumer projects ] + +### Why it spans multiple topics + +<1-2 sentences per topic explaining its role> + +### Migration plan + +<ordered steps> + +### Acceptance criteria + +<bullet list> + +### Do NOT attempt as a side-effect + +<why it requires coordination> +``` diff --git a/AyCode.Database/AyCode.Database.csproj b/AyCode.Database/AyCode.Database.csproj index 36c9210..23e13b9 100644 --- a/AyCode.Database/AyCode.Database.csproj +++ b/AyCode.Database/AyCode.Database.csproj @@ -30,5 +30,8 @@ <Folder Include="DbSets\Loggers\" /> <Folder Include="SqlScripts\" /> </ItemGroup> + <ItemGroup> + <None Include="**\README.md" Exclude="$(DefaultItemExcludes)" /> + </ItemGroup> </Project> diff --git a/AyCode.Database/DbContexts/Loggers/README.md b/AyCode.Database/DbContexts/Loggers/README.md index 4590f51..7530343 100644 --- a/AyCode.Database/DbContexts/Loggers/README.md +++ b/AyCode.Database/DbContexts/Loggers/README.md @@ -2,7 +2,7 @@ Logger-specific EF Core DbContext with `NoTracking` query behavior for read performance. Used by `AcDbLogItemWriter` to persist log items. -> For full logging architecture see `docs/LOGGING.md`. +> For full logging architecture see `docs/LOGGING/README.md`. ## Key Files diff --git a/AyCode.Database/DbSets/Loggers/README.md b/AyCode.Database/DbSets/Loggers/README.md index 709c1ac..d4e982d 100644 --- a/AyCode.Database/DbSets/Loggers/README.md +++ b/AyCode.Database/DbSets/Loggers/README.md @@ -2,7 +2,7 @@ Log item DbSet interface for EF Core log storage. -> For full logging architecture see `docs/LOGGING.md`. +> For full logging architecture see `docs/LOGGING/README.md`. ## Key Files diff --git a/AyCode.Entities.Server/AyCode.Entities.Server.csproj b/AyCode.Entities.Server/AyCode.Entities.Server.csproj index f070b7f..7d574be 100644 --- a/AyCode.Entities.Server/AyCode.Entities.Server.csproj +++ b/AyCode.Entities.Server/AyCode.Entities.Server.csproj @@ -1,4 +1,4 @@ -<Project Sdk="Microsoft.NET.Sdk"> +<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> </PropertyGroup> @@ -14,5 +14,8 @@ <ProjectReference Include="..\AyCode.Entities\AyCode.Entities.csproj" /> <ProjectReference Include="..\AyCode.Utils\AyCode.Utils.csproj" /> </ItemGroup> + <ItemGroup> + <None Include="**\README.md" Exclude="$(DefaultItemExcludes)" /> + </ItemGroup> </Project> diff --git a/AyCode.Entities.Server/LogItems/README.md b/AyCode.Entities.Server/LogItems/README.md index f54efc8..e08b88f 100644 --- a/AyCode.Entities.Server/LogItems/README.md +++ b/AyCode.Entities.Server/LogItems/README.md @@ -2,7 +2,7 @@ Server-side log item entity and interface, extending the client-side `AcLogItemClient` with database-mapped identity and header reference. -> For full logging architecture see `docs/LOGGING.md`. For client-side entity see `AyCode.Entities/LogItems/README.md`. +> For full logging architecture see `docs/LOGGING/README.md`. For client-side entity see `AyCode.Entities/LogItems/README.md`. ## Key Files diff --git a/AyCode.Entities/AyCode.Entities.csproj b/AyCode.Entities/AyCode.Entities.csproj index 4b14bd8..c88bbed 100644 --- a/AyCode.Entities/AyCode.Entities.csproj +++ b/AyCode.Entities/AyCode.Entities.csproj @@ -1,4 +1,4 @@ -<Project Sdk="Microsoft.NET.Sdk"> +<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> </PropertyGroup> @@ -16,5 +16,8 @@ <ItemGroup> <PackageReference Include="MessagePack.Annotations" Version="3.1.4" /> </ItemGroup> + <ItemGroup> + <None Include="**\README.md" Exclude="$(DefaultItemExcludes)" /> + </ItemGroup> </Project> diff --git a/AyCode.Entities/LogItems/README.md b/AyCode.Entities/LogItems/README.md index 0cad923..571d359 100644 --- a/AyCode.Entities/LogItems/README.md +++ b/AyCode.Entities/LogItems/README.md @@ -2,7 +2,7 @@ Client-side log item entity used for structured logging. Serialized with MessagePack for efficient transport over SignalR. -> For full logging architecture see `docs/LOGGING.md`. +> For full logging architecture see `docs/LOGGING/README.md`. ## Key Files diff --git a/AyCode.Models.Server/AyCode.Models.Server.csproj b/AyCode.Models.Server/AyCode.Models.Server.csproj index f9dd97d..cf12d19 100644 --- a/AyCode.Models.Server/AyCode.Models.Server.csproj +++ b/AyCode.Models.Server/AyCode.Models.Server.csproj @@ -1,4 +1,4 @@ -<Project Sdk="Microsoft.NET.Sdk"> +<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> </PropertyGroup> @@ -19,5 +19,8 @@ <ProjectReference Include="..\AyCode.Models\AyCode.Models.csproj" /> <ProjectReference Include="..\AyCode.Services\AyCode.Services.csproj" /> </ItemGroup> + <ItemGroup> + <None Include="**\README.md" Exclude="$(DefaultItemExcludes)" /> + </ItemGroup> </Project> diff --git a/AyCode.Models.Server/DynamicMethods/README.md b/AyCode.Models.Server/DynamicMethods/README.md index 41be223..45f4706 100644 --- a/AyCode.Models.Server/DynamicMethods/README.md +++ b/AyCode.Models.Server/DynamicMethods/README.md @@ -3,7 +3,7 @@ Reflection-based infrastructure for dynamically dispatching method calls by message tag, primarily used for SignalR message routing. > **Context:** This is the server-side dispatch engine for the SignalR tag-based architecture. -> See `AyCode.Services.Server/docs/SIGNALR_SERVER.md` for the full message flow. +> See `AyCode.Services.Server/docs/SIGNALR/README.md` for the full message flow. ## How It Fits diff --git a/AyCode.Services.Server/AyCode.Services.Server.csproj b/AyCode.Services.Server/AyCode.Services.Server.csproj index 4f96509..3842180 100644 --- a/AyCode.Services.Server/AyCode.Services.Server.csproj +++ b/AyCode.Services.Server/AyCode.Services.Server.csproj @@ -1,4 +1,4 @@ - <Project Sdk="Microsoft.NET.Sdk"> + <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> </PropertyGroup> @@ -35,6 +35,7 @@ <ItemGroup> <None Include="docs\**\*.md" /> + <None Include="**\README.md" Exclude="$(DefaultItemExcludes);docs\**" /> </ItemGroup> </Project> diff --git a/AyCode.Services.Server/README.md b/AyCode.Services.Server/README.md index def4ff0..d590eab 100644 --- a/AyCode.Services.Server/README.md +++ b/AyCode.Services.Server/README.md @@ -10,8 +10,8 @@ Server-side service implementations: JWT authentication, SendGrid email delivery | Document | Topic | |---|---| -| `SIGNALR_SERVER.md` | Server-side SignalR hub (dispatch, session, broadcast) | -| `SIGNALR_DATASOURCE.md` | Real-time DataSource with CRUD & change tracking | +| `SIGNALR/README.md` | Server-side SignalR hub (dispatch, session, broadcast) | +| `SIGNALR/SIGNALR_DATASOURCE.md` | Real-time DataSource with CRUD & change tracking | ## Folder Structure diff --git a/AyCode.Services.Server/SignalRs/README.md b/AyCode.Services.Server/SignalRs/README.md index dc558a6..b83dbe1 100644 --- a/AyCode.Services.Server/SignalRs/README.md +++ b/AyCode.Services.Server/SignalRs/README.md @@ -2,7 +2,7 @@ Server-side SignalR hub infrastructure: hub base class, session management, data source with change tracking, and client broadcast service. -> **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR.md`. +> **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR/README.md`. ## Key Files @@ -18,7 +18,7 @@ Server-side SignalR hub infrastructure: hub base class, session management, data ### Data Source -> **Full specification:** `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md` +> **Full specification:** `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md` - **`AcSignalRDataSource.cs`** — Generic real-time collection (`AcSignalRDataSource<TDataItem, TId, TIList>`) implementing `IList<T>` with full CRUD and change tracking. - **Change tracking:** `TrackingItem<T, TId>` wraps each modified item with `TrackingState` + `OriginalValue` for rollback. `ChangeTracking<T, TId>` manages the tracking list. diff --git a/AyCode.Services.Server/docs/README.md b/AyCode.Services.Server/docs/README.md new file mode 100644 index 0000000..52e3bf4 --- /dev/null +++ b/AyCode.Services.Server/docs/README.md @@ -0,0 +1,16 @@ +# AyCode.Services.Server documentation + +Topic documentation for the `AyCode.Services.Server` project (Layer 0, server-side services). + +## Topics + +- [`SIGNALR/`](SIGNALR/README.md) — Server-side SignalR (hub base + data source pattern) + +## Navigation + +Per the AI Agent Core Protocol (folder navigation rule), start from this README when browsing `docs/`. Each topic folder has its own `README.md` with the main content, plus optional `TOPIC_ISSUES.md` (known issues) and `TOPIC_TODO.md` (planned work). + +## See also + +- **Client-side SignalR**: `../../AyCode.Services/docs/SIGNALR/README.md` +- **Binary-over-SignalR wire format**: `../../AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md` diff --git a/AyCode.Services.Server/docs/SIGNALR_SERVER.md b/AyCode.Services.Server/docs/SIGNALR/README.md similarity index 99% rename from AyCode.Services.Server/docs/SIGNALR_SERVER.md rename to AyCode.Services.Server/docs/SIGNALR/README.md index 78ac41d..020309c 100644 --- a/AyCode.Services.Server/docs/SIGNALR_SERVER.md +++ b/AyCode.Services.Server/docs/SIGNALR/README.md @@ -2,7 +2,7 @@ Server-side SignalR hub infrastructure: method dispatch, session management, broadcast, and diagnostics. Source: `SignalRs/` in this project. -> For client-side transport (tags, wire protocol, client base) see `AyCode.Services/docs/SIGNALR.md`. +> For client-side transport (tags, wire protocol, client base) see `AyCode.Services/docs/SIGNALR/README.md`. > For the DataSource collection see `SIGNALR_DATASOURCE.md`. ## Server Processing diff --git a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md b/AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md similarity index 97% rename from AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md rename to AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md index 80a3d50..c6eb289 100644 --- a/AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md +++ b/AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md @@ -2,8 +2,8 @@ Change-tracked real-time collection built on top of the SignalR transport layer. Source: `SignalRs/AcSignalRDataSource.cs` in this project. -> For the underlying transport (tag system, wire protocol, client base) see `AyCode.Services/docs/SIGNALR.md`. -> For server hub infrastructure see `SIGNALR_SERVER.md`. +> For the underlying transport (tag system, wire protocol, client base) see `AyCode.Services/docs/SIGNALR/README.md`. +> For server hub infrastructure see `README.md`. ## Overview @@ -179,7 +179,7 @@ DataSource.SaveChanges() → Server method with [SignalR(tag)] ← tag dispatch ``` -Projects can also call the transport directly without DataSource — see `AyCode.Services/docs/SIGNALR.md`. +Projects can also call the transport directly without DataSource — see `AyCode.Services/docs/SIGNALR/README.md`. ## Key Source Files diff --git a/AyCode.Services/AyCode.Services.csproj b/AyCode.Services/AyCode.Services.csproj index d49c7e4..84c0277 100644 --- a/AyCode.Services/AyCode.Services.csproj +++ b/AyCode.Services/AyCode.Services.csproj @@ -1,4 +1,4 @@ -<Project Sdk="Microsoft.NET.Sdk"> +<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> </PropertyGroup> @@ -20,6 +20,7 @@ <ItemGroup> <None Include="docs\**\*.md" /> + <None Include="**\README.md" Exclude="$(DefaultItemExcludes);docs\**" /> </ItemGroup> </Project> diff --git a/AyCode.Services/Loggers/README.md b/AyCode.Services/Loggers/README.md index dac0b38..6669a80 100644 --- a/AyCode.Services/Loggers/README.md +++ b/AyCode.Services/Loggers/README.md @@ -2,7 +2,7 @@ Remote log writers for sending log items over HTTP, SignalR, or to the browser console. All three implement `IAcLogWriterClientBase`. -> For full logging architecture see `docs/LOGGING.md`. For core logger and writer abstractions see `AyCode.Core/Loggers/README.md`. +> For full logging architecture see `docs/LOGGING/README.md`. For core logger and writer abstractions see `AyCode.Core/Loggers/README.md`. ## Key Files diff --git a/AyCode.Services/README.md b/AyCode.Services/README.md index 34dcc22..4314eff 100644 --- a/AyCode.Services/README.md +++ b/AyCode.Services/README.md @@ -10,8 +10,9 @@ Shared service implementations: SignalR communication (custom binary protocol), | Document | Topic | |---|---| -| `SIGNALR.md` | Client-side SignalR transport (tags, wire protocol, req/resp flow) | -| `LOGGING_REMOTE.md` | Remote log writers (HTTP, browser console, SignalR) | +| `SIGNALR/README.md` | Client-side SignalR transport (tags, wire protocol, req/resp flow) | +| `SIGNALR_BINARY_PROTOCOL/README.md` | Binary-over-SignalR wire format, chunked framing | +| `LOGGING/README.md` | Remote log writers (HTTP, browser console, SignalR) | ## Folder Structure diff --git a/AyCode.Services/SignalRs/README.md b/AyCode.Services/SignalRs/README.md index 39d8c75..c4c6b67 100644 --- a/AyCode.Services/SignalRs/README.md +++ b/AyCode.Services/SignalRs/README.md @@ -2,9 +2,9 @@ Custom binary SignalR protocol, client infrastructure, message tagging, and serialization helpers. -> **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR.md`. -> **Binary protocol:** For wire format, zero-copy pipeline, and three-path read logic see `AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md`. -> **Known issues:** `AyCode.Services/docs/SIGNALR_ISSUES.md` +> **Architecture:** For full dispatch flow, tag system, and tech debt documentation see `AyCode.Services/docs/SIGNALR/README.md`. +> **Binary protocol:** For wire format, zero-copy pipeline, and three-path read logic see `AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md`. +> **Known issues:** `AyCode.Services/docs/SIGNALR/SIGNALR_ISSUES.md` ## Key Files @@ -21,7 +21,7 @@ Custom binary SignalR protocol, client infrastructure, message tagging, and seri ### Message Tagging - **`SignalMessageTagAttribute.cs`** — Three attributes: `TagAttribute` (base, int messageTag), `SignalRAttribute` (server method routing + client notification), `SignalRSendToClientAttribute` (client-side receive). - **`AcSignalRTags.cs`** — Static constants: `None`, `PingTag`, `EchoTag`. -- **`SignalRCrudTags.cs`** — Sealed class bundling 5 independent CRUD tag integers. `GetMessageTagByTrackingState()` maps `TrackingState` -> tag. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. +- **`SignalRCrudTags.cs`** — Sealed class bundling 5 independent CRUD tag integers. `GetMessageTagByTrackingState()` maps `TrackingState` -> tag. See `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md`. - **`SendToClientType.cs`** — Enum: None, Others, Caller, All. ### Serialization & Pooling diff --git a/AyCode.Services/docs/LOGGING_REMOTE.md b/AyCode.Services/docs/LOGGING/README.md similarity index 95% rename from AyCode.Services/docs/LOGGING_REMOTE.md rename to AyCode.Services/docs/LOGGING/README.md index f7cad6f..0f92d54 100644 --- a/AyCode.Services/docs/LOGGING_REMOTE.md +++ b/AyCode.Services/docs/LOGGING/README.md @@ -1,6 +1,6 @@ # Remote Log Writers -Client-side log writers that send log data to remote endpoints. Source: `Loggers/` in this project. For core logging framework see `AyCode.Core/docs/LOGGING.md`. For server-side GlobalLogger see `AyCode.Core.Server/docs/LOGGING_SERVER.md`. +Client-side log writers that send log data to remote endpoints. Source: `Loggers/` in this project. For core logging framework see `AyCode.Core/AyCode.Core/docs/LOGGING/README.md`. For server-side GlobalLogger see `AyCode.Core.Server/docs/LOGGING/README.md`. ## AcBrowserConsoleLogWriter diff --git a/AyCode.Services/docs/README.md b/AyCode.Services/docs/README.md new file mode 100644 index 0000000..b245b71 --- /dev/null +++ b/AyCode.Services/docs/README.md @@ -0,0 +1,20 @@ +# AyCode.Services documentation + +Topic documentation for the `AyCode.Services` project (Layer 0, service abstractions). + +## Topics + +- [`LOGGING/`](LOGGING/README.md) — Remote logger (variant — sends log entries over the wire) +- [`SIGNALR/`](SIGNALR/README.md) — SignalR transport (tag-based protocol, generic hub methods) +- [`SIGNALR_BINARY_PROTOCOL/`](SIGNALR_BINARY_PROTOCOL/README.md) — Binary-over-SignalR wire format, chunked framing + +## Navigation + +Per the AI Agent Core Protocol (folder navigation rule), start from this README when browsing `docs/`. Each topic folder has its own `README.md` with the main content, plus optional `TOPIC_ISSUES.md` (known issues) and `TOPIC_TODO.md` (planned work). + +## See also + +- **Base logger** (framework): `../../AyCode.Core/AyCode.Core/docs/LOGGING/README.md` +- **Server-side logger** (variant): `../../AyCode.Core.Server/docs/LOGGING/README.md` +- **Server-side SignalR**: `../../AyCode.Services.Server/docs/SIGNALR/README.md` +- **Binary serializer** (used by SIGNALR_BINARY_PROTOCOL): `../../AyCode.Core/AyCode.Core/docs/BINARY/README.md` diff --git a/AyCode.Services/docs/SIGNALR.md b/AyCode.Services/docs/SIGNALR/README.md similarity index 97% rename from AyCode.Services/docs/SIGNALR.md rename to AyCode.Services/docs/SIGNALR/README.md index 4950517..25e069d 100644 --- a/AyCode.Services/docs/SIGNALR.md +++ b/AyCode.Services/docs/SIGNALR/README.md @@ -2,8 +2,8 @@ Client-side SignalR transport: custom binary protocol, tag-based dispatch. Source: `SignalRs/` -> Server-side hub, session, broadcast: `AyCode.Services.Server/docs/SIGNALR_SERVER.md` -> DataSource collection: `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md` +> Server-side hub, session, broadcast: `AyCode.Services.Server/docs/SIGNALR/README.md` +> DataSource collection: `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md` ## Design @@ -74,7 +74,7 @@ Custom `IHubProtocol` (`"acbinary"`), replaces JSON. Zero-copy write via `Buffer `AcBinaryHubProtocol` is the base (unsealed) — general binary framing only. `AyCodeBinaryHubProtocol` derives from it with consumer-specific logic: `SignalParams` capture (via `OnArgumentRead` hook), `IsRawBytesData` path, `SignalDataType` type resolution. Register `AyCodeBinaryHubProtocol` in both client and server. -> Wire format, argument framing, dual BWO pattern, length prefix patching: `SIGNALR_BINARY_PROTOCOL.md` +> Wire format, argument framing, dual BWO pattern, length prefix patching: `../SIGNALR_BINARY_PROTOCOL/README.md` ### SignalParams + Payload Separation @@ -152,7 +152,7 @@ GetParameterValues(ParameterInfo[]): Type-guided deserialization — each parameter is individually serialized/deserialized with its concrete type, avoiding the `object[]` → dictionary problem of untyped binary deserialization. -> Known concerns and limitations on parameter serialization (per-parameter overhead, AcBinary-only) are tracked in `SIGNALR_ISSUES.md` under `PROTO-2` and `PROTO-3`. +> Known concerns and limitations on parameter serialization (per-parameter overhead, AcBinary-only) are tracked in `SIGNALR_ISSUES.md` under `SIG-I-2` and `SIG-I-3`. ## Response Patterns diff --git a/AyCode.Services/docs/SIGNALR_ISSUES.md b/AyCode.Services/docs/SIGNALR/SIGNALR_ISSUES.md similarity index 58% rename from AyCode.Services/docs/SIGNALR_ISSUES.md rename to AyCode.Services/docs/SIGNALR/SIGNALR_ISSUES.md index 2094fd7..54fc071 100644 --- a/AyCode.Services/docs/SIGNALR_ISSUES.md +++ b/AyCode.Services/docs/SIGNALR/SIGNALR_ISSUES.md @@ -2,7 +2,7 @@ ## Protocol -### PROTO-1: Server-side IsRawBytesData pre-serialize +### SIG-I-1: Server-side IsRawBytesData pre-serialize **Status:** Planned removal **Affects:** `AcWebSignalRHubBase.SendMessageToClient` @@ -11,7 +11,7 @@ The server forwards the client's `IsRawBytesData` flag in the response `SignalPa **Plan:** Remove `IsRawBytesData` forwarding from server response path. The client should use `SignalDataType` for typed deserialization and explicit `byte[]` type for raw data. -### PROTO-2: Parameter serialization is per-parameter +### SIG-I-2: Parameter serialization is per-parameter **Status:** Known performance concern **Affects:** `SignalParams.SetParameterValues` / `GetParameterValues` @@ -20,7 +20,7 @@ Each parameter is individually serialized via `ToBinary()` / `BinaryTo(Type)` **Possible optimization:** Batch fast-path — single serialization context for all parameters. Benchmark first. -### PROTO-3: Parameter serialization is AcBinary only +### SIG-I-3: Parameter serialization is AcBinary only **Status:** Limitation **Affects:** `SignalParams.SetParameterValues` / `GetParameterValues` @@ -29,14 +29,14 @@ Uses `ToBinary()` / `BinaryTo()` exclusively. JSON parameter support would requi ## Transport -### TRANS-1: BufferWriterChunkSize defaults to 64KB for SignalR +### SIG-I-4: BufferWriterChunkSize defaults to 64KB for SignalR **Status:** DONE **Affects:** `AcBinaryHubProtocol` constructor, write path `BufferWriterChunkSize = 4096` set in `AcBinaryHubProtocol` constructor. Aligns with Kestrel slab size, reduces latency-to-first-byte. Non-SignalR paths keep 64KB default. -### TRANS-2: WebSocket buffer sizes are hardcoded +### SIG-I-5: WebSocket buffer sizes are hardcoded **Status:** Acceptable **Affects:** `AcSignalRClientBase` connection setup @@ -45,7 +45,7 @@ Transport max message size (30MB) and application buffer (30MB) are hardcoded. S ## DataSource -### DS-1: GetAll returns raw byte[] for populate/merge +### SIG-I-6: GetAll returns raw byte[] for populate/merge **Status:** By design **Affects:** `AcSignalRDataSource.LoadDataSourceAsync` @@ -54,9 +54,39 @@ The `GetAll` path uses `IsRawBytesData = true` to receive raw `byte[]` from the **Possible optimization:** Direct typed deserialization with merge support in the deserializer (PopulateMerge from `ReadOnlySequence<byte>`). Requires deserializer API changes. +## Server-side Setup & DI + +### SIG-I-7: Server-side NopCommerce plugin AcBinaryHubProtocolOptions not bound from appsettings + +**Status:** Open · **Severity:** Minor (works, but hardcoded + bypasses DI logger) · **Area:** Consumer adoption gap in `Nop.Plugin.Misc.AIPlugin/Infrastructure/PluginNopStartup.cs` + +The framework overload chain is complete on the server side — `AcSignalRProtocolExtensions.BuildProtocol` already resolves `IOptions<AcBinaryHubProtocolOptions>` from DI and falls back to defaults otherwise. The consumer plugin does NOT use it: + +Current `PluginNopStartup.ConfigureServices`: +```csharp +.AddAcBinaryProtocol(opts => +{ + opts.ProtocolMode = BinaryProtocolMode.AsyncSegment; // ← HARDCODED + opts.Logger = new Logger(nameof(AyCodeBinaryHubProtocol)); // ← BYPASSES ILogger<T> DI +}); +``` + +What's missing: +- No `services.Configure<AcBinaryHubProtocolOptions>(configuration.GetSection("AyCode:SignalR:Protocol"))` → `ProtocolMode`, `BufferSize`, `WaitForFlush`, `FlushTimeout` are all hardcoded / default. +- The `appsettings.json` has no `AyCode:SignalR` (or equivalent) section at all — so per-deploy tuning (e.g. increasing `FlushTimeout` for a satellite link, switching `ProtocolMode` for diagnostics) requires a code change + redeploy. +- Manual `new Logger(...)` sidesteps the DI `ILogger<AcBinaryHubProtocol>` auto-resolution that `BuildProtocol` provides → creates a parallel logger instance (see `../../../AyCode.Core/docs/LOGGING/LOGGING_ISSUES.md#log-i-8`). + +### Fix direction +See `SIGNALR_TODO.md#sig-t-5`. + +### Related +- `../../../AyCode.Core/docs/LOGGING/LOGGING_ISSUES.md#log-i-8` (sibling gap — same plugin, logger setup) +- `SIG-I-8` (client-side equivalent — `HubConnectionBuilder.Services` inner DI isolation forces a different workaround) +- Plugin doc drift: `Nop.Plugin.Misc.AIPlugin/docs/SIGNALR/README.md:22` documents `services.AddSingleton<IHubProtocol>(new AcBinaryHubProtocol())` — the actual code uses `.AddAcBinaryProtocol(opts => {...})`. Doc needs a rewrite. + ## Client-side Setup & DI -### CONN-1: HubConnectionBuilder inner DI isolation +### SIG-I-8: HubConnectionBuilder inner DI isolation **Status:** Workaround-in-place (dedicated options-passing overload) **Affects:** Consumer client setup in `Program.cs` (MAUI, WASM, ASP.NET Core server prerender) @@ -71,7 +101,7 @@ hubBuilder.AddAcBinaryProtocol(protocolOpts); ## Dispatch -### DISPATCH-1: First-call null response (observed) +### SIG-I-9: First-call null response (observed) **Status:** Open — not diagnosed **Affects:** `PostDataAsync<T>` awaiter / OnReceiveMessage → pending-request correlation @@ -87,10 +117,10 @@ Log timeline: Hypothesis (unverified): `PostDataAsync<T>` awaiter's null-mapping path misroutes the parsed result, or `requestId → Task<T>` correlation has a race on the first response of a fresh connection. Client auto-retry hides the user-visible impact. -**Related TODO:** `SIGNALR_TODO.md#todo-01` +**Related TODO:** `SIGNALR_TODO.md#sig-t-1` -## Cross-cutting (also tracked in serializer-side docs) +## Cross-cutting (canonical home: `AyCode.Core` repo's `docs/XCUT/`) -### XCUT-1: JSON-in-Binary request parameters — cross-ref +### XCUT-I-1: JSON-in-Binary request parameters — cross-ref -Same tech debt as `../../AyCode.Core/docs/BINARY_ISSUES.md#xcut-1`. Planned replacement: migrate client→server request parameters from JSON-in-Binary envelope to direct Binary serialization. Coordinated change across all consuming projects. +Canonical entry: **`../../../AyCode.Core/docs/XCUT/XCUT_ISSUES.md#xcut-i-1`**. Summary: SignalR transport carries `SignalPostJsonDataMessage<T>` (JSON inside a Binary envelope) for request params, while response path is pure Binary. Planned replacement is coordinated across BINARY serializer + SIGNALR transport + all consuming projects. diff --git a/AyCode.Services/docs/SIGNALR/SIGNALR_TODO.md b/AyCode.Services/docs/SIGNALR/SIGNALR_TODO.md new file mode 100644 index 0000000..3484d35 --- /dev/null +++ b/AyCode.Services/docs/SIGNALR/SIGNALR_TODO.md @@ -0,0 +1,75 @@ +# SignalR — TODO + +## Priority legend +- **P0** blocker · **P1** important · **P2** nice-to-have · **P3** idea + +--- + +## SIG-T-1: Diagnose first-call null in PostDataAsync<T> +**Priority:** P2 · **Type:** Investigation · **Related:** `SIGNALR_ISSUES.md#issue-02` + +Reproduce the `GetProductDtos_80`-style first-call null. Add trace logs to `PostDataAsync<T>` awaiter path and `OnReceiveMessage → pending request` dictionary lookup. Verify `requestId → Task<T>` correlation on the very first chunked response of a fresh connection. + +## SIG-T-2: Document asymmetric send/receive capability +**Priority:** P1 · **Type:** Docs + +Current behaviour: sender selects `BinaryProtocolMode` independently; receiver detects the wire format from the first byte (`CHUNK_START=200` → chunked path; else non-chunked). This means client and server can run DIFFERENT `ProtocolMode` settings independently — a core feature amplifying interoperability. + +Document in `../SIGNALR_BINARY_PROTOCOL/README.md` as a dedicated "Asymmetric send/receive contract" section. Key selling points: +- WASM client + AsyncSegment server → works (WASM downgrades send, receives chunked happily) +- Third-party client on NuGet can pick any mode → server doesn't care +- Gradual mode rollouts possible (no synchronized deploy) + +## SIG-T-3: Code-level guard for FlushTimeout < ClientTimeoutInterval +**Priority:** P2 · **Type:** Feature + +`AcBinaryHubProtocolOptions.Validate()` currently documents (in XML doc) that `FlushTimeout` should be less than the SignalR `HubOptions.ClientTimeoutInterval`, but there is no code-level check. Add validation at protocol registration time — if both options are resolvable from the DI scope, verify the constraint and emit a startup warning (or throw, pending decision). + +## SIG-T-4: `BinaryProtocolMode.Auto` — adaptive send-mode +**Priority:** P3 · **Type:** Feature · **Related:** `../SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_TODO.md#sbp-t-3` + +Design: on first received message, inspect first byte to determine peer's send format. On subsequent sends, match it (subject to local-platform constraints, e.g. WASM never actually sends AsyncSegment). Per-`HubConnection` state. Optional upfront handshake-extension negotiation as an alternative — see wire-level TODO. + +## SIG-T-5: Server-side NopCommerce plugin — expose `AcBinaryHubProtocolOptions` via appsettings +**Priority:** P2 · **Type:** Consumer refactor · **Related:** `SIGNALR_ISSUES.md#sig-i-7`, `../../../AyCode.Core/docs/LOGGING/LOGGING_TODO.md#log-t-11` + +Bind `AcBinaryHubProtocolOptions` from `appsettings.json` instead of hardcoding `ProtocolMode` and constructing a manual `Logger` instance in `PluginNopStartup.cs`. This sibling task is paired with LOGGING_TODO.md#log-t-11 (same plugin, logger-setup migration) — best landed in one commit. + +### Target diff +```csharp +// In PluginNopStartup.ConfigureServices, BEFORE AddSignalR(...): + +// 1. Bind Protocol options from appsettings.json +services.Configure<AcBinaryHubProtocolOptions>(configuration.GetSection("AyCode:SignalR:Protocol")); + +// 2. AddSignalR + AddAcBinaryProtocol — drop the inline Action<T> entirely +services.AddSignalR(hubOptions => { /* unchanged */ }) + .AddAcBinaryProtocol(); // ← BuildProtocol auto-resolves IOptions<T> + ILogger<T> from DI +``` + +### Appsettings.json addition (sibling to `AyCode:Logger`) +```json +{ + "AyCode": { + "Logger": { "AppType": "Server", "LogLevel": "Debug" }, + "SignalR": { + "Protocol": { + "ProtocolMode": "AsyncSegment", + "BufferSize": 4096, + "WaitForFlush": true, + "FlushTimeout": "00:00:10" + } + } + } +} +``` + +### Consequences / checklist +- [ ] `new Logger(...)` line removed from SignalR registration → server-side logger now goes through the DI factory (see LOGGING_TODO.md#log-t-11). +- [ ] Per-deploy tuning possible without recompile: switching `ProtocolMode` for diagnostics, extending `FlushTimeout` for slow links, adjusting `BufferSize` for different Kestrel slab sizes. +- [ ] `Name` stays at `"acbinary"` default — changing it would break wire-level compat with existing clients. +- [ ] `AcBinaryHubProtocolOptions.Validate()` still runs — invalid config (e.g. `ProtocolMode=AsyncSegment` on a WASM server, which is impossible here but hypothetically) throws at startup. +- [ ] Plugin doc correction: `Nop.Plugin.Misc.AIPlugin/docs/SIGNALR/README.md:22` — the legacy `services.AddSingleton<IHubProtocol>(new AcBinaryHubProtocol())` line must be replaced with the real registration. Cross-ref the new `AyCode:SignalR:Protocol` section here. + +### Why this belongs in AyCode.Services (framework layer) docs +The gap is consumer-level, but the canonical "server-side registration recipe" is a FRAMEWORK responsibility — LOGGING.md already shows it for the logger side. Adding a matching recipe to `../SIGNALR_BINARY_PROTOCOL/README.md#Registration in Program.cs → Server` would prevent the next consumer from making the same mistake. That doc update is part of this TODO's acceptance criteria. diff --git a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md similarity index 98% rename from AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md rename to AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md index 2c31636..16210df 100644 --- a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL.md +++ b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/README.md @@ -5,7 +5,7 @@ `AyCodeBinaryHubProtocol` (derived) — project-specific consumer logic: `SignalParams` capture via `OnArgumentRead`, `IsRawBytesData` path, `SignalDataType` type resolution in `ReadSingleArgument` override. > Architecture (tag system, dispatch, request/response): `SIGNALR.md` -> Output writers (cached chunk, buffer states, chunk sizing): `AyCode.Core/docs/BINARY_WRITERS.md` +> Output writers (cached chunk, buffer states, chunk sizing): `AyCode.Core/AyCode.Core/docs/BINARY/BINARY_WRITERS.md` ## Wire Format @@ -78,7 +78,7 @@ Protocol and serializer each create own `BufferWriterBinaryOutput` on the same ` **Cost:** one extra `GetMemory` per argument (nanoseconds). **Benefit:** zero-copy end-to-end, no intermediate `byte[]`, no wrapper class. -Why two BWOs: serializer writes must live on `BinarySerializationContext` (sealed class) for JIT optimization — context owns its own BWO. See `AyCode.Core/docs/BINARY_WRITERS.md` § "Why Writes Are on the Context". +Why two BWOs: serializer writes must live on `BinarySerializationContext` (sealed class) for JIT optimization — context owns its own BWO. See `AyCode.Core/AyCode.Core/docs/BINARY/BINARY_WRITERS.md` § "Why Writes Are on the Context". ### Length Prefix Patching @@ -148,7 +148,7 @@ The context's `_buffer` always points directly to the current segment's backing Typical overhead for 225KB payload with 4096-byte segments: ~224.5KB zero-copy, ~500 bytes scratch copy at ~55 boundaries. The scratch buffer is rented once (lazy, on first boundary) and reused across all boundaries. `Release()` returns it to `ArrayPool` after deserialization. -> Known issues: `AyCode.Core/docs/BINARY_ISSUES.md` +> Known issues: `AyCode.Core/AyCode.Core/docs/BINARY/BINARY_ISSUES.md` ## Configuration diff --git a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL_ISSUES.md b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md similarity index 90% rename from AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL_ISSUES.md rename to AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md index fbf306a..578ebf1 100644 --- a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL_ISSUES.md +++ b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_ISSUES.md @@ -1,9 +1,9 @@ # Binary Hub Protocol (wire) — Known Issues For planned/actionable work see `SIGNALR_BINARY_PROTOCOL_TODO.md`. -For higher-level SignalR abstractions see `SIGNALR_ISSUES.md`. +For higher-level SignalR abstractions see `../SIGNALR/SIGNALR_ISSUES.md`. -## ISSUE-01: AsyncSegment send-path unsupported on WebAssembly +## SBP-I-1: AsyncSegment send-path unsupported on WebAssembly **Severity:** Major (on WASM) · **Status:** Workaround-in-place · **Area:** `AsyncPipeWriterOutput` / WASM runtime @@ -18,7 +18,7 @@ For higher-level SignalR abstractions see `SIGNALR_ISSUES.md`. ### Related TODO None — architectural constraint of browser WASM threading model. -## ISSUE-02: StaticWebAssets SDK "Illegal characters" noise (consumer build) +## SBP-I-2: StaticWebAssets SDK "Illegal characters" noise (consumer build) **Severity:** Cosmetic (non-blocking) · **Status:** Upstream SDK limitation · **Area:** SDK, not our code diff --git a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL_TODO.md b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_TODO.md similarity index 82% rename from AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL_TODO.md rename to AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_TODO.md index bccf161..45640a2 100644 --- a/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL_TODO.md +++ b/AyCode.Services/docs/SIGNALR_BINARY_PROTOCOL/SIGNALR_BINARY_PROTOCOL_TODO.md @@ -5,7 +5,7 @@ --- -## TODO-01: SegmentBufferReader isolated unit tests +## SBP-T-1: SegmentBufferReader isolated unit tests **Priority:** P1 · **Type:** Test coverage Original `vast-brewing-moonbeam` refactor plan (chunked receive-path) listed these tests in its verification section; they were never written. Needed: @@ -14,7 +14,7 @@ Original `vast-brewing-moonbeam` refactor plan (chunked receive-path) listed the - Missed-signal double-check pattern under `ManualResetEventSlim` reset - `Dispose` lifecycle (buffer pool return, old-buffer cleanup) -## TODO-02: Chunked protocol integration test +## SBP-T-2: Chunked protocol integration test **Priority:** P1 · **Type:** Test coverage End-to-end round-trip: @@ -27,8 +27,8 @@ Cover asymmetric cases: - Bytes sender, AsyncSegment receiver - WASM-downgraded sender (Segment), server AsyncSegment receiver -## TODO-03: `BinaryProtocolMode.Auto` wire-detection implementation -**Priority:** P3 · **Type:** Feature · **Related:** `SIGNALR_TODO.md#todo-04` +## SBP-T-3: `BinaryProtocolMode.Auto` wire-detection implementation +**Priority:** P3 · **Type:** Feature · **Related:** `../SIGNALR/SIGNALR_TODO.md#sig-t-4` Client-side adaptive mode: on first received message, inspect first payload byte: - `CHUNK_START (200)` → peer uses AsyncSegment → match it on subsequent sends (subject to local-platform constraint — WASM safety-net overrides to Segment) @@ -36,8 +36,8 @@ Client-side adaptive mode: on first received message, inspect first payload byte Per-`HubConnection` state. Requires changes to `BinaryProtocolMode` enum + detection wiring in `TryParseMessage`. -## TODO-04: SignalR handshake-extension for upfront mode negotiation -**Priority:** P3 · **Type:** Feature · **Related:** TODO-03 +## SBP-T-4: SignalR handshake-extension for upfront mode negotiation +**Priority:** P3 · **Type:** Feature · **Related:** SBP-T-3 Alternative to wire-detection: use SignalR handshake message's `extensions` JSON field to carry protocol-capability info — e.g. ```json diff --git a/AyCode.Services/docs/SIGNALR_TODO.md b/AyCode.Services/docs/SIGNALR_TODO.md deleted file mode 100644 index 157c990..0000000 --- a/AyCode.Services/docs/SIGNALR_TODO.md +++ /dev/null @@ -1,31 +0,0 @@ -# SignalR — TODO - -## Priority legend -- **P0** blocker · **P1** important · **P2** nice-to-have · **P3** idea - ---- - -## TODO-01: Diagnose first-call null in PostDataAsync<T> -**Priority:** P2 · **Type:** Investigation · **Related:** `SIGNALR_ISSUES.md#issue-02` - -Reproduce the `GetProductDtos_80`-style first-call null. Add trace logs to `PostDataAsync<T>` awaiter path and `OnReceiveMessage → pending request` dictionary lookup. Verify `requestId → Task<T>` correlation on the very first chunked response of a fresh connection. - -## TODO-02: Document asymmetric send/receive capability -**Priority:** P1 · **Type:** Docs - -Current behaviour: sender selects `BinaryProtocolMode` independently; receiver detects the wire format from the first byte (`CHUNK_START=200` → chunked path; else non-chunked). This means client and server can run DIFFERENT `ProtocolMode` settings independently — a core feature amplifying interoperability. - -Document in `SIGNALR_BINARY_PROTOCOL.md` as a dedicated "Asymmetric send/receive contract" section. Key selling points: -- WASM client + AsyncSegment server → works (WASM downgrades send, receives chunked happily) -- Third-party client on NuGet can pick any mode → server doesn't care -- Gradual mode rollouts possible (no synchronized deploy) - -## TODO-03: Code-level guard for FlushTimeout < ClientTimeoutInterval -**Priority:** P2 · **Type:** Feature - -`AcBinaryHubProtocolOptions.Validate()` currently documents (in XML doc) that `FlushTimeout` should be less than the SignalR `HubOptions.ClientTimeoutInterval`, but there is no code-level check. Add validation at protocol registration time — if both options are resolvable from the DI scope, verify the constraint and emit a startup warning (or throw, pending decision). - -## TODO-04: `BinaryProtocolMode.Auto` — adaptive send-mode -**Priority:** P3 · **Type:** Feature · **Related:** `SIGNALR_BINARY_PROTOCOL_TODO.md#todo-03` - -Design: on first received message, inspect first byte to determine peer's send format. On subsequent sends, match it (subject to local-platform constraints, e.g. WASM never actually sends AsyncSegment). Per-`HubConnection` state. Optional upfront handshake-extension negotiation as an alternative — see wire-level TODO. diff --git a/README.md b/README.md index 1fad42e..00df25b 100644 --- a/README.md +++ b/README.md @@ -24,10 +24,10 @@ Project-level docs — each project's `docs/` folder documents the code it defin | Project | Documents | |---|---| -| `AyCode.Core/docs/` | `BINARY_FORMAT.md`, `BINARY_FEATURES.md`, `BINARY_OPTIONS.md`, `LOGGING.md` | -| `AyCode.Core.Server/docs/` | `LOGGING_SERVER.md` | -| `AyCode.Services/docs/` | `SIGNALR.md`, `LOGGING_REMOTE.md` | -| `AyCode.Services.Server/docs/` | `SIGNALR_SERVER.md`, `SIGNALR_DATASOURCE.md` | +| `AyCode.Core/docs/` | `BINARY/` (README, FORMAT, FEATURES, OPTIONS, …), `LOGGING/` (README, ISSUES, TODO) | +| `AyCode.Core.Server/docs/` | `LOGGING/README.md` (server-side variant) | +| `AyCode.Services/docs/` | `SIGNALR/`, `SIGNALR_BINARY_PROTOCOL/`, `LOGGING/README.md` (remote variant) | +| `AyCode.Services.Server/docs/` | `SIGNALR/` (README + SIGNALR_DATASOURCE) | ## Solution Structure diff --git a/ToonExtendedInfo.txt b/ToonExtendedInfo.txt deleted file mode 100644 index fe90a12..0000000 --- a/ToonExtendedInfo.txt +++ /dev/null @@ -1,1185 +0,0 @@ -# AcToonSerializer - Complete Documentation - -## Overview - -**Token-Oriented Object Notation (Toon)** is a revolutionary serialization format specifically designed for Large Language Models (LLMs) like Claude, GPT-4, and others. Unlike JSON or XML, Toon prioritizes **maximum clarity and understanding** for AI systems while maintaining human readability. - -### Key Design Goals -1. **LLM-First**: Every design decision optimized for AI comprehension -2. **Zero Ambiguity**: Explicit structure markers eliminate parsing uncertainty -3. **Context-Aware**: Rich metadata provides semantic understanding -4. **Token Efficient**: Smart separation of schema and data -5. **Developer Friendly**: Works with or without custom attributes - ---- - -## Core Architecture - -### Three-Layer System - -``` -┌─────────────────────────────────────┐ -│ @meta Section │ ← Version, format, type registry -├─────────────────────────────────────┤ -│ @types Section │ ← Schema, descriptions, constraints -├─────────────────────────────────────┤ -│ @data Section │ ← Actual values -└─────────────────────────────────────┘ -``` - -### Why This Matters for LLMs - -**Traditional JSON:** -```json -{ - "id": 42, - "email": "john@example.com", - "tags": ["developer", "senior"] -} -``` -❌ LLM must infer: What is "id"? Is email validated? How many tags? - -**Toon Format:** -```toon -@types { - Person: "User account entity" - id: int32 - description: "Unique identifier" - purpose: "Primary key" - constraints: "required, auto-increment" - email: string - description: "Contact email" - constraints: "required, email-format, unique" - tags: string[] - description: "User role tags" -} - -@data { - Person { - id = 42 - email = "john@example.com" - tags = <string[]> (count: 2) [ - "developer" - "senior" - ] - } -} -``` -✅ LLM instantly knows: id is primary key, email is validated, exactly 2 tags - ---- - -## Feature Showcase - -### 1. Explicit Structure Boundaries - -**Problem with indentation-only formats (YAML):** -```yaml -person: - name: John - address: - street: Main St - city: Springfield -``` -❓ Where does `address` end? LLM must track indentation levels. - -**Toon Solution:** -```toon -Person { - Name = "John" - Address { - Street = "Main St" - City = "Springfield" - } -} -``` -✅ Clear `{}` boundaries - zero ambiguity - ---- - -### 2. Meta/Data Separation (Token Efficiency) - -**Multi-turn Conversation Pattern:** - -**Turn 1: Send Schema Once** -```csharp -var meta = AcToonSerializer.Serialize(person, AcToonSerializerOptions.MetaOnly); -// Output: Only @meta and @types sections -``` - -Output (~500 tokens): -```toon -@meta { - version = "1.0" - types = ["Person", "Address", "Company"] -} - -@types { - Person: "User account entity" - Id: int32 - description: "Unique identifier" - purpose: "Primary key" - constraints: "required" - Name: string - description: "Full name" - constraints: "required, max-length: 100" - Email: string - description: "Contact email" - constraints: "required, email-format" - // ... all properties -} -``` - -**Turn 2-N: Send Only Data** -```csharp -var data = AcToonSerializer.Serialize(person, AcToonSerializerOptions.DataOnly); -// Output: Only @data section -``` - -Output (~200 tokens): -```toon -@data { - Person { - Id = 42 - Name = "John Doe" - Email = "john@example.com" - } -} -``` - -**Result: 60% token savings in subsequent requests!** - ---- - -### 3. Type Hints Everywhere - -**Arrays with Count:** -```toon -Employees = <Person[]> (count: 150) [ - Person { Id = 1, Name = "Alice" } - Person { Id = 2, Name = "Bob" } - // ... 148 more -] -``` - -✅ LLM instantly knows: -- Collection type: Person array -- Exact count: 150 employees -- No need to iterate to count - -**Dictionaries with Count:** -```toon -Metrics = <dict> (count: 5) { - "Revenue" => 1500000.50 - "Growth" => 25.5 - "Expenses" => 800000.00 - "Profit" => 700000.50 - "Margin" => 46.67 -} -``` - -✅ LLM sees structure immediately - -**Inline Type Hints (Verbose Mode):** -```toon -@data { - Person { - Id = 42 <int32> - Name = "John" <string> - Age = 30 <int32> - Balance = 1234.56 <decimal> - IsActive = true <bool> - CreatedAt = "2024-01-10T10:30:00Z" <datetime> - } -} -``` - ---- - -### 4. Custom Attributes for Explicit Documentation - -**Define Rich Metadata:** -```csharp -using AyCode.Core.Serializers.Toons; - -[ToonDescription("Represents a user account in the system")] -public class Person -{ - [ToonDescription("Unique identifier for the person", - Purpose = "Primary key / database identity", - Constraints = "required, auto-increment, positive")] - public int Id { get; set; } - - [ToonDescription("Email address for contact and authentication", - Purpose = "User login and communication", - Constraints = "required, email-format, unique", - Examples = "user@example.com, admin@company.org")] - public string Email { get; set; } - - [ToonDescription("Age in years", - Constraints = "required, range: 0-150")] - public int Age { get; set; } -} -``` - -**Generated Output:** -```toon -@types { - Person: "Represents a user account in the system" - Id: int32 - description: "Unique identifier for the person" - purpose: "Primary key / database identity" - constraints: "required, auto-increment, positive" - Email: string - description: "Email address for contact and authentication" - purpose: "User login and communication" - constraints: "required, email-format, unique" - examples: "user@example.com, admin@company.org" - Age: int32 - description: "Age in years" - constraints: "required, range: 0-150" -} -``` - ---- - -### 5. Smart Inference (No Attributes Required) - -**Automatic Pattern Recognition:** - -```csharp -public class Person -{ - public int Id { get; set; } - public string Name { get; set; } - public string Email { get; set; } - public string PhoneNumber { get; set; } - public bool IsActive { get; set; } - public bool HasPremium { get; set; } - public DateTime CreatedAt { get; set; } - public DateTime? UpdatedAt { get; set; } - public int EmployeeCount { get; set; } -} -``` - -**Auto-Generated Descriptions:** -```toon -@types { - Person: "Object of type Person" - Id: int32 - description: "Unique identifier for Person" - purpose: "Primary key / unique identification" - constraints: "required" - Name: string - description: "Name of the Person" - constraints: "required" - Email: string - description: "Email address" - constraints: "required, email-format" - PhoneNumber: string - description: "Phone number" - constraints: "required" - IsActive: bool - description: "Boolean flag indicating Active" - purpose: "Status flag" - constraints: "required" - HasPremium: bool - description: "Boolean flag indicating possession of Premium" - purpose: "Status flag" - constraints: "required" - CreatedAt: datetime - description: "Date/time value for CreatedAt" - purpose: "Timestamp when entity was created" - constraints: "required" - UpdatedAt: datetime? - description: "Date/time value for UpdatedAt" - purpose: "Timestamp of last update" - constraints: "nullable" - EmployeeCount: int32 - description: "Count of Employee" - constraints: "required, non-negative" -} -``` - -**Detected Patterns:** -- `Id` → Primary key -- `Name` → Entity name -- `Email`, `Phone`, `Address` → Contact info -- `IsXxx`, `HasXxx` → Boolean flags -- `CreatedAt`, `UpdatedAt`, `DeletedAt` → Audit timestamps -- `XxxCount` → Counters (non-negative) - ---- - -### 6. Multi-line String Support - -**Problem with Escaped Strings:** -```json -{ - "bio": "Line 1\nLine 2\nLine 3\n\nSpecialties:\n- C#\n- .NET\n- Azure" -} -``` -❌ Hard to read, especially with code snippets - -**Toon Solution:** -```toon -Bio = """ - Senior Software Engineer with 10+ years of experience. - - Specialties: - - C# and .NET development - - Cloud architecture (Azure, AWS) - - Microservices and distributed systems - - Passionate about clean code and mentoring. -""" -``` -✅ Preserves formatting, easy to read - -**Automatically Triggered:** -- Strings > 80 characters (configurable) -- Manual override available - ---- - -### 7. Reference Handling for Circular Objects - -**Circular Reference Example:** -```csharp -var company = new Company { Name = "ACME Corp" }; -var ceo = new Person { Name = "John Doe" }; -company.CEO = ceo; -ceo.Company = company; // Circular! -``` - -**Toon Output:** -```toon -@data { - @1 Company { - Name = "ACME Corp" - CEO = @2 Person { - Name = "John Doe" - Company = @ref:1 - } - } -} -``` - -✅ `@1` marks first occurrence -✅ `@ref:1` references it -✅ No infinite loops or duplication - ---- - -## Complete Usage Examples - -### Example 1: E-commerce System - -```csharp -using AyCode.Core.Serializers.Toons; - -[ToonDescription("Online store product listing")] -public class Product -{ - [ToonDescription("Unique product identifier", - Purpose = "Primary key", - Constraints = "required, auto-increment")] - public int Id { get; set; } - - [ToonDescription("Product display name", - Constraints = "required, max-length: 200")] - public string Name { get; set; } - - [ToonDescription("Detailed product description", - Constraints = "nullable, max-length: 2000")] - public string? Description { get; set; } - - [ToonDescription("Price in USD", - Constraints = "required, positive, precision: 2")] - public decimal Price { get; set; } - - [ToonDescription("Available inventory count", - Constraints = "required, non-negative")] - public int Stock { get; set; } - - [ToonDescription("Product category tags")] - public List<string> Tags { get; set; } -} - -// Serialize -var product = new Product -{ - Id = 101, - Name = "Premium Wireless Headphones", - Description = "High-quality noise-canceling headphones.\n\nFeatures:\n- 40-hour battery\n- Active noise cancellation\n- Premium sound quality", - Price = 299.99m, - Stock = 47, - Tags = new List<string> { "electronics", "audio", "premium" } -}; - -var toon = AcToonSerializer.Serialize(product); -``` - -**Output:** -```toon -@meta { - version = "1.0" - format = "toon" - types = ["Product"] -} - -@types { - Product: "Online store product listing" - Id: int32 - description: "Unique product identifier" - purpose: "Primary key" - constraints: "required, auto-increment" - Name: string - description: "Product display name" - constraints: "required, max-length: 200" - Description: string? - description: "Detailed product description" - constraints: "nullable, max-length: 2000" - Price: decimal - description: "Price in USD" - constraints: "required, positive, precision: 2" - Stock: int32 - description: "Available inventory count" - constraints: "required, non-negative" - Tags: string[] - description: "Product category tags" - constraints: "nullable" -} - -@data { - Product { - Id = 101 - Name = "Premium Wireless Headphones" - Description = """ - High-quality noise-canceling headphones. - - Features: - - 40-hour battery - - Active noise cancellation - - Premium sound quality - """ - Price = 299.99 - Stock = 47 - Tags = <string[]> (count: 3) [ - "electronics" - "audio" - "premium" - ] - } -} -``` - ---- - -### Example 2: Token-Efficient Workflow - -```csharp -// === TURN 1: Initial Request - Send Full Context === -var person = new Person { Id = 1, Name = "Alice", Email = "alice@example.com" }; -var fullToon = AcToonSerializer.Serialize(person, AcToonSerializerOptions.Default); -// LLM learns schema (~600 tokens) - -// === TURN 2-10: Updates - Send Only Data === -var updates = new[] -{ - new Person { Id = 2, Name = "Bob", Email = "bob@example.com" }, - new Person { Id = 3, Name = "Charlie", Email = "charlie@example.com" }, - new Person { Id = 4, Name = "Diana", Email = "diana@example.com" } -}; - -foreach (var update in updates) -{ - var dataToon = AcToonSerializer.Serialize(update, AcToonSerializerOptions.DataOnly); - // Each ~150 tokens instead of ~600 - // Total savings: 450 tokens × 3 = 1,350 tokens saved! -} -``` - ---- - -## Configuration Options - -### Preset Modes - -```csharp -// 1. Default - Full context (first time) -AcToonSerializerOptions.Default - -// 2. MetaOnly - Schema only (send once) -AcToonSerializerOptions.MetaOnly - -// 3. DataOnly - Values only (subsequent requests) -AcToonSerializerOptions.DataOnly - -// 4. Compact - Minimal output (no indentation) -AcToonSerializerOptions.Compact - -// 5. Verbose - All hints inline (debugging) -AcToonSerializerOptions.Verbose -``` - -### Custom Configuration - -```csharp -var options = new AcToonSerializerOptions -{ - Mode = ToonSerializationMode.Full, - UseMeta = true, - UseEnhancedMetadata = true, - ShowCollectionCount = true, - UseMultiLineStrings = true, - MultiLineStringThreshold = 80, - UseInlineTypeHints = false, - OmitDefaultValues = true, - UseReferenceHandling = true, - MaxDepth = 10 -}; -``` - ---- - -## Performance Characteristics - -### Token Efficiency - -| Scenario | JSON | Toon Full | Toon DataOnly | Savings | -|----------|------|-----------|---------------|---------| -| First Request | 800 | 1000 | - | -25% | -| Subsequent (×10) | 8000 | - | 4000 | **50%** | -| **Total Conversation** | **8800** | - | **5000** | **43%** | - -### Speed Benchmarks - -``` -Serialization Speed (relative to JSON): -- First time (Full): ~85% (builds metadata cache) -- Subsequent (DataOnly): ~95% (cache hit) -- With attributes: ~90% (reflection overhead) -``` - ---- - -## Why Toon is Superior for LLMs - -### 1. **Cognitive Load Reduction** - -**JSON:** -```json -{"users": [{"id": 1}, {"id": 2}]} -``` -LLM thinks: *"What's in users? How many? What properties exist?"* - -**Toon:** -```toon -users = <Person[]> (count: 2) [ - Person { id = 1 } - Person { id = 2 } -] -``` -LLM knows: *"Array of Person, exactly 2 items, each has id property"* - -### 2. **Semantic Understanding** - -**JSON:** -```json -{"email": "test@example.com"} -``` -LLM: *"Is this validated? Required? Format?"* - -**Toon:** -```toon -email: string - description: "Contact email" - constraints: "required, email-format, unique" -``` -LLM: *"Must be valid email, required, unique in system"* - -### 3. **Context Preservation** - -**Multi-turn JSON:** -``` -Turn 1: {"id": 1, "name": "Alice"} -Turn 2: {"id": 2, "name": "Bob"} -Turn 3: {"id": 3, "name": "Charlie"} -``` -LLM: *"Same structure? Any changes? Must infer each time"* - -**Multi-turn Toon:** -``` -Turn 1: @types { Person: ... } @data { ... } -Turn 2: @data { Person { id = 2 } } -Turn 3: @data { Person { id = 3 } } -``` -LLM: *"Schema known from Turn 1, only data changes"* - ---- - -## Best Practices - -### 1. Use MetaOnly/DataOnly Pattern - -```csharp -// Start of conversation -var schemaToon = AcToonSerializer.Serialize(typeof(MyClass), AcToonSerializerOptions.MetaOnly); -await SendToLLM(schemaToon); - -// Subsequent messages -var dataToon = AcToonSerializer.Serialize(instance, AcToonSerializerOptions.DataOnly); -await SendToLLM(dataToon); -``` - -### 2. Add Custom Attributes for Domain Models - -```csharp -[ToonDescription("Core business entity")] -public class Customer -{ - [ToonDescription("Customer identifier", Purpose = "Primary key")] - public int Id { get; set; } -} -``` - -### 3. Rely on Smart Inference for DTOs - -```csharp -// No attributes needed - smart inference handles it -public class UserDto -{ - public int Id { get; set; } // → "Unique identifier" - public string Email { get; set; } // → "Email address" - public bool IsActive { get; set; } // → "Boolean flag" -} -``` - ---- - -## Summary - -**AcToonSerializer** is the first serialization format designed specifically for LLM understanding: - -✅ **Zero Ambiguity** - Explicit boundaries (`{}`, `[]`) -✅ **Rich Context** - Descriptions, constraints, purpose -✅ **Token Efficient** - 30-50% savings with Meta/Data split -✅ **Type Clear** - Count hints, type annotations -✅ **Flexible** - Works with or without custom attributes -✅ **Smart** - Auto-infers common patterns -✅ **Complete** - Handles circular refs, multi-line strings, all C# types - -**Result: LLMs understand your data structures perfectly with minimal token cost!** - ---- - -## Toon vs JSON vs XML - Comprehensive Comparison - -### Overview Table - -| Feature | Toon | JSON | XML | -|---------|------|------|-----| -| **LLM Readability** | ⭐⭐⭐⭐⭐ Excellent | ⭐⭐⭐ Good | ⭐⭐ Fair | -| **Human Readability** | ⭐⭐⭐⭐⭐ Excellent | ⭐⭐⭐⭐ Good | ⭐⭐ Fair | -| **Structure Clarity** | Explicit `{}` `[]` | Implicit (commas) | Verbose tags | -| **Type Information** | Built-in + hints | None | Via schema only | -| **Metadata Support** | Rich (desc, purpose, constraints) | None | Via schema only | -| **Schema Separation** | Yes (@meta/@types/@data) | No | External XSD | -| **Token Efficiency** | ⭐⭐⭐⭐⭐ (43% savings) | ⭐⭐⭐ Baseline | ⭐ Verbose | -| **Multi-line Strings** | Native `"""` | Escaped `\n` | CDATA or escaped | -| **Collection Count** | Yes `<type[]> (count: N)` | No | No | -| **Reference Handling** | Built-in `@1, @ref:1` | Manual | Via id/idref | -| **Smart Inference** | Yes (15+ patterns) | No | No | -| **Custom Attributes** | Yes (ToonDescription) | No | No | -| **Parsing Complexity** | Simple | Simple | Complex | -| **Size (bytes)** | Medium | Small | Large | -| **Ambiguity Level** | Zero | Low | Medium | - ---- - -### Detailed Comparison - -#### 1. Structure Clarity - -**Toon:** -```toon -Person { - Name = "John" - Address { - City = "NYC" - } -} -``` -✅ Clear scope boundaries -✅ Explicit start/end -✅ No punctuation confusion - -**JSON:** -```json -{ - "person": { - "name": "John", - "address": { - "city": "NYC" - } - } -} -``` -⚠️ Commas required -⚠️ Easy to miss closing braces -⚠️ No type information - -**XML:** -```xml -<Person> - <Name>John</Name> - <Address> - <City>NYC</City> - </Address> -</Person> -``` -❌ Verbose -❌ Opening/closing tags redundant -❌ More bytes for same data - ---- - -#### 2. Type Information & Metadata - -**Toon:** -```toon -@types { - Person: "User account entity" - Id: int32 - description: "Unique identifier" - purpose: "Primary key" - constraints: "required, auto-increment" - Email: string - description: "Contact email" - constraints: "required, email-format, unique" - examples: "user@example.com" -} -``` -✅ Types inline -✅ Rich metadata -✅ Descriptions, constraints, purpose, examples -✅ LLM understands semantics immediately - -**JSON:** -```json -{ - "id": 42, - "email": "user@example.com" -} -``` -❌ No type info -❌ No metadata -❌ LLM must infer everything -❌ Requires separate documentation - -**XML with XSD:** -```xml -<!-- Data --> -<Person> - <Id>42</Id> - <Email>user@example.com</Email> -</Person> - -<!-- Separate schema file --> -<xs:schema> - <xs:element name="Id" type="xs:int"/> - <xs:element name="Email" type="xs:string"/> -</xs:schema> -``` -⚠️ Schema in separate file -⚠️ Complex schema language -⚠️ No semantic descriptions - ---- - -#### 3. Collection Handling - -**Toon:** -```toon -Tags = <string[]> (count: 3) [ - "developer" - "senior" - "remote" -] -``` -✅ Type visible: `string[]` -✅ Count visible: `3` -✅ Clear boundaries -✅ LLM knows structure instantly - -**JSON:** -```json -{ - "tags": ["developer", "senior", "remote"] -} -``` -⚠️ No type info (could be mixed types) -⚠️ No count (must iterate) -⚠️ Square brackets only marker - -**XML:** -```xml -<Tags> - <Tag>developer</Tag> - <Tag>senior</Tag> - <Tag>remote</Tag> -</Tags> -``` -❌ Verbose (3x more bytes) -❌ No count -❌ No type info -❌ Repetitive tags - ---- - -#### 4. Multi-line Strings - -**Toon:** -```toon -Bio = """ - Senior Software Engineer - - Specialties: - - C# Development - - Cloud Architecture -""" -``` -✅ Natural formatting -✅ Readable -✅ No escaping needed - -**JSON:** -```json -{ - "bio": "Senior Software Engineer\n\nSpecialties:\n- C# Development\n- Cloud Architecture" -} -``` -❌ Escaped newlines -❌ Hard to read -❌ Error-prone - -**XML:** -```xml -<Bio><![CDATA[ -Senior Software Engineer - -Specialties: -- C# Development -- Cloud Architecture -]]></Bio> -``` -⚠️ CDATA verbose -⚠️ Extra syntax - ---- - -#### 5. Token Efficiency (Multi-turn Conversations) - -**Scenario: 10-turn conversation with same schema** - -**Toon:** -- Turn 1: 1000 tokens (Full mode with @meta/@types/@data) -- Turn 2-10: 200 tokens each (DataOnly mode) -- **Total: 1000 + (9 × 200) = 2,800 tokens** - -**JSON:** -- Turn 1-10: 600 tokens each (no schema separation) -- **Total: 10 × 600 = 6,000 tokens** - -**XML:** -- Turn 1-10: 900 tokens each (verbose) -- **Total: 10 × 900 = 9,000 tokens** - -**Result:** -- Toon saves **53% vs JSON** -- Toon saves **69% vs XML** - ---- - -#### 6. Reference Handling (Circular Objects) - -**Toon:** -```toon -@1 Company { - Name = "ACME" - CEO = @2 Person { - Name = "John" - Company = @ref:1 - } -} -``` -✅ Built-in -✅ Clear syntax -✅ Automatic detection - -**JSON (manual):** -```json -{ - "$id": "1", - "name": "ACME", - "ceo": { - "$id": "2", - "name": "John", - "company": { "$ref": "1" } - } -} -``` -⚠️ Manual implementation -⚠️ Not standard -⚠️ Library-dependent - -**XML:** -```xml -<Company id="c1"> - <Name>ACME</Name> - <CEO id="p1"> - <Name>John</Name> - <Company idref="c1"/> - </CEO> -</Company> -``` -⚠️ Attribute-based -⚠️ Requires schema -⚠️ Complex validation - ---- - -#### 7. Semantic Understanding for LLMs - -**Example: Understanding an "email" field** - -**Toon:** -```toon -@types { - Person: "User account" - Email: string - description: "Contact email address" - purpose: "User authentication and communication" - constraints: "required, email-format, unique" - examples: "user@example.com" -} -@data { - Person { Email = "john@example.com" } -} -``` - -**LLM understands:** -1. ✅ It's an email address (description) -2. ✅ Used for authentication (purpose) -3. ✅ Must be valid email format (constraints) -4. ✅ Required field (constraints) -5. ✅ Must be unique (constraints) -6. ✅ Format example provided - -**JSON:** -```json -{ - "email": "john@example.com" -} -``` - -**LLM infers:** -1. ⚠️ Probably email (from name) -2. ❌ Purpose unknown -3. ❌ Constraints unknown -4. ❌ Validation rules unknown -5. ❌ Uniqueness unknown - -**XML:** -```xml -<Email>john@example.com</Email> -``` - -**LLM infers:** -1. ⚠️ Probably email (from tag name) -2. ❌ All other context missing - ---- - -#### 8. Smart Inference (No Manual Documentation) - -**Toon (Automatic):** -```csharp -public class Person -{ - public int Id { get; set; } - public string Email { get; set; } - public bool IsActive { get; set; } - public DateTime CreatedAt { get; set; } -} -``` - -**Auto-generated:** -```toon -@types { - Person: "Object of type Person" - Id: int32 - description: "Unique identifier for Person" - purpose: "Primary key / unique identification" - Email: string - description: "Email address" - constraints: "required, email-format" - IsActive: bool - description: "Boolean flag indicating Active" - purpose: "Status flag" - CreatedAt: datetime - description: "Date/time value for CreatedAt" - purpose: "Timestamp when entity was created" -} -``` -✅ 15+ patterns recognized -✅ Zero manual work -✅ Intelligent descriptions - -**JSON/XML:** -❌ No automatic metadata -❌ Requires manual documentation -❌ No pattern recognition - ---- - -#### 9. Real-World Size Comparison - -**Sample: Person object with 10 properties** - -**Toon (Full mode, first time):** -``` -@meta + @types: 600 bytes -@data: 250 bytes -Total: 850 bytes -``` - -**Toon (DataOnly mode, subsequent):** -``` -@data only: 250 bytes -``` - -**JSON:** -``` -Data + field names: 400 bytes (every time) -``` - -**XML:** -``` -Data + tags (opening/closing): 800 bytes (every time) -``` - -**10 requests total:** -- Toon: 850 + (9 × 250) = **3,100 bytes** -- JSON: 10 × 400 = **4,000 bytes** (+29%) -- XML: 10 × 800 = **8,000 bytes** (+158%) - ---- - -#### 10. Parsing Complexity for LLMs - -**Toon:** -1. Read @meta → know version, types -2. Read @types → understand schema completely -3. Read @data → parse values with full context -4. Clear boundaries (`{}`, `[]`) → no ambiguity - -**Complexity: ⭐ Low** - -**JSON:** -1. Parse entire structure -2. Infer types from values -3. Guess semantic meaning from keys -4. Track nested braces and commas -5. No schema context - -**Complexity: ⭐⭐⭐ Medium** - -**XML:** -1. Parse opening/closing tags -2. Match tag pairs -3. Handle attributes vs elements -4. External schema lookup (if used) -5. Namespace handling -6. CDATA sections - -**Complexity: ⭐⭐⭐⭐⭐ High** - ---- - -### Summary: Toon Advantages - -#### vs JSON: - -✅ **Semantic richness**: Descriptions, purpose, constraints, examples -✅ **Type clarity**: Explicit types, not inferred -✅ **Token efficiency**: 43% savings in conversations (Meta/Data split) -✅ **Structure clarity**: Explicit boundaries vs implicit commas -✅ **Smart inference**: Automatic metadata generation -✅ **Multi-line strings**: Native support vs escaped -✅ **Collection hints**: Count and type visible -✅ **Reference handling**: Built-in vs manual -✅ **LLM understanding**: Rich context vs bare values - -**When to use JSON:** Legacy systems, browser APIs, minimal bandwidth (single request) - -#### vs XML: - -✅ **Conciseness**: 50-70% smaller -✅ **Readability**: Clean syntax vs verbose tags -✅ **Type information**: Inline vs external schema -✅ **Metadata**: Built-in vs external -✅ **Parsing**: Simple vs complex -✅ **Modern**: Designed for LLMs vs 1998 technology -✅ **Token efficiency**: 69% savings -✅ **No redundancy**: Single property names vs opening/closing tags -✅ **Clean collections**: Arrays vs repetitive elements - -**When to use XML:** Legacy enterprise systems, SOAP, strict schema validation requirements - ---- - -### The Toon Advantage: Real-World Impact - -**Use Case: Multi-turn LLM conversation (analyzing 100 customer records)** - -| Format | Tokens Used | Cost (Claude 3.5) | Processing Time | -|--------|-------------|-------------------|-----------------| -| Toon | 15,000 | $0.30 | Fast (schema parsed once) | -| JSON | 35,000 | $0.70 | Medium (infer schema each time) | -| XML | 52,000 | $1.04 | Slow (parse verbose structure) | - -**Toon Savings:** -- **57% fewer tokens** vs JSON -- **71% fewer tokens** vs XML -- **57% cost reduction** vs JSON -- **71% cost reduction** vs XML -- **Better LLM accuracy** (full semantic context) - ---- - -### Conclusion - -**Toon is superior when:** -1. Working with LLMs (Claude, GPT-4, etc.) -2. Multi-turn conversations (schema reuse) -3. Need semantic understanding (not just data) -4. Want automatic documentation -5. Prefer clarity over brevity -6. Handle complex object graphs -7. Need both human and AI readability - -**JSON is better when:** -1. Browser/web API compatibility required -2. Single-request scenarios -3. Absolute minimum size critical -4. No LLM processing involved - -**XML is better when:** -1. Legacy enterprise systems -2. Strict schema validation via XSD -3. SOAP/WS-* protocols -4. Industry standards require it - -**For modern LLM-powered applications, Toon is the clear winner.** 🏆 diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8988642..5f01c8b 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -85,7 +85,7 @@ AyCode.Services ← AyCode.Services.Server - **AyCode.Services.Server** — Server-side: SignalR hub with custom binary protocol, email (SendGrid), JWT auth. - **AyCode.Models.Server/DynamicMethods** — Reflection-based tag→method dispatch used by the SignalR hub. -> **SignalR Dispatch:** Both directions use a single method `OnReceiveMessage(int messageTag, int? requestId, SignalParams signalParams, object data)` with integer tag-based routing instead of standard Hub methods. Write path: zero-copy via `AcBinarySerializer.Serialize(value, output)` directly to pipe. Read path: protocol eagerly deserializes `data` to typed object via `SignalParams.SignalDataType`, or returns raw `byte[]` for `IsRawBytesData`/byte[] fast-path. See `AyCode.Services/docs/SIGNALR.md` for full details. +> **SignalR Dispatch:** Both directions use a single method `OnReceiveMessage(int messageTag, int? requestId, SignalParams signalParams, object data)` with integer tag-based routing instead of standard Hub methods. Write path: zero-copy via `AcBinarySerializer.Serialize(value, output)` directly to pipe. Read path: protocol eagerly deserializes `data` to typed object via `SignalParams.SignalDataType`, or returns raw `byte[]` for `IsRawBytesData`/byte[] fast-path. See `AyCode.Services/docs/SIGNALR/README.md` for full details. ### Server Extensions - **AyCode.Core.Server**, **AyCode.Interfaces.Server**, **AyCode.Entities.Server**, **AyCode.Models.Server** — Server-only additions that don't belong in shared code. diff --git a/docs/CONVENTIONS.md b/docs/CONVENTIONS.md index 7efc916..3480373 100644 --- a/docs/CONVENTIONS.md +++ b/docs/CONVENTIONS.md @@ -24,16 +24,16 @@ ## SignalR Conventions -See `AyCode.Services/docs/SIGNALR.md` for full architecture documentation. +See `AyCode.Services/docs/SIGNALR/README.md` for full architecture documentation. - **Single dispatch method** — all communication goes through `OnReceiveMessage(int messageTag, int? requestId, SignalParams signalParams, object data)`. Do not add new hub methods. - **Tag-based routing** — associate methods with integer tags via `[SignalR(tag)]` (server) or `[SignalRSendToClient(tag)]` (client). Tags must be unique across the entire system. -- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. +- **CRUD bundles** — entities use `SignalRCrudTags(getAllTag, getItemTag, addTag, updateTag, removeTag)` with 5 independent tag integers. Tags must be unique across the system. See `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md`. - **Binary protocol** — `AyCodeBinaryHubProtocol` (derived from `AcBinaryHubProtocol`) is the transport protocol. Zero-copy write: `AcBinarySerializer.Serialize(value, output)` directly to pipe. Zero-copy read: `SequenceReader<byte>` + type-aware deserialization via `SignalParams.SignalDataType`. Three read paths: byte[] fast-path (0x44 tag), IsRawBytesData (raw byte[]), typed deserialization. ### ⚠️ Temporary: JSON-in-Binary Request Parameters -Client→server request parameters currently use a JSON-inside-Binary envelope — a cross-cutting tech debt planned for migration to pure Binary. Details in `AyCode.Core/docs/BINARY_ISSUES.md#xcut-1` + `AyCode.Services/docs/SIGNALR_ISSUES.md#xcut-1`. Migration is tracked in `BINARY_TODO.md#todo-01`. Do NOT attempt as a side-effect of unrelated work — requires coordinated client+server+consuming-project changes. +Client→server request parameters currently use a JSON-inside-Binary envelope — a cross-cutting tech debt planned for migration to pure Binary. Canonical entry: `AyCode.Core/AyCode.Core/docs/XCUT/XCUT_ISSUES.md#xcut-i-1`. Cross-refs: `BINARY_ISSUES.md#xcut-i-1` (serializer side) and `SIGNALR_ISSUES.md#xcut-i-1` (transport side). Migration is tracked in `BINARY_TODO.md#bin-t-1`. Do NOT attempt as a side-effect of unrelated work — requires coordinated client+server+consuming-project changes. ## Testing diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index 945773d..01c8eed 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -19,11 +19,11 @@ Core terminology for the AyCode framework. Read this before working on unfamilia | **Toon** | Token-Oriented Object Notation. LLM-optimized format with @meta (schema) and @data (values) sections. Designed for maximum LLM comprehension accuracy. | | **AcJson** | Newtonsoft.Json wrapper with $id/$ref reference handling, IId-based resolution, and chain deserialization API. | | **Chain API** | Fluent deserialization: `CreateDeserializeChain<T>().ThenDeserialize<U>()...Execute()`. Resolves cross-references across multiple types. | -| **String Interning** | Binary serializer deduplicates repeated strings. Controlled via `[AcStringIntern]` attribute or `StringInterningMode`. See `AyCode.Core/docs/BINARY_FEATURES.md`. | +| **String Interning** | Binary serializer deduplicates repeated strings. Controlled via `[AcStringIntern]` attribute or `StringInterningMode`. See `AyCode.Core/AyCode.Core/docs/BINARY/BINARY_FEATURES.md`. | ## Binary Wire Format -For full specification see `AyCode.Core/docs/BINARY_FORMAT.md`. +For full specification see `AyCode.Core/AyCode.Core/docs/BINARY/BINARY_FORMAT.md`. | Term | Definition | |---|---| @@ -68,7 +68,7 @@ For full specification see `AyCode.Core/docs/BINARY_FORMAT.md`. ## SignalR Infrastructure -For full architecture see `AyCode.Services/docs/SIGNALR.md`. +For full architecture see `AyCode.Services/docs/SIGNALR/README.md`. | Term | Definition | |---|---| @@ -76,7 +76,7 @@ For full architecture see `AyCode.Services/docs/SIGNALR.md`. | **SignalParams** | Metadata sent alongside message payload as separate hub argument. Contains `Status`, `DataSerializerType`, `Parameters` (`byte[]?` — packed `byte[][]` as single blob), `SignalDataType` (`string?` — response type for eager deserialization), `IsRawBytesData` (`bool` — return raw bytes without deserialization). Typed access via `SetParameterValues(object[])` / `GetParameterValues(ParameterInfo[])` — PostDataJson pattern. `[AcBinarySerializable]`. Never null — only fields inside are nullable. | | **Message Tag** | Integer identifier mapping to a method via `[SignalR(tag)]` or `[SignalRSendToClient(tag)]` attributes. | | **DynamicMethodRegistry** | Resolves message tags to `MethodInfo` at runtime. Static `ConcurrentDictionary` cache with lazy scan on miss. | -| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See `AyCode.Services.Server/docs/SIGNALR_DATASOURCE.md`. | +| **SignalRCrudTags** | Sealed class bundling 5 independent tag integers (getAllTag, getItemTag, addTag, updateTag, removeTag) for entity CRUD. See `AyCode.Services.Server/docs/SIGNALR/SIGNALR_DATASOURCE.md`. | | **AcBinaryHubProtocol** | Unsealed base `IHubProtocol` replacing SignalR's JSON+Base64 with `AcBinarySerializer`. Protocol name: `"acbinary"` (configurable). Options-based ctor: `new AcBinaryHubProtocol(AcBinaryHubProtocolOptions)`. Write: `BufferWriterBinaryOutput` / `AsyncPipeWriterOutput` zero-copy to pipe. Read: `ArrayBinaryInput` via `GetArgBytes` (zero-copy single-seg / pool-rent multi-seg) for non-chunked; chunked receive via `SegmentBufferReader` + `SegmentBufferReaderInput` with platform-aware fallback. | | **AyCodeBinaryHubProtocol** | Consumer-specific derived protocol (per-message header with `DataFlags`, `IsRawBytesData`, type resolution). Registered via `services.AddSignalR().AddAcBinaryProtocol(...)` on the server and `hubBuilder.AddAcBinaryProtocol(...)` on the client. | | **AcBinaryHubProtocolOptions** | Mutable config class for protocol registration. Properties: `SerializerOptions`, `ProtocolMode`, `BufferSize`, `WaitForFlush`, `FlushTimeout`, `Name`, `Logger`. `Validate()` enforces invariants (incl. WASM + AsyncSegment block). `Clone()` for DI `IOptions<T>` safety. | @@ -94,7 +94,7 @@ For full architecture see `AyCode.Services/docs/SIGNALR.md`. ## Logging -For full architecture see `AyCode.Core/docs/LOGGING.md`. +For full architecture see `AyCode.Core/AyCode.Core/docs/LOGGING/README.md`. | Term | Definition | |---|---| diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..42d9a4d --- /dev/null +++ b/docs/README.md @@ -0,0 +1,20 @@ +# AyCode.Core documentation + +Top-level documentation for the `AyCode.Core` repo (Layer 0 — core framework). + +## Reference docs (flat) + +- [`ARCHITECTURE.md`](ARCHITECTURE.md) — Repo architecture overview +- [`CONVENTIONS.md`](CONVENTIONS.md) — Coding conventions +- [`GLOSSARY.md`](GLOSSARY.md) — Domain glossary + +## Sub-projects with docs + +- `AyCode.Core/docs/` — Logger, Binary serializer (paired topics: LOGGING/, BINARY/) +- `AyCode.Core.Server/docs/` — Server-side logger variant (LOGGING/) +- `AyCode.Services/docs/` — Remote logger variant, SignalR, SignalR binary protocol +- `AyCode.Services.Server/docs/` — Server-side SignalR + data source + +## Navigation + +Per the AI Agent Core Protocol (folder navigation rule), start from this README when browsing `docs/`. Single-file reference docs remain flat at the repo-root level; multi-file topics live in named subfolders at the sub-project level.