AyCode.Core/AyCode.Benchmark/README.md

# AyCode.Benchmark

BenchmarkDotNet performance suite **plus** the shared workload / reporting infrastructure used by both BDN and the Console runner. Targets .NET 9.

## Role: dual-purpose project

This project plays **two roles**:

1. **BDN runner Exe** — standalone benchmark host (`Program.cs` + `[Benchmark]`-decorated classes). Invoke via `dotnet run -c Release --project AyCode.Benchmark -- <switch>`.
2. **Shared workload + reporting library** — exposes `public` types under [`Workloads/Scenarios/`](Workloads/Scenarios/) and [`Reporting/`](Reporting/) that [`AyCode.Core.Serializers.Console`](../AyCode.Core.Serializers.Console/README.md) consumes via `<ProjectReference>`.

Both runners feed the SAME `ISerializerBenchmark` workload (same test data graphs, same wire options, same payload sizes) — so Console's adaptive-engine numbers and BDN's iteration-based numbers are **directly comparable**.

## Output convention

Both runners emit a unified `.log` / `.LLM` / `.output` triplet to `Test_Benchmark_Results/Benchmark/` (resolved at runtime via walk-up to the nearest `AyCode.Core.sln` — worktree-aware):

| File | Source | Content |
|---|---|---|
| `Console.FullBenchmark_<Build>_<ts>.log` | Console runner | Human-readable formatted view |
| `Console.FullBenchmark_<Build>_<ts>.LLM` | Console runner | Markdown table, LLM-paste-friendly |
| `Console.FullBenchmark_<Build>_<ts>.output` | Console runner | Hex dump of Large cell binary |
| `Bdn.FullBenchmark_<Build>_<ts>.log` | BDN runner | Same format as Console |
| `Bdn.FullBenchmark_<Build>_<ts>.LLM` | BDN runner | Same |
| `Bdn.FullBenchmark_<Build>_<ts>.output` | BDN runner | Same |

BDN-native artifacts (BDN's own reports, raw measurements, run logs) go to `Test_Benchmark_Results/Benchmark/BDN/` — kept separate so the unified Console+BDN `.log/.LLM/.output` triplet stays uncluttered.

## Architecture

```
┌────────────────────────────────────────────────────────────────┐
│ AyCode.Benchmark (this project)                                │
│                                                                 │
│  Workloads/Scenarios/    public — shared workload types        │
│    ISerializerBenchmark, BenchmarkOptions, BenchmarkEnums,     │
│    AcBinaryBenchmark<T>, MemoryPackBenchmark<T>,               │
│    AcBinaryBufferWriterBenchmark<T>, ... (12 concretes),       │
│    RoundTripValidator                                           │
│                                                                 │
│  Reporting/              public — shared reporting types       │
│    BenchmarkResult, ReportingContext, BenchmarkReportWriter    │
│                                                                 │
│  AcBinaryVsMemPackBenchmark.cs   BDN [Benchmark] class          │
│                                  (mirrors Console "F" menu)    │
│  BdnSummaryAdapter.cs            Summary → BenchmarkResult →   │
│                                  BenchmarkReportWriter.SaveAll │
│  Program.cs              BDN entry + CLI dispatch              │
│                                                                 │
│  + KEEP: JitDisassemblyBenchmark, RefForeachBenchmark,         │
│          TaskHelperBenchmarks, ValueTypePassingBenchmark,      │
│          SourceGeneratorBenchmarks,                            │
│          SignalRCommunicationBenchmarks,                       │
│          SignalRRoundTripBenchmarks                            │
└────────────────────────────────────────────────────────────────┘
                              ▲
                              │ ProjectReference (one-way)
                              │
┌────────────────────────────────────────────────────────────────┐
│ AyCode.Core.Serializers.Console                                │
│                                                                 │
│  BenchmarkLoop.cs        custom adaptive measure engine        │
│    (CPU 0 pin, High priority, phase-isolated warmup,           │
│    10-sample median + pilot, ~250ms/cell calibration)          │
│  Menu.cs / Configuration.cs / Program.cs    Console UX         │
│                                                                 │
│  Uses Benchmark's:                                             │
│   - Workloads/Scenarios/* (interface + concrete benchmarks)    │
│   - Reporting/BenchmarkReportWriter (SaveAll, Print...)        │
└────────────────────────────────────────────────────────────────┘
```

## Two runners — same workload, different measurement engines

| Aspect | Console (custom engine) | BDN |
|---|---|---|
| Use case | Fast iteration during micro-opt loops | Statistically confident before-commit validation |
| Measurement | Adaptive per-cell iter (target ~250ms), 10 samples + pilot, median | Warmup + N iterations, outlier removal, JIT-stabilized, process-spawn isolation |
| Time per full run | ~1-3 min | ~5-15 min |
| Noise floor | ~3-5% inter-engine delta visible | ~1-2% |
| Output format | Identical (same `BenchmarkReportWriter` writes both) | |

The Console and BDN outputs use the SAME `BenchmarkResult` DTO and the SAME formatter, so cells are directly comparable: pick a cell in `Console.FullBenchmark_*.LLM`, find the same cell in `Bdn.FullBenchmark_*.LLM` — deltas should agree within BDN's tighter CI.

## CLI

```
dotnet run -c Release --project AyCode.Benchmark -- <switch>
```

| Switch | Description |
|---|---|
| `--serializers` | AcBinary FastMode Byte[] vs MemoryPack Default Byte[] across 5 TestData cells (mirrors Console "F" menu / FastestByte). Emits `Bdn.FullBenchmark_*.{log,LLM,output}` + BDN-native artifacts under `BDN/`. |
| `--jitasm` | JIT disassembly analysis (x64 asm of serialize/deserialize hot path). |
| `--quick` | Quick inline benchmark (custom Stopwatch-based, not BDN). |
| `--test` / `--testmsgpack` | Quick smoke tests. |
| `--save-coverage <file>` | Save coverage file into `Test_Benchmark_Results/CoverageReport/`. |
| _(no args)_ | Interactive `BenchmarkSwitcher` — pick from all `[Benchmark]` classes in the assembly. |

## Key files

### Serializer benchmark stack (the refactor scope)
- [`AcBinaryVsMemPackBenchmark.cs`](AcBinaryVsMemPackBenchmark.cs) — BDN `[MemoryDiagnoser]` class. `[ParamsSource]`(TestData = Small/Medium/Large/Repeated/Deep) × `[Params]`(Engine = AcBinary/MemoryPack). `[GlobalSetup]` hidrátálja a Workloads scenario-ját + round-trip-verify.
- [`BdnSummaryAdapter.cs`](BdnSummaryAdapter.cs) — `Summary → List<BenchmarkResult>` translator (groups per `(TestData × Engine)`, ns → ms conversion, GcStats → allocated-bytes-per-op). Calls `BenchmarkReportWriter.PrintGroupedResults` + `SaveAll(ctx with SourceTag="Bdn", ...)`.
- [`Program.cs`](Program.cs) — BDN entry. Sets global `WithArtifactsPath(.../Benchmark/BDN)`; `--serializers` switch wires `BenchmarkRunner.Run<AcBinaryVsMemPackBenchmark>` + adapter.
- [`Workloads/Scenarios/`](Workloads/Scenarios/) — shared workload types (see folder README).
- [`Reporting/`](Reporting/) — shared reporting types (see folder README).

### KEEP benchmarks (independent — not in the serializer-refactor scope)
- [`JitDisassemblyBenchmark.cs`](JitDisassemblyBenchmark.cs) — JIT analysis: emits `.asm` files for serialize/deserialize hot paths.
- [`TaskHelperBenchmarks.cs`](TaskHelperBenchmarks.cs) — Task/timing utilities (WaitToAsync, custom ThreadPool, UtcNow.Ticks vs TickCount64).
- [`ValueTypePassingBenchmark.cs`](ValueTypePassingBenchmark.cs) — Copy-by-value vs `in` parameter for 16-byte types.
- [`RefForeachBenchmark.cs`](RefForeachBenchmark.cs) — Collection iteration patterns (array vs list, foreach vs index, ref readonly).
- [`SourceGeneratorBenchmarks.cs`](SourceGeneratorBenchmarks.cs) — Source-generated vs runtime reflection serializers (PureContractlessBenchmark, RepeatedStringBenchmark).
- [`SignalRCommunicationBenchmarks.cs`](SignalRCommunicationBenchmarks.cs) — Full-stack SignalR perf (client → server → response → round-trip).
- [`SignalRRoundTripBenchmarks.cs`](SignalRRoundTripBenchmarks.cs) — SignalR primitives/complex/collections benchmarks.

## Dependencies

| Dependency | Purpose |
|---|---|
| `BenchmarkDotNet` | BDN harness |
| `MemoryPack` | Comparison target (used by Workloads scenarios + BDN class) |
| `MessagePack` | Comparison target (KEEP benchmarks + Workloads MessagePackBenchmark scenario) |
| `MongoDB.Bson` | KEEP-side comparison target |
| `Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers` | VS Profiler integration |
| `AyCode.Core` (ProjectReference) | AcBinary serializer |
| `AyCode.Core.Tests` (ProjectReference) | Test data factory (`TestDataFactory`, `TestOrder_All_False/True`, `BenchmarkTestDataProvider*`) |
| `AyCode.Core.Serializers.SourceGenerator` (Analyzer-only) | SGen for `[AcBinarySerializable]`-tagged types |