AyCode.Core/AyCode.Benchmark

AyCode.Benchmark

BenchmarkDotNet performance suite plus the shared workload / reporting infrastructure used by both BDN and the Console runner. Targets .NET 9.

Role: dual-purpose project

This project plays two roles:

1. BDN runner Exe — standalone benchmark host (Program.cs + [Benchmark]-decorated classes). Invoke via dotnet run -c Release --project AyCode.Benchmark -- <switch>.
2. Shared workload + reporting library — exposes public types under Workloads/Scenarios/ and Reporting/ that AyCode.Core.Serializers.Console consumes via <ProjectReference>.

Both runners feed the SAME ISerializerBenchmark workload (same test data graphs, same wire options, same payload sizes) — so Console's adaptive-engine numbers and BDN's iteration-based numbers are directly comparable.

Output convention

Both runners emit a unified .log / .LLM / .output triplet to Test_Benchmark_Results/Benchmark/ (resolved at runtime via walk-up to the nearest AyCode.Core.sln — worktree-aware):

File	Source	Content
Console.FullBenchmark_<Build>_<ts>.log	Console runner	Human-readable formatted view
Console.FullBenchmark_<Build>_<ts>.LLM	Console runner	Markdown table, LLM-paste-friendly
Console.FullBenchmark_<Build>_<ts>.output	Console runner	Hex dump of Large cell binary
Bdn.FullBenchmark_<Build>_<ts>.log	BDN runner	Same format as Console
Bdn.FullBenchmark_<Build>_<ts>.LLM	BDN runner	Same
Bdn.FullBenchmark_<Build>_<ts>.output	BDN runner	Same

BDN-native artifacts (BDN's own reports, raw measurements, run logs) go to Test_Benchmark_Results/Benchmark/BDN/ — kept separate so the unified Console+BDN .log/.LLM/.output triplet stays uncluttered.

Architecture

┌────────────────────────────────────────────────────────────────┐
│ AyCode.Benchmark (this project)                                │
│                                                                 │
│  Workloads/Scenarios/    public — shared workload types        │
│    ISerializerBenchmark, BenchmarkOptions, BenchmarkEnums,     │
│    AcBinaryBenchmark<T>, MemoryPackBenchmark<T>,               │
│    AcBinaryBufferWriterBenchmark<T>, ... (12 concretes),       │
│    RoundTripValidator                                           │
│                                                                 │
│  Reporting/              public — shared reporting types       │
│    BenchmarkResult, ReportingContext, BenchmarkReportWriter    │
│                                                                 │
│  AcBinaryVsMemPackBenchmark.cs   BDN [Benchmark] class          │
│                                  (mirrors Console "F" menu)    │
│  BdnSummaryAdapter.cs            Summary → BenchmarkResult →   │
│                                  BenchmarkReportWriter.SaveAll │
│  Program.cs              BDN entry + CLI dispatch              │
│                                                                 │
│  + KEEP: JitDisassemblyBenchmark, RefForeachBenchmark,         │
│          TaskHelperBenchmarks, ValueTypePassingBenchmark,      │
│          SourceGeneratorBenchmarks,                            │
│          SignalRCommunicationBenchmarks,                       │
│          SignalRRoundTripBenchmarks                            │
└────────────────────────────────────────────────────────────────┘
                              ▲
                              │ ProjectReference (one-way)
                              │
┌────────────────────────────────────────────────────────────────┐
│ AyCode.Core.Serializers.Console                                │
│                                                                 │
│  BenchmarkLoop.cs        custom adaptive measure engine        │
│    (CPU 0 pin, High priority, phase-isolated warmup,           │
│    10-sample median + pilot, ~250ms/cell calibration)          │
│  Menu.cs / Configuration.cs / Program.cs    Console UX         │
│                                                                 │
│  Uses Benchmark's:                                             │
│   - Workloads/Scenarios/* (interface + concrete benchmarks)    │
│   - Reporting/BenchmarkReportWriter (SaveAll, Print...)        │
└────────────────────────────────────────────────────────────────┘

Two runners — same workload, different measurement engines

Aspect	Console (custom engine)	BDN
Use case	Fast iteration during micro-opt loops	Statistically confident before-commit validation
Measurement	Adaptive per-cell iter (target ~250ms), 10 samples + pilot, median	Warmup + N iterations, outlier removal, JIT-stabilized, process-spawn isolation
Time per full run	~1-3 min	~5-15 min
Noise floor	~3-5% inter-engine delta visible	~1-2%
Output format	Identical (same BenchmarkReportWriter writes both)

The Console and BDN outputs use the SAME BenchmarkResult DTO and the SAME formatter, so cells are directly comparable: pick a cell in Console.FullBenchmark_*.LLM, find the same cell in Bdn.FullBenchmark_*.LLM — deltas should agree within BDN's tighter CI.

CLI

dotnet run -c Release --project AyCode.Benchmark -- <switch>

Switch	Description
--serializers	AcBinary FastMode Byte[] vs MemoryPack Default Byte[] across 5 TestData cells (mirrors Console "F" menu / FastestByte). Emits Bdn.FullBenchmark_*.{log,LLM,output} + BDN-native artifacts under BDN/.
--jitasm	JIT disassembly analysis (x64 asm of serialize/deserialize hot path).
--quick	Quick inline benchmark (custom Stopwatch-based, not BDN).
--test / --testmsgpack	Quick smoke tests.
--save-coverage <file>	Save coverage file into Test_Benchmark_Results/CoverageReport/.
(no args)	Interactive BenchmarkSwitcher — pick from all [Benchmark] classes in the assembly.

Key files

Serializer benchmark stack (the refactor scope)

• AcBinaryVsMemPackBenchmark.cs — BDN [MemoryDiagnoser] class. [ParamsSource](TestData = Small/Medium/Large/Repeated/Deep) × [Params](Engine = AcBinary/MemoryPack). [GlobalSetup] hidrátálja a Workloads scenario-ját + round-trip-verify.
• BdnSummaryAdapter.cs — Summary → List<BenchmarkResult> translator (groups per (TestData × Engine), ns → ms conversion, GcStats → allocated-bytes-per-op). Calls BenchmarkReportWriter.PrintGroupedResults + SaveAll(ctx with SourceTag="Bdn", ...).
• Program.cs — BDN entry. Sets global WithArtifactsPath(.../Benchmark/BDN); --serializers switch wires BenchmarkRunner.Run<AcBinaryVsMemPackBenchmark> + adapter.
• Workloads/Scenarios/ — shared workload types (see folder README).
• Reporting/ — shared reporting types (see folder README).

KEEP benchmarks (independent — not in the serializer-refactor scope)

• JitDisassemblyBenchmark.cs — JIT analysis: emits .asm files for serialize/deserialize hot paths.
• TaskHelperBenchmarks.cs — Task/timing utilities (WaitToAsync, custom ThreadPool, UtcNow.Ticks vs TickCount64).
• ValueTypePassingBenchmark.cs — Copy-by-value vs in parameter for 16-byte types.
• RefForeachBenchmark.cs — Collection iteration patterns (array vs list, foreach vs index, ref readonly).
• SourceGeneratorBenchmarks.cs — Source-generated vs runtime reflection serializers (PureContractlessBenchmark, RepeatedStringBenchmark).
• SignalRCommunicationBenchmarks.cs — Full-stack SignalR perf (client → server → response → round-trip).
• SignalRRoundTripBenchmarks.cs — SignalR primitives/complex/collections benchmarks.

Dependencies

Dependency	Purpose
BenchmarkDotNet	BDN harness
MemoryPack	Comparison target (used by Workloads scenarios + BDN class)
MessagePack	Comparison target (KEEP benchmarks + Workloads MessagePackBenchmark scenario)
MongoDB.Bson	KEEP-side comparison target
Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers	VS Profiler integration
AyCode.Core (ProjectReference)	AcBinary serializer
AyCode.Core.Tests (ProjectReference)	Test data factory (TestDataFactory, TestOrder_All_False/True, BenchmarkTestDataProvider*)
AyCode.Core.Serializers.SourceGenerator (Analyzer-only)	SGen for [AcBinarySerializable]-tagged types